Unsupervised learning
The data is only a sample of objects without associated target values, the problem is known as unsupervised learning. In unsupervised learning there is no teacher. Hence a concise description of the data could be a set of clusters or a probability density stating how likely it is to observe a certain object in the future.
Example of unsupervised learning: Image segmentation
Given a training sample of objects (like images), the aim is to extract a structure from them. For instance, identifying indoor or outdoor images or extracting face pixels in an image.
Training data is used to extract a structure from the data objects. If it exists, then it can take advantage of the redundancy and find a short description of data. A general way to represent data is to specify a similarity between any pairs of objects. If two objects share much structure, it should be possible to reproduce the data from the same prototype.
This idea underlies clustering algorithms that form a rich subclass of unsupervised algorithms.
Clustering Algorithms
This class of algorithms are based on this idea: Given a fixed number of clusters, we aim to find a grouping of the objects such that similar objects belong to the same cluster.
If it is possible to find a clustering such that the similarities of the objects in one cluster are much greater than the similarities among objects from different clusters, we have extracted structure from training sample so that the whole cluster can be represented by one representative data point.
Dimensionality Reduction Methods
DRM are techniques whose aim is to represent high-dimensionality data in low dimension spaces, trying to preserve the original information of data.
These techniques are important for the following reason: using more dimension that the strictly necessary leads to several problems:
- The space needed to store the data. Compression for storage purposes is important in case you have large datasets
- The speed of algorithms depends on the dimension of the vectors, so a reduction of the dimension can result in reduced computation time.
- Curse of dimensionality: it can be hard to make reliable classifiers when the dimensionality of input data is high.