Main Challenges of Machine Learning
- 🧑🏫 Splunk Machine Learning Toolkit - Introduction & Demo | Somerford
- 📘 Hands-on Machine Learning with Scikit-learn, Keras & Tensorflow | Aurélien Géron
Irrelevant Features: Red tie example
Say we want to train a model to predict how much is gonna cost to travel in the UK. Some common metrics are, start date, end date, the distance, if you wanna go in bus, car, train and so on. But if i start to add useless metrics like if i have a blue or red tie, if i have a watch and so on, these useless metrics will impact negatively on the accuracy of the model.
Key Lesson
Use only data that is relevant to the question you want answered
Selecting less than optimal algorithm
Key Lesson
Different classifier algorithms generates false predictions and have more/less accuracy. Try different algorithms and pick the one that generates better results.
Insufficient Quantity of Training Data
For a toddler to learn what an apple is, all it takes is for you to point to an apple and say “apple” (possibly repeating this procedure a few times). Now the child is able to recognize apples in all sorts of colors and shapes. Genius. Machine Learning is not quite there yet; it takes a lot of data for most Machine Learning algorithms
- Hands-on Machine Learning with Scikit-learn, Keras & Tensorflow | Aurélien Géron | Ch 1
In this paper, researchers showed that very different Machine Learning Algorithms, given enough data, performed almost identically. So, a key idea is that data quality and size is very important. But small and medium-sized datasets are still very common, and it’s not always easy or cheap to get extra training data, so don’t abandon algorithm just yet.
Nonrepresentative Training Data
- Sampling Noise → when the sample is too small
- Sampling Bias → when the sampling methos is flawed, so even a large dataset can be nonrepresentative.
Poor-quality data
- A significant part of time of a data scientist is spent on training data.
- Data Quality
If some instances are clearly outliers, it may help to simply discard them or try to fix the errors manually.
Some decision must be done if some instances are missing a few features. Like, if your customers did not specify their age, then you must decide if this attribute must be either ignored, filled with median age or train one model with the feature and one without, and so on.
Overfitting the training data
Overfitting happens when a model learned too well on the training dataset but has a low accuracy on the test set. Here are some solutions:
- Simplify the model by selecting one with fewer parameters, by reducing the number of attributes in the training data or by constraining the model
- To gather more training data
- To reduce the noise in the training data (e.g, fix data errors and remove outliers).
Underfitting the training data
It’s the opposite of overfitting: it occurs when the model is too simple to learn the underlying structure of data.