Cross-validation
- 📖 Cross-Validation: evaluating estimator performance
- 📘 Hands-on Machine Learning with Scikit-learn, Keras & Tensorflow | Aurélien Géron
- 🟢 Machine Learning Fundamentals: Cross Validation | StatQuest with Josh Starmer
Cross-validation is a technique used to measure the generalization accuracy of a model.
How does it work? Imagine you have a column that represent your data, and you divide this data in two block: the training set that consists of the first 75% and the test set that consists of the last 25%. This approach is good, but what if we use the first 25% for test and the last 75% for training, or middle blocks? Cross Validation uses them all, one at time, and summarize the results at the end (for example calculating the mean of all the results)
Cross Validation can also be used to find the best hyperparamether for an algorithm.
K-Fold
K-fold is the basic approach of cross-validation where the dataset is split into K smaller sets (folds). For each fold:
- A model is trained using k-1 of the folds training data
- The resulting model is validated on the remaining part of the data
The performance is calculated as the average of the values computed in the loop.
- 🟢 Does not waste much data
- 🟢 Good for dataset so large that doesn’t fit into memory
- 🟢 Avoid bias
- 🔴 Computationally expensive: the training time is k-times longer
Stratified K-Fold
- ✍Dealing with imbalanced data: undersampling, oversampling and proper cross-validation | Marco Altini
- sklearn.model_selection.StratifiedKfold() | scikit-learn.org
Is the same as Kfold, but on data split, this technique preserves the percentage of samples for each class. It is used when the percentage of each class in train & test is imbalanced.