- Machine Learning Basics
- Machine Learning - Home
- Machine Learning - Getting Started
- Machine Learning - Basic Concepts
- Machine Learning - Python Libraries
- Machine Learning - Applications
- Machine Learning - Life Cycle
- Machine Learning - Required Skills
- Machine Learning - Implementation
- Machine Learning - Challenges & Common Issues
- Machine Learning - Limitations
- Machine Learning - Reallife Examples
- Machine Learning - Data Structure
- Machine Learning - Mathematics
- Machine Learning - Artificial Intelligence
- Machine Learning - Neural Networks
- Machine Learning - Deep Learning
- Machine Learning - Getting Datasets
- Machine Learning - Categorical Data
- Machine Learning - Data Loading
- Machine Learning - Data Understanding
- Machine Learning - Data Preparation
- Machine Learning - Models
- Machine Learning - Supervised
- Machine Learning - Unsupervised
- Machine Learning - Semi-supervised
- Machine Learning - Reinforcement
- Machine Learning - Supervised vs. Unsupervised
- Machine Learning Data Visualization
- Machine Learning - Data Visualization
- Machine Learning - Histograms
- Machine Learning - Density Plots
- Machine Learning - Box and Whisker Plots
- Machine Learning - Correlation Matrix Plots
- Machine Learning - Scatter Matrix Plots
- Statistics for Machine Learning
- Machine Learning - Statistics
- Machine Learning - Mean, Median, Mode
- Machine Learning - Standard Deviation
- Machine Learning - Percentiles
- Machine Learning - Data Distribution
- Machine Learning - Skewness and Kurtosis
- Machine Learning - Bias and Variance
- Machine Learning - Hypothesis
- Regression Analysis In ML
- Machine Learning - Regression Analysis
- Machine Learning - Linear Regression
- Machine Learning - Simple Linear Regression
- Machine Learning - Multiple Linear Regression
- Machine Learning - Polynomial Regression
- Classification Algorithms In ML
- Machine Learning - Classification Algorithms
- Machine Learning - Logistic Regression
- Machine Learning - K-Nearest Neighbors (KNN)
- Machine Learning - Naïve Bayes Algorithm
- Machine Learning - Decision Tree Algorithm
- Machine Learning - Support Vector Machine
- Machine Learning - Random Forest
- Machine Learning - Confusion Matrix
- Machine Learning - Stochastic Gradient Descent
- Clustering Algorithms In ML
- Machine Learning - Clustering Algorithms
- Machine Learning - Centroid-Based Clustering
- Machine Learning - K-Means Clustering
- Machine Learning - K-Medoids Clustering
- Machine Learning - Mean-Shift Clustering
- Machine Learning - Hierarchical Clustering
- Machine Learning - Density-Based Clustering
- Machine Learning - DBSCAN Clustering
- Machine Learning - OPTICS Clustering
- Machine Learning - HDBSCAN Clustering
- Machine Learning - BIRCH Clustering
- Machine Learning - Affinity Propagation
- Machine Learning - Distribution-Based Clustering
- Machine Learning - Agglomerative Clustering
- Dimensionality Reduction In ML
- Machine Learning - Dimensionality Reduction
- Machine Learning - Feature Selection
- Machine Learning - Feature Extraction
- Machine Learning - Backward Elimination
- Machine Learning - Forward Feature Construction
- Machine Learning - High Correlation Filter
- Machine Learning - Low Variance Filter
- Machine Learning - Missing Values Ratio
- Machine Learning - Principal Component Analysis
- Machine Learning Miscellaneous
- Machine Learning - Performance Metrics
- Machine Learning - Automatic Workflows
- Machine Learning - Boost Model Performance
- Machine Learning - Gradient Boosting
- Machine Learning - Bootstrap Aggregation (Bagging)
- Machine Learning - Cross Validation
- Machine Learning - AUC-ROC Curve
- Machine Learning - Grid Search
- Machine Learning - Data Scaling
- Machine Learning - Train and Test
- Machine Learning - Association Rules
- Machine Learning - Apriori Algorithm
- Machine Learning - Gaussian Discriminant Analysis
- Machine Learning - Cost Function
- Machine Learning - Bayes Theorem
- Machine Learning - Precision and Recall
- Machine Learning - Adversarial
- Machine Learning - Stacking
- Machine Learning - Epoch
- Machine Learning - Perceptron
- Machine Learning - Regularization
- Machine Learning - Overfitting
- Machine Learning - P-value
- Machine Learning - Entropy
- Machine Learning - MLOps
- Machine Learning - Data Leakage
- Machine Learning - Resources
- Machine Learning - Quick Guide
- Machine Learning - Useful Resources
- Machine Learning - Discussion
Machine Learning - Cross Validation
Cross-validation is a powerful technique used in machine learning to estimate the performance of a model on unseen data. It is an essential step in building a robust machine learning model, as it helps to identify overfitting or underfitting, and helps to determine the optimal model hyperparameters.
What is Cross-Validation?
Cross-validation is a technique used to evaluate the performance of a model by partitioning the dataset into subsets, training the model on a portion of the data, and then validating the model on the remaining data. The basic idea behind cross-validation is to use a subset of the data to train the model and another subset to test its performance. This allows the machine learning model to be trained on a variety of data and to generalize better to new data.
There are different types of cross-validation techniques available, but the most commonly used technique is k-fold cross-validation. In k-fold cross-validation, the data is partitioned into k equally sized folds. The model is then trained on k-1 folds and tested on the remaining fold. This process is repeated k times, with each of the k folds used once as the validation data. The final performance of the model is then averaged over the k iterations to obtain an estimate of the model's performance.
Why is Cross-Validation Important?
Cross-validation is an essential technique in machine learning because it helps to prevent overfitting or underfitting of a model. Overfitting occurs when the model is too complex and fits the training data too closely, resulting in poor performance on new data. On the other hand, underfitting occurs when the model is too simple and does not capture the underlying patterns in the data, resulting in poor performance on both the training and test data.
Cross-validation also helps to determine the optimal model hyperparameters. Hyperparameters are the settings that control the behavior of the model. For example, in a decision tree algorithm, the maximum depth of the tree is a hyperparameter that determines the level of complexity of the model. By using cross-validation to evaluate the performance of the model at different hyperparameter values, we can select the optimal hyperparameters that maximize the model's performance.
Implementing Cross-Validation in Python
In this section, we will discuss how to implement k-fold cross-validation in Python using the Scikit-learn library. Scikit-learn is a popular Python library for machine learning that provides a range of algorithms and tools for data preprocessing, model selection, and evaluation.
To demonstrate how to implement cross-validation in Python, we will use the famous Iris dataset. The Iris dataset contains measurements of the sepal length, sepal width, petal length, and petal width of three different species of iris flowers. The goal is to build a model that can predict the species of an iris flower based on its measurements.
First, we need to load the dataset using the Scikit-learn load_iris() function and split it into a training set and a test set using the train_test_split() function. The training set will be used to train the model, and the test set will be used to evaluate the performance of the model.
from sklearn.datasets import load_iris from sklearn.model_selection import train_test_split # Load the Iris dataset iris = load_iris() # Split the data into a training set and a test set X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)
Next, we will create a decision tree classifier using the Scikit-learn DecisionTreeClassifier() function.
from sklearn.tree import DecisionTree
Create a decision tree classifier.
clf = DecisionTreeClassifier(random_state=42)
Now, we can use k-fold cross-validation to evaluate the performance of the model. We will use the cross_val_score() function from Scikit-learn to perform k-fold cross-validation. The function takes as input the model, the training data, the target variable, and the number of folds. It returns an array of scores, one for each fold.
from sklearn.model_selection import cross_val_score # Perform k-fold cross-validation scores = cross_val_score(clf, X_train, y_train, cv=5)
Here, we have specified the number of folds as 5, meaning that the data will be partitioned into 5 equally sized folds. The cross_val_score() function will train the model on 4 folds and test it on the remaining fold. This process will be repeated 5 times, with each fold used once as the validation data. The function returns an array of scores, one for each fold.
Finally, we can calculate the mean and standard deviation of the scores to get an estimate of the model's performance.
import numpy as np
# Calculate the mean and standard deviation of the scores
mean_score = np.mean(scores)
std_score = np.std(scores)
print("Mean cross-validation score: {:.2f}".format(mean_score))
print("Standard deviation of cross-validation score: {:.2f}".format(std_score))
The output of this code will be the mean and standard deviation of the scores. The mean score represents the average performance of the model across all folds, while the standard deviation represents the variability of the scores.
Example
Here is the complete implementation of Cross-Validation in Python −
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import cross_val_score
import numpy as np
# Load the iris dataset
iris = load_iris()
# Define the features and target variables
X = iris.data
y = iris.target
# Create a decision tree classifier
clf = DecisionTreeClassifier(random_state=42)
# Perform k-fold cross-validation
scores = cross_val_score(clf, X, y, cv=5)
# Calculate the mean and standard deviation of the scores
mean_score = np.mean(scores)
std_score = np.std(scores)
print("Mean cross-validation score: {:.2f}".format(mean_score))
print("Standard deviation of cross-validation score: {:.2f}".format(std_score))
Output
When you execute this code, it will produce the following output −
Mean cross-validation score: 0.95 Standard deviation of cross-validation score: 0.03