Machine Learning - Naive Bayes Algorithm

Previous
Next

The Naive Bayes algorithm is a classification algorithm based on Bayes' theorem. The algorithm assumes that the features are independent of each other, which is why it is called "naive." It calculates the probability of a sample belonging to a particular class based on the probabilities of its features. For example, a phone may be considered as smart if it has touch-screen, internet facility, good camera, etc. Even if all these features are dependent on each other, but all these features independently contribute to the probability of that the phone is a smart phone.

In Bayesian classification, the main interest is to find the posterior probabilities i.e. the probability of a label given some observed features, P(𝐿L | features). With the help of Bayes theorem, we can express this in quantitative form as follows −

$$P\left ( L| features\right )=\frac{P\left ( L \right )P\left (features| L\right )}{P\left (features\right )}$$

Here,

  • $P\left ( L| features\right )$ is the posterior probability of class.

  • $P\left ( L \right )$ is the prior probability of class.

  • $P\left (features| L\right )$ is the likelihood which is the probability of predictor given class.

  • $P\left (features\right )$ is the prior probability of predictor.

In the Naive Bayes algorithm, we use Bayes' theorem to calculate the probability of a sample belonging to a particular class. We calculate the probability of each feature of the sample given the class and multiply them to get the likelihood of the sample belonging to the class. We then multiply the likelihood with the prior probability of the class to get the posterior probability of the sample belonging to the class. We repeat this process for each class and choose the class with the highest probability as the class of the sample.

Types of Naive Bayes Algorithm

There are three types of Naive Bayes algorithm −

  • Gaussian Naive Bayes − This algorithm is used when the features are continuous variables that follow a normal distribution. It assumes that the probability distribution of each feature is Gaussian, which means it is a bell-shaped curve.

  • Multinomial Naive Bayes − This algorithm is used when the features are discrete variables. It is commonly used in text classification tasks where the features are the frequency of words in a document.

  • Bernoulli Naive Bayes − This algorithm is used when the features are binary variables. It is also commonly used in text classification tasks where the features are whether a word is present or not in a document.

Implementation in Python

Here we will implement the Gaussian Naive Bayes algorithm in Python. We will use the iris dataset, which is a popular dataset for classification tasks. It contains 150 samples of iris flowers, each with four features: sepal length, sepal width, petal length, and petal width. The flowers belong to three classes: setosa, versicolor, and virginica.

First, we will import the necessary libraries and load the datase −

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB

# load the iris dataset
iris = load_iris()

# split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(iris.data,
iris.target, test_size=0.35, random_state=0)

We then create an instance of the Gaussian Naive Bayes classifier and train it on the training set −

# Create a Gaussian Naive Bayes classifier
gnb = GaussianNB()

#fit the classifier to the training data:
gnb.fit(X_train, y_train)

We can now use the trained classifier to make predictions on the testing set −

#make predictions on the testing data
y_pred = gnb.predict(X_test)

We can evaluate the performance of the classifier by calculating its accuracy −

#Calculate the accuracy of the classifier
accuracy = np.sum(y_pred == y_test) / len(y_test) print("Accuracy:", accuracy)

Complete Implementation Example

Given below is the complete implementation example of Naïve Bayes Classification algorithm in python using the iris dataset −

import numpy as np
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB

# load the iris dataset
iris = load_iris()

# split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(iris.data,
iris.target, test_size=0.35, random_state=0)

# Create a Gaussian Naive Bayes classifier
gnb = GaussianNB()

#fit the classifier to the training data:
gnb.fit(X_train, y_train)

#make predictions on the testing data
y_pred = gnb.predict(X_test)

#Calculate the accuracy of the classifier
accuracy = np.sum(y_pred == y_test) / len(y_test)
print("Accuracy:", accuracy)

Output

When you execute this program, it will produce the following output −

Accuracy: 0.9622641509433962
Print Page
Previous
Next
Advertisements