Here’s How Logistic Regression Algorithm Works

In Machine Learning, Logistic Regression is a statistical model used for binary classification problems. It is used to predict the probability of an outcome based on the input features. It uses a sigmoid function to map the input features to output the probability. So, if you are new to Machine Learning and want to know how the Logistic Regression algorithm works, this article is for you. In this article, I will introduce how the Logistic Regression algorithm works in simple words and its implementation using Python.

Here’s How Logistic Regression Algorithm Works

Let’s understand how the Logistic Regression algorithm works by taking an example of a real-time business problem.

Imagine you are a bank that wants to decide whether or not to approve a loan for a customer. You want to use the customer’s credit score as a factor in your decision.

To predict whether to approve a loan, you can use Logistic Regression. You’ll start by looking at historical data to see how credit scores relate to loan approvals. For example, you can look at the data for the past year and notice that customers with high credit scores were generally approved for loans, while customers with low credit scores were disapproved.

So, in this case, the logistic regression model would take the customer’s credit score as an input feature and produce a probability of whether the loan should be approved or not.

For example, suppose the model predicts a 90% chance that the loan will be approved based on the customer’s credit score. You can then use this prediction to make decisions for your bank.

This is how the logistic regression algorithm works. It helps you predict whether an event will happen or not based on input features that you think are relevant to the event.

Implementation of Logistic Regression Algorithm using Python

Now let’s see how to implement the logistic regression algorithm using Python. To implement it using Python, we can use the scikit-learn library in Python, which provides the functionality of implementing all Machine Learning algorithms and concepts using Python.

Let’s first import the necessary Python libraries and create a sample data based on the example we discussed above:

import pandas as pd
import numpy as np

#sample dataset
credit_scores = np.random.randint(300, 850, size=1000)
loan_approved = np.random.binomial(1, p=1 / (1 + np.exp(-0.02 * (credit_scores - 700))), size=1000)

data = pd.DataFrame({
    'credit_score': credit_scores,
    'loan_approved': loan_approved
})

print(data.head())

   credit_score  loan_approved
0           495              0
1           816              1
2           741              1
3           625              0
4           651              0

Now here’s how to train a Machine Learning model using the logistic regression algorithm:

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(data[['credit_score']], data['loan_approved'])

Now here’s how we can make a prediction using the Logistic Regression algorithm:

new_customer_credit_score = 750
new_customer_data = pd.DataFrame({'credit_score': [new_customer_credit_score]})
predicted_class = model.predict(new_customer_data[['credit_score']])[0]

# print the predicted class
if predicted_class == 0:
    print(f'A loan is not approved for a customer with a credit score of {new_customer_credit_score}.')
else:
    print(f'A loan is approved for a customer with a credit score of {new_customer_credit_score}.')

A loan is approved for a customer with a credit score of 750.

So this is how the Logistic Regression Algorithm works.

Advantages & Disadvantages of Logistic Regression

Here are some advantages and disadvantages of the Logistic Regression algorithm that you should know:

Advantages:

1. It works well for binary classification problems, where the output variable has only two possible values.
2. It can handle both continuous and categorical input variables.

Disadvantages:

1. Logistic Regression assumes that the relationship between the input variable and the output variable is linear, which may not be true in all cases.
2. It can be sensitive to outliers or biased data, which affects predictions.

Summary

I hope you have understood how the Logistic Regression algorithm works and how to implement it using Python. Logistic Regression is used to predict the probability of an outcome based on the input features. It uses a sigmoid function to map the input features to output the probability. Feel free to ask valuable questions in the comments section below.