Linear Regression Algorithm without Scikit-Learn

In this article, I will teach you how you can easily create your algorithms instead of using any package like Scikit-Learn provided with Python. I will create a Linear Regression Algorithm using mathematical equations, and I will not use Scikit-Learn in this task.

If you are using Scikit-Learn, you can easily use a lot of algorithms that are already made by some famous Researchers, Data Scientists, and other Machine Learning experts. Have you ever thought of building your algorithm instead of using a module like Scikit-Learn? All the Machine Learning Algorithms that Scikit-Learn provides are easy to use but to be a Machine Learning Expert in a brand like Google and Microsoft, you need to build your algorithms instead of using any package so that you could easily create an algorithm according to your needs.

The role of a Data Scientist and a Machine Learning Expert are not just to fit a model and training and testing. These are only the basic stuff that you need to know. Without these, you cannot be called as a practitioner in Machine Learning. But if you started to build your algorithms, it will make you an ideal Expert of all.

Also Read: Audio Processing with Python.

What is a Linear Regression Algorithm?

A Linear Regression algorithm makes a prediction by simply computing a weighted sum of the input features, plus a constant called the bias term. In mathematics a linear regression algorithm looks like:

Linear Regression algorithm
Linear Regression

Linear Regression Algorithm without Scikit-Learn

Let’s create our own linear regression algorithm, I will first create this algorithm using the mathematical equation. Then I will visualize our algorithm using the Matplotlib module in Python. I will only use the NumPy module in Python to build our algorithm because NumPy is used in all the mathematical computations in Python. I will start here by creating linear-looking data so that I can use that data in creating my Linear Regression Algorithm:

import numpy as np X = 2 * np.random.rand(100, 1) y = 4 + 3 * X + np.random.randn(100, 1)

Before moving forward let’s visualize this data:

import matplotlib as mpl import matplotlib.pyplot as plt plt.plot(X, y, "b.") plt.xlabel("$x_1$", fontsize=18) plt.ylabel("$y$", rotation=0, fontsize=18) plt.axis([0, 2, 0, 15])

Now, let’s move forward by creating a Linear regression mathematical algorithm. I will use the inv() function from NumPy’s linear algebra module (np.linalg) to compute the inverse of the matrix, and the dot() method for matrix multiplication:

X_b = np.c_[np.ones((100, 1)), X] # add x0 = 1 to each instance linear = np.linalg.inv(

The function that we used to generate the data is y = 3xi + Gaussian noise. Let’s see what our algorithm found:


That’s looks good as a linear regression model. Now let’s make predictions using our algorithm:

X_new = np.array([[0], [2]]) X_new_b = np.c_[np.ones((2, 1)), X_new] # add x0 = 1 to each instance y_predict = y_predict

Now, let’s plot the predictions of our linear regression:

plt.plot(X_new, y_predict, "r-") plt.plot(X, y, "b.") plt.axis([0, 2, 0, 15])
Linear Regression algorithm

Now let’s use the same model with the linear regression algorithm, which is provided with Scikit-Learn.

Linear Regression with Scikit-Learn

You saw above how we can create our own algorithm, you can practice creating your own algorithm by creating an algorithm which is already existing. So that you can evaluate your algorithm using the already existing algorithm. Like here I will cross-check the linear regressing algorithm that I made with the algorithm that Scikit-Learn provides. The results of my algorithm were:


Now, let’s see what results we get from the scikit-learn linear regression model:

from sklearn.linear_model import LinearRegression lin_reg = LinearRegression(), y) lin_reg.intercept_, lin_reg.coef_ lin_reg.predict(X_new)

Also Read: 10 Machine Learning Projects to Boost your Portfolio.

So, as you can see we got the same results from both the algorithms. I hope you liked this article. Feel free to ask your valuable questions in the comments section below.

Follow Us:

Default image
Aman Kharwal

I am a programmer from India, and I am here to guide you with Data Science, Machine Learning, Python, and C++ for free. I hope you will learn a lot in your journey towards Coding, Machine Learning and Artificial Intelligence with me.

Leave a Reply