Independent Component Analysis (ICA) is one of the alternatives of PCA that is used to find the underlying factors or components from a multivariate statistical dataset. This is different from a standard PCA because it looks for components that are statistically independent and uncorrelated. If you don’t know what ICA is in machine learning, this article is for you. In this article, I’ll give you an introduction to Independent Component Analysis in machine learning and its implementation using Python.
What is Independent Component Analysis?
Sometimes it is very useful to process a dataset to extract independent and uncorrelated components from it. Carrying out such a task using a standard PCA is very difficult because in a standard PCA there are no constraints on the independence of the components.
This is where ICA comes in, which is used to extract the independent components from a dataset. It is based on two main assumptions:
- The Independent Components must have a non-Gaussian distribution
- The mixing matrix is invisible
The first assumption is very important because there is a very strong link between Gaussianity and independence. If a dataset satisfies these assumptions, then it is possible to estimate the independent components by using the ICA.
Independent Component Analysis using Python
To implement Independent Component Analysis using Python, we first need a dataset whose values are centred on zero. This means that the average of the values in a dataset should be zero. So let’s start by importing a dataset and centring its values to zero:
(1797, 64)
Now the next step is to use the ICA algorithm on this data to find the independent components. We can use the FastICA class provided by the scikit-learn library to implement ICA using Python:
[[ 0. -0.30383973 -2.20478575 ... 0.2359488 -2.06789093 -0.36449638] [ 0. 4.69616027 10.79521425 ... -6.7640512 -2.06789093 -0.36449638] [ 0. -0.30383973 2.79521425 ... 0.2359488 -2.06789093 -0.36449638] ... [ 0. -0.30383973 5.79521425 ... -5.7640512 -2.06789093 -0.36449638] [ 0. 0.69616027 2.79521425 ... 0.2359488 -2.06789093 -0.36449638] [ 0. -0.30383973 4.79521425 ... -0.7640512 -2.06789093 -0.36449638]]
The resulting components are always independent, and you can also rebuild a sample dataset from these values as a weighted sum of them.
Summary
So this is how we can implement ICA in machine learning by using the Python programming language. ICA is different from a standard PCA because it looks for components that are statistically independent and uncorrelated. I hope you liked this article on what is Independent Component Analysis in Machine Learning and its implementation using Python. Feel free to ask your valuable questions in the comments section below.
