Feature Selection means figuring out which signals you can use to identify patterns, and then integrate them into your training and scoring pipeline. In this article, I’ll walk you through what feature selection is and how it affects the formation of our machine learning models.
What is Feature Selection?
If you have an efficient machine learning infrastructure, then most of your time and energy will be wasted on selecting the best features. To get the most out of your efforts, you only want to use features that offer high discriminating power; adding each feature should significantly improve your model.
Also, Read – Machine Learning Full Course for free.
In addition to requiring additional effort for construction and maintenance, redundant features can adversely affect the quality of your model. If the number of features is greater than the number of data points, your model will be overfitted: there are enough model parameters to draw a curve through all of the training data.
Additionally, highly correlated characteristics can cause instability in model decisions. For example, if you have a feature that is “number of connections yesterday” and another that is “number of connections in the last two days”, the information you are trying to collect will be split between the two features in a way essentially arbitrary, and the model might not learn that either of these characteristics is important.
How To Select Features?
You can solve the feature selection problem by calculating covariance matrices between your features and combining highly correlated features. There are several techniques to resolve the feature selection issue:
Logistic regression, SVMs, and decision trees/forests have methods for determining the relative importance of features; you can run them and keep only the most important features.
You can use L1 regularization for the selection of characteristics in logistic regression and SVM classifiers.
If the number n of entities is reasonably small (say, n <100), you can use a “construction” approach: build n single-feature models and determine which is best on your validation set; then build n – 1 two-feature model, and so on, until the gain from adding additional functionality is below a certain threshold.
Likewise, you can use a “leave out” approach: build a model on n features, then n models on n – 1 feature and keep the best ones, and so on until the loss of deletion of d. ‘extra functionality is too important.
I hope you liked this article on what is feature selection in machine learning. You can learn about its practical implementation from here. Feel free to ask your valuable questions in the comments section below.
