{"id":690,"date":"2020-08-25T14:08:08","date_gmt":"2020-08-25T14:08:08","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/08\/25\/getting-started-with-feature-selection\/"},"modified":"2020-08-25T14:08:08","modified_gmt":"2020-08-25T14:08:08","slug":"getting-started-with-feature-selection","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/08\/25\/getting-started-with-feature-selection\/","title":{"rendered":"Getting Started with Feature Selection"},"content":{"rendered":"
\n

By Kurtis Pykes<\/a>, AI Writer<\/b>.<\/p>\n

You wouldn\u2019t use the amount of press-ups you can do to determine the bus arrival time would you? In the same way, in predictive modelling, we prune away the non-useful features in order to reduce the complexity of the final model. Simply put, Feature selection reduces the number of input features when developing a predictive model.<\/p>\n

In this article, I discuss the 3 main categories that feature selection falls into; filter methods, wrapper methods, and embedded methods. Additionally, I use Python examples and leverage frameworks such as scikit-learn (see the\u00a0Documentation<\/a>) for Machine learning, Pandas (Documentation<\/a>) for data manipulation, and Plotly (Documentation<\/a>) for interactive data visualization. For access to the code used in this article, visit my GitHub<\/a>.<\/p>\n

<\/p>\n

Figure 1: Clothing Rack Photo by\u00a0Zui Hoang<\/a>\u00a0on\u00a0Unsplash<\/a>.<\/em><\/p>\n

\u00a0<\/p>\n

Why do Feature Selection?<\/h3>\n

\u00a0<\/p>\n

The message that the first paragraph aimed to convey is that sometimes there are features that do not contribute enough useful information to predicting the final outcome, so by including it in our model, we are making our model unneededly more complex. Discarding of non-useful features results in a parsimonious model, which in turn leads to a reduced scoring time. Additionally, Feature Selection also makes interpreting models much easier, which is extremely important in most business cases.<\/p>\n

\n

\u201cIn most real-world cases, applying feature selection is unlikely to provide large gains in performance. However, it is still a valuable tool in the toolbox of the feature engineer.\u201d \u2014<\/em> (M\u00fcller, (2016), Introduction to Machine Learning with Python, O\u2019Reilley Media )<\/p>\n<\/blockquote>\n

Methods<\/strong><\/p>\n

There are various methods that could be used to perform Feature Selection, of which they fall into one of 3 categories. Each method has its own advantages and disadvantages. The categories are described in Guyon & Elisseeff (2003) as follows:<\/p>\n