{"id":8054,"date":"2021-01-11T03:55:59","date_gmt":"2021-01-11T03:55:59","guid":{"rendered":"https:\/\/wealthrevelation.com\/data-science\/2021\/01\/11\/five-key-assumptions-of-linear-regression-algorithm\/"},"modified":"2021-01-11T03:55:59","modified_gmt":"2021-01-11T03:55:59","slug":"five-key-assumptions-of-linear-regression-algorithm","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2021\/01\/11\/five-key-assumptions-of-linear-regression-algorithm\/","title":{"rendered":"Five Key Assumptions of Linear Regression Algorithm"},"content":{"rendered":"<div id=\"tve_editor\" data-post-id=\"8294\">\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b272c8b0\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/1-Linear-Regression-Assumptions.png?resize=626%2C376&amp;ssl=1\" class=\"tve_image wp-image-8296\" alt=\"Linear Regression Assumptions\" data-id=\"8296\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Linear Regression Assumptions\" loading=\"lazy\" data-width=\"626\" data-height=\"376\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8296\" alt=\"Linear Regression Assumptions\" data-id=\"8296\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Linear Regression Assumptions\" loading=\"lazy\" src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/1-Linear-Regression-Assumptions.png?resize=626%2C376&amp;ssl=1\" data-width=\"626\" data-height=\"376\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element tve-froala fr-box fr-basic\" data-css=\"tve-u-176b272c8ba\">\n<p dir=\"ltr\">\u00a0 Nearly <strong>80%<\/strong> of the people <a href=\"https:\/\/dataaspirant.com\/linear-regression-implementation-in-python\/\" target=\"_blank\" class=\"tve-froala\" rel=\"noopener\"><strong>build linear regression models<\/strong><\/a> without checking the basic assumptions of linear regression.<\/p>\n<p dir=\"ltr\">Just hold for a second and think. How many times have you built linear regression models without checking the <a href=\"https:\/\/dataaspirant.com\/linear-regression\/\" target=\"_blank\" rel=\"noopener\"><strong>linear regression<\/strong><\/a> assumptions?<\/p>\n<p dir=\"ltr\">If you are not aware about the linear regression algorithm. It is a famous <a href=\"https:\/\/dataaspirant.com\/supervised-and-unsupervised-learning\/\" target=\"_blank\" class=\"tve-froala\" rel=\"noopener\"><strong>s<\/strong><strong>upervised machine learning algorithm<\/strong><\/a> that represents the linear relationship between a <strong>dependent<\/strong> variable and <strong>independent<\/strong> variables.<\/p>\n<p dir=\"ltr\">It is easy to understand and implement. However, just writing a few lines of code won\u2019t work as expected.<\/p>\n<p dir=\"ltr\">Because before implementing the linear regression, we have to take care of certain assumptions made by linear regression.\u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper thrv_tw_qs tve_clearfix\" data-url=\"https:\/\/twitter.com\/intent\/tweet\" data-via=\"\" data-use_custom_url=\"\" data-css=\"tve-u-176b272c8f8\">\n<div class=\"thrv_tw_qs_container\">\n<div class=\"thrv_tw_quote\">\n<p class=\"\">Learn the 5 key linear regression assumptions, we need to consider before building the regression model. #datascience #machinelearning #ai #regression #python<\/p>\n<\/p><\/div>\n<p>\n\t\t\t<span><br \/>\n\t\t\t\t<i><\/i><br \/>\n\t\t\t\t<span class=\"thrv_tw_qs_button_text  thrv-inline-text tve_editable\">Click to Tweet<\/span><br \/>\n\t\t\t<\/span>\n\t\t<\/p>\n<\/p><\/div>\n<\/div>\n<div class=\"thrv_wrapper thrv_text_element\" data-css=\"tve-u-176b272c8fa\">\n<p dir=\"ltr\">It is important to understand these assumptions to improve the <a href=\"https:\/\/dataaspirant.com\/difference-between-r-squared-and-adjusted-r-squared\/\" target=\"_blank\" rel=\"noopener\"><strong>regression model\u2019s performance<\/strong><\/a>.\u00a0<\/p>\n<p dir=\"ltr\">So In this article, we are going to discuss these assumptions in-depth and ways to <strong>fix them if violated<\/strong>. After gaining proper knowledge of linear regression assumptions, you can bring excessive improvement in regression models.\u00a0<\/p>\n<p dir=\"ltr\">Before we dive further, let\u2019s look at the topic you are going to learn in this article.<\/p>\n<\/div>\n<div class=\"thrv_wrapper thrv_text_element\" data-css=\"tve-u-176b272c8fc\">\n<h2 class=\"\" id=\"t-1609311604906\">Linear Regression Algorithm<\/h2>\n<p dir=\"ltr\">Before explaining the algorithm, let\u2019s see what regression is. <\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b275a561\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/2-Linear-Regression.png?resize=551%2C336&amp;ssl=1\" class=\"tve_image wp-image-7046\" alt=\"Linear Regression\" data-id=\"7046\" width=\"551\" data-init-width=\"1570\" height=\"336\" data-init-height=\"958\" title=\"Linear Regression\" loading=\"lazy\" data-width=\"551\" data-height=\"336\" data-css=\"tve-u-176b2d254ae\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-7046\" alt=\"Linear Regression\" data-id=\"7046\" width=\"551\" data-init-width=\"1570\" height=\"336\" data-init-height=\"958\" title=\"Linear Regression\" loading=\"lazy\" src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/2-Linear-Regression.png?resize=551%2C336&amp;ssl=1\" data-width=\"551\" data-height=\"336\" data-css=\"tve-u-176b2d254ae\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element tve-froala fr-box fr-basic\">\n<p dir=\"ltr\">Regression is a method used to determine the degree of relationship between a dependent variable(y) and one or more independent variables (x).\u00a0<\/p>\n<p dir=\"ltr\">Linear regression determines the relationship between one or more independent variable (s) and one target variable.\u00a0<\/p>\n<p dir=\"ltr\"><a href=\"https:\/\/dataaspirant.com\/category\/machine-learning-2\/\" target=\"_blank\" rel=\"noopener\"><strong>In machine learning<\/strong><\/a>, linear regression is a commonly used supervised machine learning algorithm for <a href=\"https:\/\/dataaspirant.com\/classification-and-prediction\/\" target=\"_blank\" rel=\"noopener\"><strong>regression kind of problems<\/strong><\/a>. It is easy to implement and understand.\u00a0<\/p>\n<p dir=\"ltr\">Supervised means that the algorithm can make predictions based on the labeled data feed to the algorithm.<\/p>\n<p dir=\"ltr\">\u00a0Mathematically, linear regression can be represented as<\/p>\n<p dir=\"ltr\" data-css=\"tve-u-176b2d2eee5\"><strong>Y = mx+c<\/strong><\/p>\n<p dir=\"ltr\">Here,<\/p>\n<ul class=\"\">\n<li>\u00a0<strong>y =<\/strong> dependent variable (Target variable)<\/li>\n<li>\u00a0<strong>x =<\/strong> independent variable<\/li>\n<li>\u00a0<strong>m =<\/strong> regression coefficient\u00a0<\/li>\n<li>\u00a0<strong>c \u00a0=<\/strong> intercept of the line<\/li>\n<\/ul>\n<p dir=\"ltr\">In linear regression, the target variable has continuous or real values.<\/p>\n<p dir=\"ltr\">For example,<\/p>\n<p dir=\"ltr\">We are predicting the <a href=\"https:\/\/dataaspirant.com\/linear-regression-implementation-in-python\/\" target=\"_blank\" class=\"tve-froala\" rel=\"noopener\"><strong>price of houses based on certain features<\/strong><\/a>. Here, the houses\u2019 prices are the target(dependent) variable, and the features determining the price are independent variables.\u00a0<\/p>\n<p dir=\"ltr\">When the target variable can be determined using <strong>one independent<\/strong> variable, it is known as <a href=\"https:\/\/dataaspirant.com\/simple-linear-regression-python-without-any-machine-learning-libraries\/\" target=\"_blank\" rel=\"noopener\"><strong>simple linear regression<\/strong><\/a>.\u00a0<\/p>\n<p dir=\"ltr\">When it\u2019s(target) dependent on multiple variables, it is known as multiple linear regression.\u00a0<\/p>\n<p dir=\"ltr\">I hope we have given a high-level overview of the linear regression algorithm. If you want to know more, you can refer to the below articles.<\/p>\n<p dir=\"ltr\">Generally, most people don\u2019t check the linear regression assumption before building any linear regression models. But we need to check these assumptions.<\/p>\n<p dir=\"ltr\">Let me list down the linear regression assumptions we need to check, and then we can discuss each of these in detail.<\/p>\n<ol class=\"\">\n<li class=\" class=\">Linear Relationship<\/li>\n<li class=\" dir=\">Normal Distribution of Residuals<\/li>\n<li class=\" class=\">Multicollinearity<\/li>\n<li class=\" class=\">Autocorrelation<\/li>\n<li class=\" class=\">Homoscedasticity<\/li>\n<\/ol>\n<p>Ideally you need to check these for <a href=\"https:\/\/dataaspirant.com\/lasso-regression\/\" target=\"_blank\" rel=\"noopener\"><strong>Lasso regression<\/strong><\/a> and <a href=\"https:\/\/dataaspirant.com\/ridge-regression\/\" target=\"_blank\" class=\"tve-froala\" rel=\"noopener\"><strong>Ridge regression<\/strong><\/a> models too.<\/p>\n<h2 id=\"t-1609311604907\" class=\"\">Linear Relationship<\/h2>\n<p dir=\"ltr\">This is the first and most important assumption of linear regression. It states that the dependent and independent variables should be <strong>linearly related<\/strong>. It is also necessary to check for <strong>outliers<\/strong> because linear regression is sensitive to outliers.\u00a0<\/p>\n<p dir=\"ltr\">Now the question is\u00a0<\/p>\n<blockquote class=\"\"><p>How to check whether the linearity assumption is met or not.\u00a0<\/p><\/blockquote>\n<p dir=\"ltr\">For determining this, we can use scatter plots. Scatter plots help you to visualize if there is a linear relationship between variables or not. Let me take an example to elaborate on it.\u00a0<\/p>\n<p dir=\"ltr\">Suppose you have to check the relationship between the student\u2019s marks and the number of hours they study.<\/p>\n<div class=\"\">You can find this student\u2019s marks dataset in our <strong><a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/linear_regression_assumptions\" class=\"tve-froala\">Github repo<\/a>.<\/strong> Go to the <strong>inputs<\/strong> folder to download the file.<\/div>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b2783c83\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/3-House-Price-Linear-Relationship.png?resize=503%2C315&amp;ssl=1\" class=\"tve_image wp-image-8304\" alt=\"House Price Linear Relationship\" data-id=\"8304\" width=\"503\" data-init-width=\"503\" height=\"315\" data-init-height=\"315\" title=\"House Price Linear Relationship\" loading=\"lazy\" data-width=\"503\" data-height=\"315\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8304\" alt=\"House Price Linear Relationship\" data-id=\"8304\" width=\"503\" data-init-width=\"503\" height=\"315\" data-init-height=\"315\" title=\"House Price Linear Relationship\" loading=\"lazy\" src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/3-House-Price-Linear-Relationship.png?resize=503%2C315&amp;ssl=1\" data-width=\"503\" data-height=\"315\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<p dir=\"ltr\">From the above plot, we can see that devoting more hours does not necessarily increase marks, even though the relationship is still a linear one.\u00a0<\/p>\n<p dir=\"ltr\">Let\u2019s take another example where the linear relationship doesn\u2019t hold.\u00a0<\/p>\n<p dir=\"ltr\">In the given plot (Ozone vs. Radiation), we can see that the linear relationship isn\u2019t held between ozone and radiation.\u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b2791b07\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/4-Ozone-Radiation-Linear-Relationship.png?resize=504%2C504&amp;ssl=1\" class=\"tve_image wp-image-8307\" alt=\"Ozone Radiation Linear Relationship\" data-id=\"8307\" width=\"504\" data-init-width=\"504\" height=\"504\" data-init-height=\"504\" title=\"Ozone Radiation Linear Relationship\" loading=\"lazy\" data-width=\"504\" data-height=\"504\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8307\" alt=\"Ozone Radiation Linear Relationship\" data-id=\"8307\" width=\"504\" data-init-width=\"504\" height=\"504\" data-init-height=\"504\" title=\"Ozone Radiation Linear Relationship\" loading=\"lazy\" src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/4-Ozone-Radiation-Linear-Relationship.png?resize=504%2C504&amp;ssl=1\" data-width=\"504\" data-height=\"504\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<p dir=\"ltr\">Here, you can see there is <strong>no linear relationship<\/strong> between ozone and radiation.<\/p>\n<p dir=\"ltr\">It is important to check this assumption because if you fit a linear model to a non-linear one, the regression algorithm would fail to capture the trend.\u00a0<\/p>\n<p dir=\"ltr\">Hence, it will result in an <strong>inefficient model<\/strong>. Also, this will lead to erroneous predictions on the unseen data sets.<\/p>\n<p dir=\"ltr\">Now comes the question <\/p>\n<blockquote class=\"\"><p>What to do if the features and target relationship is not linear?<\/p><\/blockquote>\n<p>Let&#8217;s learn this.<\/p>\n<h3 id=\"t-1609311604908\" class=\"\">What to do if linear relationship assumption isn\u2019t met<\/h3>\n<p dir=\"ltr\">Let us discuss the options you can go with.\u00a0<\/p>\n<ol class=\"\">\n<li>You can apply <a href=\"https:\/\/dataaspirant.com\/svm-kernels\/\" target=\"_blank\" rel=\"noopener\"><strong>nonlinear transformations<\/strong><\/a> to the independent and dependent variables.<\/li>\n<li>You can add another feature to the model.\n<ol>\n<li>For example, if the plot of x\u2019 vs. y\u2019 has a parabolic shape, then it might be possible to add x2 as an additional feature in the model.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<h2 id=\"t-1609311604909\" class=\"\">Normal Distribution of Residuals<\/h2>\n<p dir=\"ltr\">The second assumption of linear regression is all the residuals or error terms should be <strong>normally distributed<\/strong>. If residuals are non-normally distributed, the estimation may become too wide or narrow.\u00a0<\/p>\n<p dir=\"ltr\">If there is non-normal distribution in residuals. You can conclude that there are some unusual data points that we have to observe closely to make a good model.\u00a0<\/p>\n<h3 id=\"t-1609311604910\" class=\"\">Ways to Check Normal Distribution\u00a0<\/h3>\n<p dir=\"ltr\">To check the normal distribution, we can leverage the help from the two plots<\/p>\n<ul class=\"\">\n<li>Distribution Plots<\/li>\n<li>Q-Q Plots<\/li>\n<\/ul>\n<h4 class=\"\">Distribution Plot<\/h4>\n<p dir=\"ltr\">We can use the <strong>distribution plot<\/strong> on the residuals to check if it is normally distributed.\u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b27bee4d\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/5-Normal-Distribution.png?resize=462%2C308&amp;ssl=1\" class=\"tve_image wp-image-8311\" alt=\"Normal Distribution\" data-id=\"8311\" width=\"462\" data-init-width=\"462\" height=\"308\" data-init-height=\"308\" title=\"Normal Distribution\" loading=\"lazy\" data-width=\"462\" data-height=\"308\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8311\" alt=\"Normal Distribution\" data-id=\"8311\" width=\"462\" data-init-width=\"462\" height=\"308\" data-init-height=\"308\" title=\"Normal Distribution\" loading=\"lazy\" src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/5-Normal-Distribution.png?resize=462%2C308&amp;ssl=1\" data-width=\"462\" data-height=\"308\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<p dir=\"ltr\">Here, the <strong>black line<\/strong> is showing the normal (standard) distribution, and the blue line is showing the current distribution.\u00a0<\/p>\n<p dir=\"ltr\">We can see that there is a slight shift in the normal and current distribution. We can use the non-linear transformation of the given features if the residuals are not normally distributed.<\/p>\n<h4 class=\"\">Q-Q Plot<\/h4>\n<p dir=\"ltr\">Which stands for <strong>\u201cquantile-quantile\u201d<\/strong> plot, can also be used to check if the residuals of a model follow a normal distribution or not.\u00a0<\/p>\n<p dir=\"ltr\">If the residuals are normally distributed, then the plot will show a <strong>straight line<\/strong>. However, the deviation in the straight line shows the absence of normality.\u00a0<\/p>\n<p dir=\"ltr\">Normality can be checked by doing statistical tests, too, like &#8211; the Kolmogorov-Smirnov test, Jarque-Barre, or D\u2019Agostino-Pearson. \u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b27d30b2\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/6-Residuals-Q-Q-Plots.png?resize=456%2C317&amp;ssl=1\" class=\"tve_image wp-image-8312\" alt=\"Residuals Q-Q Plots\" data-id=\"8312\" width=\"456\" data-init-width=\"456\" height=\"317\" data-init-height=\"317\" title=\"Residuals Q-Q Plots\" loading=\"lazy\" data-width=\"456\" data-height=\"317\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8312\" alt=\"Residuals Q-Q Plots\" data-id=\"8312\" width=\"456\" data-init-width=\"456\" height=\"317\" data-init-height=\"317\" title=\"Residuals Q-Q Plots\" loading=\"lazy\" src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/6-Residuals-Q-Q-Plots.png?resize=456%2C317&amp;ssl=1\" data-width=\"456\" data-height=\"317\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<h2 id=\"t-1609311604911\" class=\"\">Multicollinearity<\/h2>\n<p dir=\"ltr\">The next assumption of linear regression is that there should be <strong>less or no <\/strong>multicollinearity in the given dataset.\u00a0<\/p>\n<p dir=\"ltr\">This situation occurs when the features or independent variables of a given dataset are highly correlated to each other.\u00a0<\/p>\n<p dir=\"ltr\">In a model having correlated variables, it becomes difficult to determine which variable is contributing to predict the target variable. Another thing is, the standard errors tend to <strong>increase<\/strong> due to the presence of correlated variables.\u00a0<\/p>\n<p dir=\"ltr\">Also, when independent variables are highly correlated, the predicted regression coefficient of a correlated variable depends on other variables that are available in the model.\u00a0<\/p>\n<p dir=\"ltr\">If you drop one correlated variable from the model, its predicted regression coefficients will change. \u00a0It can lead to wrong conclusions and poor <a href=\"https:\/\/dataaspirant.com\/difference-between-r-squared-and-adjusted-r-squared\/\" target=\"_blank\" rel=\"noopener\"><strong>performance of our model<\/strong><\/a>.\u00a0<\/p>\n<h3 id=\"t-1609311604912\" class=\"\">How to Test Multicollinearity<\/h3>\n<p dir=\"ltr\">We can test multicollinearity by using the following approaches.<\/p>\n<ul class=\"\">\n<li>Correlation Matrix<\/li>\n<li>Tolerance<\/li>\n<li>Variance Inflation Factor<\/li>\n<\/ul>\n<p dir=\"ltr\">Let\u2019s discuss the above in detail.<\/p>\n<h4 class=\"\">Correlation matrix<\/h4>\n<p dir=\"ltr\">Correlation represents the changes between the two variables. While calculating Pearson\u2019s Bivariate Correlation matrix, it is recommended that the correlation coefficient among all independent variables should be less than 1.\u00a0<\/p>\n<p dir=\"ltr\">Let us check the correlation of the variables in our <strong>student_score<\/strong> dataset.\u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b27ea93b\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/7-Heatmap.png?resize=435%2C287&amp;ssl=1\" class=\"tve_image wp-image-8314\" alt=\"Heatmap\" data-id=\"8314\" width=\"435\" data-init-width=\"435\" height=\"287\" data-init-height=\"287\" title=\"Heatmap\" loading=\"lazy\" data-width=\"435\" data-height=\"287\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8314\" alt=\"Heatmap\" data-id=\"8314\" width=\"435\" data-init-width=\"435\" height=\"287\" data-init-height=\"287\" title=\"Heatmap\" loading=\"lazy\" src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/7-Heatmap.png?resize=435%2C287&amp;ssl=1\" data-width=\"435\" data-height=\"287\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<p dir=\"ltr\">In this dataset, we are having one <strong>independent <\/strong>variable(hours) only to determine our <strong>target<\/strong> variable (score). We can see that hours devoted are highly correlated with marks scored by the student.\u00a0<\/p>\n<h4 class=\"\">Tolerance<\/h4>\n<p dir=\"ltr\">Tolerance helps us to determine the effect of one independent variable on all other independent variables.\u00a0<\/p>\n<p dir=\"ltr\">Mathematically, it can be defined as T = 1-R\u00b2, where R2 is computed by regressing the independent variable of concern onto the remaining independent variables. If the value of T is <strong>less than 0.01<\/strong>, i.e., T&lt;0.01, then your data has multicollinearity. \u00a0\u00a0<\/p>\n<h4 class=\"\">Variance Inflation Factor<\/h4>\n<p dir=\"ltr\">VIF approach chooses each feature and regresses it against the remaining features. It is calculated by using the given formula\u00a0<\/p>\n<p dir=\"ltr\" data-css=\"tve-u-176b287de9e\"><strong>VIF = 1 \/ 1 &#8211; R^2<\/strong><\/p>\n<ul class=\"\">\n<li>If VIF <strong>value &lt;=4<\/strong>, it implies no multicollinearity<\/li>\n<li>If VIF <strong>value&gt;=10<\/strong>, it implies significant multicollinearity<\/li>\n<\/ul>\n<h3 id=\"t-1609311604913\" class=\"\">Methods to handle Multicollinearity<\/h3>\n<ol class=\"\">\n<li class=\" dir=\">You can <strong>drop<\/strong> one of those features which are <strong>highly correlated<\/strong> in the given data.<\/li>\n<li class=\" dir=\">Derive a <strong>new feature<\/strong> from collinear features and drop these features (used for making new features).<\/li>\n<\/ol>\n<h2 id=\"t-1609311604914\" class=\"\">Autocorrelation<\/h2>\n<p dir=\"ltr\">One of the analytical assumptions of linear regression is that the given dataset should not be autocorrelated. This phenomenon occurs when residuals or error terms are not independent of each other. \u00a0<\/p>\n<p dir=\"ltr\">In simple terms, when the value of f(x+1) is not independent of the value of f(x). \u00a0This situation usually occurs in the case of <strong>stock prices<\/strong>, where the price of a stock is dependent on its previous one.\u00a0<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b280d1bf\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/8-Auto-Correlation-Example.png?resize=626%2C376&amp;ssl=1\" class=\"tve_image wp-image-8318\" alt=\"Auto Correlation Example\" data-id=\"8318\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Auto Correlation Example\" loading=\"lazy\" data-width=\"626\" data-height=\"376\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8318\" alt=\"Auto Correlation Example\" data-id=\"8318\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Auto Correlation Example\" loading=\"lazy\" src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/8-Auto-Correlation-Example.png?resize=626%2C376&amp;ssl=1\" data-width=\"626\" data-height=\"376\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element tve-froala fr-box fr-basic\">\n<h3 id=\"t-1609311604915\" class=\"\">How to Test Autocorrelation Assumption is met?<\/h3>\n<p dir=\"ltr\">The easiest way to check if this assumption is met to look at a residual time series plot. This is a plot of residuals vs. time.<\/p>\n<p dir=\"ltr\">Usually, most of the residual autocorrelations should fall within the <strong>95%<\/strong> confidence intervals around zero. Which are located at about <strong>+\/- 2-over<\/strong> the square root of N, where N is the dataset\u2019s size.\u00a0<\/p>\n<p dir=\"ltr\">It can also be checked using the <strong>Durbin-Watson tes<a href=\"https:\/\/en.wikipedia.org\/wiki\/Durbin%E2%80%93Watson_statistic\" class=\"tve-froala fr-basic\">t<\/a><\/strong>.<\/p>\n<p dir=\"ltr\">Durbin-Watson tes<a href=\"https:\/\/en.wikipedia.org\/wiki\/Durbin%E2%80%93Watson_statistic\">t<\/a> statistics can be implemented using <strong>statsmodels.durbin_watson()<\/strong> method.\u00a0<\/p>\n<p dir=\"ltr\"><strong>Formula:<\/strong><\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b281c002\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/9-Durbin-watson-test.png?resize=300%2C102&amp;ssl=1\" class=\"tve_image wp-image-8320\" alt=\"Durbin watson test\" data-id=\"8320\" width=\"300\" data-init-width=\"300\" height=\"102\" data-init-height=\"102\" title=\"Durbin watson test\" loading=\"lazy\" data-width=\"300\" data-height=\"102\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8320\" alt=\"Durbin watson test\" data-id=\"8320\" width=\"300\" data-init-width=\"300\" height=\"102\" data-init-height=\"102\" title=\"Durbin watson test\" loading=\"lazy\" src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/9-Durbin-watson-test.png?resize=300%2C102&amp;ssl=1\" data-width=\"300\" data-height=\"102\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element\">\n<p dir=\"ltr\"><strong>Output : <\/strong>\u00a00.07975460122699386<\/p>\n<ul class=\"\">\n<li>If the value of <strong>durbin_watson \u00a0= 2<\/strong>, it implies no autocorrelation<\/li>\n<li>If the value of durbin_watson lies between \u00a0<strong>0 and 2<\/strong>, it implies <strong>positive<\/strong> autocorrelation.<\/li>\n<li>If the value of durbin_watson lies between \u00a0<strong>2 and 4,<\/strong> it implies <strong>negative<\/strong> autocorrelation.<\/li>\n<\/ul>\n<h3 id=\"t-1609311604916\" class=\"\">Methods to Handle Autocorrelation<\/h3>\n<ol class=\"\">\n<li class=\" dir=\">Include the <strong>dummy variables<\/strong> in the data.<\/li>\n<li class=\" dir=\">Predicted Generalized Least Squares<\/li>\n<li class=\" dir=\">Include a linear sequence, \u00a0if the residuals showing a consistent increment \u00a0or decrement in pattern<\/li>\n<\/ol>\n<h2 id=\"t-1609311604917\" class=\"\">Homoscedasticity<\/h2>\n<p dir=\"ltr\">The fifth assumption of linear regression analysis is homoscedasticity. Homoscedasticity depicts a circumstance in which the residuals \u00a0(that is, the \u201cnoise\u201d or error terms in between the independent variables and the dependent variable) is the same across all values of the independent variables.\u00a0<\/p>\n<p dir=\"ltr\">Simply put, residuals should have <strong>constant variance<\/strong>. If this condition is not followed, it is known as <strong>heteroscedasticity<\/strong>. <\/p>\n<p dir=\"ltr\">Heteroscedasticity leads to the unbalanced scatter of residuals or error terms. Generally, non-constant variation arises in the presence of outliers. <\/p>\n<p dir=\"ltr\">It seems like these values get too much importance, thereby disproportionately impact the model\u2019s performance. The presence of heteroscedasticity in a regression analysis makes it <strong>difficult to trust <\/strong>the results of the analysis.\u00a0<\/p>\n<h3 id=\"t-1609311604918\" class=\"\">How to Test if Homoscedasticity Assumption is met?<\/h3>\n<p>The most basic approach to test for heteroscedasticity is by plotting fitted values against residual values. <\/p>\n<p>The plot will show a <strong>funnel-shaped<\/strong> pattern if heteroscedasticity exists.<\/p>\n<\/div>\n<div class=\"thrv_wrapper tve_image_caption\" data-css=\"tve-u-176b283d190\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/10-Homoscedasticity-Vs-Heteroscedasticity.png?resize=626%2C424&amp;ssl=1\" class=\"tve_image wp-image-8324\" alt=\"Homoscedasticity Vs Heteroscedasticity\" data-id=\"8324\" width=\"626\" data-init-width=\"1024\" height=\"424\" data-init-height=\"693\" title=\"Homoscedasticity Vs Heteroscedasticity\" loading=\"lazy\" data-width=\"626\" data-height=\"424\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-8324\" alt=\"Homoscedasticity Vs Heteroscedasticity\" data-id=\"8324\" width=\"626\" data-init-width=\"1024\" height=\"424\" data-init-height=\"693\" title=\"Homoscedasticity Vs Heteroscedasticity\" loading=\"lazy\" src=\"https:\/\/i1.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/10-Homoscedasticity-Vs-Heteroscedasticity.png?resize=626%2C424&amp;ssl=1\" data-width=\"626\" data-height=\"424\" data-recalc-dims=\"1\"><\/span><\/div>\n<div class=\"thrv_wrapper thrv_text_element tve-froala fr-box fr-basic\">\n<p dir=\"ltr\">The presence of heteroscedasticity can also be computed using the <strong>statistical approach<\/strong>. They are as following:<\/p>\n<p dir=\"ltr\"><strong>The Breush \u2013 Pegan Test:\u00a0<\/strong><\/p>\n<p dir=\"ltr\">It determines whether the variance of the residuals from regression depends on the values of the independent variables. If it is so then, heteroscedasticity is present.<\/p>\n<p dir=\"ltr\"><strong>White Test:<\/strong><\/p>\n<p dir=\"ltr\">White test determines if the variance of the residuals in a regression analysis model is fixed or constant.<\/p>\n<h3 id=\"t-1609311604919\" class=\"\">Methods to handle Heteroscedasticity<\/h3>\n<p>We are having two ways to handle the Heteroscedasticity, let&#8217;s understand both.<\/p>\n<h4 class=\"\">Transform the Dependent Variables\u00a0<\/h4>\n<p dir=\"ltr\">We can transform the dependent variables to avoid heteroskedasticity. The most commonly used transformation is taking the log of dependent variables.\u00a0<\/p>\n<p dir=\"ltr\">For instance,<\/p>\n<p dir=\"ltr\">If we are using independent variables(input features) \u00a0to predict the number of cosmetic shops in a city (target variable). We may try to use input features to predict the log of the number of cosmetic shops in a city.<\/p>\n<p dir=\"ltr\">Using the <strong>log of the target<\/strong> variable helps to reduce the heteroskedasticity. To some extent.\u00a0<\/p>\n<h4 class=\"\">Use weighted regression<\/h4>\n<p dir=\"ltr\">Another approach to deal with heteroskedasticity is by using weighted regression. In this method, a weight is assigned to each data point based on the variance of its fitted value.<\/p>\n<h2 id=\"t-1609311604920\" class=\"\">Conclusion\u00a0<\/h2>\n<p dir=\"ltr\" id=\"t-1609311604921\">This is the end of this article. We discussed the assumptions of linear regression analysis, ways to check if the assumptions are met or not, and what to do if these assumptions are violated.\u00a0<\/p>\n<p dir=\"ltr\" id=\"t-1609311604922\">It is necessary to consider the assumptions of linear regression for statistics. The model\u2019s performance will be very good if these assumptions are met.<\/p>\n<p dir=\"ltr\">The <a href=\"https:\/\/dataaspirant.com\/linear-regression-implementation-in-python\/\" target=\"_blank\" rel=\"noopener\"><strong>classical linear regression model<\/strong><\/a> is one of the most systematic predictors if all the assumptions hold.\u00a0<\/p>\n<p dir=\"ltr\" id=\"t-1609311604923\">The best thing about this concept is that the <strong>efficiency increases<\/strong> as the sample size increases to infinity.\u00a0<\/p>\n<h2 id=\"t-1609311604924\" class=\"\">What next<\/h2>\n<p dir=\"ltr\">After reading the article, please take any of the regression algorithm you have developed in the past and check these linear regression assumptions.<\/p>\n<p dir=\"ltr\">For implementing and understanding the linear regression concepts. I would suggest reading <strong><a href=\"https:\/\/dataaspirant.com\/linear-regression\/\" class=\"tve-froala fr-basic\">this<\/a> <\/strong>article to understand the <strong><a href=\"https:\/\/dataaspirant.com\/linear-regression-implementation-in-python\/\" class=\"tve-froala\">linear regression<\/a><\/strong> concept in a more practical way.<\/p>\n<p dir=\"ltr\">Also, explore remaining machine learning algorithms on <a href=\"https:\/\/dataaspirant.com\/for-beginners\/\" target=\"_blank\" rel=\"noopener\"><strong>our platform<\/strong><\/a> to enhance your knowledge.<\/p>\n<\/div>\n<h4 class=\"\">Recommended Machine Learning Courses<\/h4>\n<div class=\"thrv_wrapper thrv-page-section thrv-lp-block\" data-inherit-lp-settings=\"1\" data-css=\"tve-u-176b272c7a7\" data-keep-css_id=\"1\">\n<div class=\"tve-page-section-in tve_empty_dropzone  \" data-css=\"tve-u-17481b960b8\">\n<div class=\"thrv_wrapper thrv-columns dynamic-group-kbt3q0q7\" data-css=\"tve-u-17481b95e2b\">\n<div class=\"tcb-flex-row v-2 tcb--cols--3 tcb-medium-no-wrap tcb-mobile-wrap m-edit\" data-css=\"tve-u-176b272c7a8\">\n<div class=\"tcb-flex-col\">\n<div class=\"tcb-col dynamic-group-kbt3pyfd\" data-css=\"tve-u-17481b95e2d\">\n<div class=\"thrv_wrapper thrv_contentbox_shortcode thrv-content-box tve-elem-default-pad dynamic-group-kbt3pwhk\" data-css=\"tve-u-176b272c7bf\">\n<div class=\"tve-cb\">\n<div class=\"thrv_wrapper tve_image_caption dynamic-group-kbt3pu4z\" data-css=\"tve-u-176b272c7c2\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/deeplearning-course.jpg?resize=176%2C176&amp;ssl=1\" class=\"tve_image wp-image-5170\" alt=\"Deep Learning python\" data-id=\"5170\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"deeplearning-course\" loading=\"lazy\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7c3\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-5170\" alt=\"Deep Learning python\" data-id=\"5170\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"deeplearning-course\" loading=\"lazy\" src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/deeplearning-course.jpg?resize=176%2C176&amp;ssl=1\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7c3\" data-recalc-dims=\"1\"><span class=\"tve-image-overlay\"><\/span><\/span><\/div>\n<h4 class=\"\" data-css=\"tve-u-176b272c7aa\">Machine Learning A to Z Course<\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"tcb-flex-col\">\n<div class=\"tcb-col dynamic-group-kbt3pyfd\" data-css=\"tve-u-17481b95e2d\">\n<div class=\"thrv_wrapper thrv_contentbox_shortcode thrv-content-box tve-elem-default-pad dynamic-group-kbt3pwhk\" data-css=\"tve-u-176b272c7c0\">\n<div class=\"tve-cb\">\n<div class=\"thrv_wrapper tve_image_caption dynamic-group-kbt3pu4z\" data-css=\"tve-u-176b272c7ce\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/10\/Deep-learning-for-computer-vision2.png.jpeg?resize=176%2C176&amp;ssl=1\" class=\"tve_image wp-image-6739\" alt=\"Deep-learning-for-computer-vision2.png\" data-id=\"6739\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"Deep-learning-for-computer-vision2.png\" loading=\"lazy\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7cf\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-6739\" alt=\"Deep-learning-for-computer-vision2.png\" data-id=\"6739\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"Deep-learning-for-computer-vision2.png\" loading=\"lazy\" src=\"https:\/\/i0.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/10\/Deep-learning-for-computer-vision2.png.jpeg?resize=176%2C176&amp;ssl=1\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7cf\" data-recalc-dims=\"1\"><span class=\"tve-image-overlay\"><\/span><\/span><\/div>\n<h4 class=\"\" data-css=\"tve-u-176b272c7b1\">Python Data Science Specialization Course<\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<div class=\"tcb-flex-col\">\n<div class=\"tcb-col dynamic-group-kbt3pyfd\" data-css=\"tve-u-17481b95e2d\">\n<div class=\"thrv_wrapper thrv_contentbox_shortcode thrv-content-box tve-elem-default-pad dynamic-group-kbt3pwhk\" data-css=\"tve-u-176b272c7c1\">\n<div class=\"tve-cb\">\n<div class=\"thrv_wrapper tve_image_caption dynamic-group-kbt3pu4z\" data-css=\"tve-u-176b272c7d0\"><span class=\"tve_image_frame\"><img src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/plugins\/lazy-load\/images\/1x1.trans.gif?ssl=1\" data-lazy-src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/supervised-learning.png?resize=176%2C176&amp;ssl=1\" class=\"tve_image wp-image-4696\" alt=\"supervised learning\" data-id=\"4696\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"supervised learning\" loading=\"lazy\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7d1\" data-recalc-dims=\"1\"><img class=\"tve_image wp-image-4696\" alt=\"supervised learning\" data-id=\"4696\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"supervised learning\" loading=\"lazy\" src=\"https:\/\/i2.wp.com\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/supervised-learning.png?resize=176%2C176&amp;ssl=1\" data-width=\"176\" data-height=\"176\" data-css=\"tve-u-176b272c7d1\" data-recalc-dims=\"1\"><span class=\"tve-image-overlay\"><\/span><\/span><\/div>\n<h4 class=\"\" data-css=\"tve-u-176b272c7b8\">Complete Supervised Learning Algorithms<\/h4>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/dataaspirant.com\/assumptions-of-linear-regression-algorithm\/<\/p>\n","protected":false},"author":0,"featured_media":8055,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8054"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=8054"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8054\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/8055"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=8054"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=8054"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=8054"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}