{"id":8143,"date":"2021-03-08T04:35:56","date_gmt":"2021-03-08T04:35:56","guid":{"rendered":"https:\/\/wealthrevelation.com\/data-science\/2021\/03\/08\/house-price-prediction-using-machine-learning\/"},"modified":"2021-03-08T04:35:56","modified_gmt":"2021-03-08T04:35:56","slug":"house-price-prediction-using-machine-learning","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2021\/03\/08\/house-price-prediction-using-machine-learning\/","title":{"rendered":"House Price Prediction using Machine Learning"},"content":{"rendered":"<div>\n<h2>Introduction<\/h2>\n<p>Purchasing a new house is always a big decision ! Is it the location ? Is it the overall quality of the house ? Is it the size ? Could it be sold at a good price in future ? Moreover, economic growth plays an important role as housing demand is often seen as elastic in terms of income, leading to an increase in revenues for households. Undoubtedly it is a \u00a0tough call to consider which features should be of most importance. To ease this decision-making, in today\u2019s world, Machine Learning allows an entrepreneur \u00a0on forecasting the house price with a maximum accuracy of market trend and building model of the historical dataset on \u2018what happened and why\u2019 to predict \u2018What is going to happen\u2019.<\/p>\n<p>In this blog, a step-by-step technical approach have\u00a0 been described for predicting house sale price for a <a title=\"Kaggle data-set of Ames, Iowa\" href=\"https:\/\/www.kaggle.com\/c\/house-prices-advanced-regression-techniques\" target=\"_blank\" rel=\"noopener\">Kaggle data-set of Ames, Iowa<\/a>. The project endeavors to extensive data analysis and implementation of different machine learning techniques in python for having the best model with most important features of a house on insight of both business value and realistic perspective. The dataset consists of 79 different features for 1460 houses in Ames which can be used as training data to predict the sale price of another 1459 test data set of machine learning model.<\/p>\n<h2>Process Steps<\/h2>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-300x140.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-600x280.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-768x358.jpg 768w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-1024x478.jpg 1024w\" loading=\"lazy\" width=\"1024\" height=\"478\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-1024x478.jpg\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" class=\"wp-image-71883 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"1024\" height=\"478\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ds-process-steps.jpg-950578-HPpgFyfk-1024x478.jpg\" alt=\"\" class=\"wp-image-71883\"><\/figure>\n<p>The entire process of machine learning can be divided in 4 main steps to get the desired prediction.<\/p>\n<ul>\n<li>Get Data \u2013 Data can be collected from the source in this step for exploratory data analysis and visualization for understanding the current\/historic data and determine the next step.<\/li>\n<li>Data Pre-Processing \u2013 Pre-Processing aka data wrangling is the technique of cleaning and transforming the raw data (which can be incomplete and inconsistent) to a proper format to be used for modeling.<\/li>\n<li>Model Evaluation \u2013 Different machine learning algorithms can be used and evaluated in this step to measure the accuracy and other performance metrics.<\/li>\n<li>Prediction \u2013 Depending on choice of the best model, prediction is done.<\/li>\n<\/ul>\n<h2>Exploratory Data Analysis<\/h2>\n<p>This is the way of analyzing, visualizing, summarizing and interpreting the dataset to achieve some insights on statistical measure and validate the hypothesis.<\/p>\n<p>For this dataset, the first step of EDA is to check the distribution of Sale Price on train data-set. It\u2019s observed in the histogram that the data is rightly skewed \u00a0with a skewness of 1.88 and kurtosis as 6.53 which needs a log-transformation to correct the distribution in feature-engineering step.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-300x128.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-600x257.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-768x329.jpg 768w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-1024x438.jpg 1024w\" loading=\"lazy\" width=\"1024\" height=\"438\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-1024x438.jpg\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" class=\"wp-image-71887 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"1024\" height=\"438\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/hist1.jpg-098565-kRUEDEnu-1024x438.jpg\" alt=\"\" class=\"wp-image-71887\"><\/figure>\n<p>In the next step, further data analysis done to reveal that the percentage of houses built before 1980 is 58.7% and percentage of houses remodeled is 47.8% as Ames is an old place.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-300x129.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-600x257.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-768x330.jpg 768w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-1024x439.jpg 1024w\" loading=\"lazy\" width=\"1024\" height=\"439\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-1024x439.jpg\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" class=\"wp-image-71889 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"1024\" height=\"439\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/yrblt.jpg-159192-syaeK8ww-1024x439.jpg\" alt=\"\" class=\"wp-image-71889\"><\/figure>\n<p><strong>Correlation Analysis &#8211; <\/strong>Correlation analysis helps to identify top features highly correlated with Sale Price. Multicollinearity found as well for couple of features which needs to be handled in feature engineering step to get best model.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/correlation.jpg-937215-iFASBNaY-300x238.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/correlation.jpg-937215-iFASBNaY-600x476.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/correlation.jpg-937215-iFASBNaY.jpg 663w\" loading=\"lazy\" width=\"663\" height=\"526\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/correlation.jpg-937215-iFASBNaY.jpg\" data-sizes=\"(max-width: 663px) 100vw, 663px\" class=\"wp-image-71916 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"663\" height=\"526\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/correlation.jpg-937215-iFASBNaY.jpg\" alt=\"\" class=\"wp-image-71916\"><\/figure>\n<p><strong>Outlier Analysis<\/strong> &#8211; Couple of outliers found for Living Area, Lot Area and neighborhood which can be removed in pre-processing. Since the train data does not contain school district information and crime rate, neighborhood is an important factor, implying above factors. It&#8217;s observed that\u00a0the variance of an expensive neighborhood is typically higher, which explains the skewness of the sales price density as well.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-300x137.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-600x275.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-768x352.jpg 768w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-1024x469.jpg 1024w\" loading=\"lazy\" width=\"1024\" height=\"469\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-1024x469.jpg\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" class=\"wp-image-71919 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"1024\" height=\"469\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/outliers.jpg-586456-MiKSAtyb-1024x469.jpg\" alt=\"\" class=\"wp-image-71919\"><\/figure>\n<h2>Pre-Processing<\/h2>\n<p><strong>Missing Data Handling<\/strong>\u00a0 &#8211; Overall, 2.68% data is missing in the training data<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/msd.jpg-181109-xSPPJ7D8-300x199.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/msd.jpg-181109-xSPPJ7D8-600x398.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/msd.jpg-181109-xSPPJ7D8.jpg 739w\" loading=\"lazy\" width=\"739\" height=\"490\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/msd.jpg-181109-xSPPJ7D8.jpg\" data-sizes=\"(max-width: 739px) 100vw, 739px\" class=\"wp-image-71924 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"739\" height=\"490\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/msd.jpg-181109-xSPPJ7D8.jpg\" alt=\"\" class=\"wp-image-71924\"><\/figure>\n<p>Below are the steps done for missing data<\/p>\n<ul>\n<li>As PoolQC, MiscFeature, Alley and Fence features have more than 90% data missing, dropped these features as there will not be any impact.<\/li>\n<li>LotFrontage missing values imputed based on median value of neighborhood LotFrontage.<\/li>\n<li>The features having many missing data but having correlation with sale-price less than 0.5, have been dropped as there will not be impact.<\/li>\n<li>For some numerical predictors, missing values are imputed as mean value or 0 based on analysis. For some categorical predictors, missing values are imputed as NA or the value mostly used.<\/li>\n<\/ul>\n<p><strong>Feature Selection\/Engineering &#8211;\u00a0<\/strong>This step identifies the selective features needed and having most significance with the target feature.<\/p>\n<ul>\n<li>Log transformation applied to SalePrice to correct the skewness.<\/li>\n<li>4 new features created to get more business importance from usability standpoint \u2013 TotalSF, Total_Bathrooms, Years_SinceRemodel,HasGarage.<\/li>\n<li>Some numeric predictors dropped to avoid multicollinearity based on correlation analysis. Outliers removed from the training data accordingly.<\/li>\n<li>Predictors having negative correlation with Sale Price \u2013 dropped.<\/li>\n<li>Categorical predictors dummified to use in regression process and some of them dropped if only one value is dominating.<\/li>\n<\/ul>\n<h2>Model Evaluation<\/h2>\n<p>Model selection and evaluation is a critical step of any machine learning project as identifying the pattern and applying the correct algorithm is not a very easy process. Machine Learning provides multiple number of models \u00a0to generalize it to the unseen data from the same population and measuring the performance. Along with meeting the business objective, the model should take care of accuracy, execution time, complexity and scalability as well to be considered as best model. Sometimes size of training data set and numbers of predictor features can be decision-making\u00a0 criteria for model selection. RMSE (Root Mean Square Error) is the performance metrics for this project.<\/p>\n<p>For this particular dataset, it&#8217;s observed to have linearity and normal distribution of Sale Price data, which directs to evaluate the linear regression model first. To reduce the overfitting issue, both Ridge and Lasso also comes under consideration for evaluation. To check whether the non-linear models outperform, Random Forest and XGBoost also added. Firstly to split train-test data set, 10-fold cross validation method applied which shows an interesting statistics as below:<\/p>\n<figure class=\"wp-block-image size-large is-resized\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT-300x124.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT-600x248.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT-768x318.jpg 768w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT.jpg 846w\" loading=\"lazy\" alt=\"\" width=\"470\" height=\"193\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT.jpg\" data-sizes=\"(max-width: 846px) 100vw, 846px\" class=\"wp-image-71932 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/cv.jpg-790456-5Ca5KQyT.jpg\" alt=\"\" class=\"wp-image-71932\" width=\"470\" height=\"193\"><\/figure>\n<p>Performance metrics of RMSE resulted to be very close for Ridge, Random Forest and XGBoost. As Random Forest took significant time to get the result, further analysis is done between Ridge and XGBoost for the best model selection.<\/p>\n<p>For Ridge, by introducing an additional hyperparameter lambda, a fine-tuned penalized regression trades off a slight increase of biases for a significant drop of its model variance, which results in an improved accuracy. With tuning the hyperparameter setting alpha as .01, the best RMSE and R-Square value received for this model in training data which reduces the risk of overfitting to a great extent.<\/p>\n<figure class=\"wp-block-image size-large is-resized\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge.jpg-669298-ltIqbC2d-300x65.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge.jpg-669298-ltIqbC2d-600x131.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge.jpg-669298-ltIqbC2d.jpg 662w\" loading=\"lazy\" alt=\"\" width=\"397\" height=\"86\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge.jpg-669298-ltIqbC2d.jpg\" data-sizes=\"(max-width: 662px) 100vw, 662px\" class=\"wp-image-71961 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge.jpg-669298-ltIqbC2d.jpg\" alt=\"\" class=\"wp-image-71961\" width=\"397\" height=\"86\"><\/figure>\n<p>Bur exploring the feature-importance of the ridge model, it is not convincing that most important feature of this model will\u00a0 meet the business value.<\/p>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge-feature.jpg-306493-LsVoR5rf-300x254.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge-feature.jpg-306493-LsVoR5rf-600x509.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge-feature.jpg-306493-LsVoR5rf.jpg 730w\" loading=\"lazy\" width=\"730\" height=\"619\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge-feature.jpg-306493-LsVoR5rf.jpg\" data-sizes=\"(max-width: 730px) 100vw, 730px\" class=\"wp-image-71962 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"730\" height=\"619\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/ridge-feature.jpg-306493-LsVoR5rf.jpg\" alt=\"\" class=\"wp-image-71962\"><\/figure>\n<p>As a next step, XGBoost is analyzed thoroughly through grid search to get the best parameter value of max depth and n_estimators. XGBoost is a tree based ensemble machine learning algorithm which is a scalable machine learning system for tree boosting. XGBoost stands for Extreme Gradient Boosting. It uses second order partial derivatives as approximation of loss function to provide more information about the direction of the gradient and the way to get to the minimum of the loss function. That algorithm smoothens the final learnt weights and thus avoids overfitting. Grid search resulted best max_depth as 3 and n_estimators as 100. Using these values as parameter, the best model performance achieved as RMSE of .0616 and R-Square as .97. The feature-importance bar chart also resulted very realistic in terms of business features of a house sale.<\/p>\n<figure class=\"wp-block-image size-large is-resized\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost.jpg-774563-J4iFFCIV-300x67.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost.jpg-774563-J4iFFCIV-600x134.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost.jpg-774563-J4iFFCIV.jpg 662w\" loading=\"lazy\" alt=\"\" width=\"394\" height=\"87\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost.jpg-774563-J4iFFCIV.jpg\" data-sizes=\"(max-width: 662px) 100vw, 662px\" class=\"wp-image-71963 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost.jpg-774563-J4iFFCIV.jpg\" alt=\"\" class=\"wp-image-71963\" width=\"394\" height=\"87\"><\/figure>\n<figure class=\"wp-block-image size-large is-resized\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost-feature.jpg-133715-KvUBV3pJ-300x284.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost-feature.jpg-133715-KvUBV3pJ-600x567.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost-feature.jpg-133715-KvUBV3pJ.jpg 697w\" loading=\"lazy\" alt=\"\" width=\"547\" height=\"517\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost-feature.jpg-133715-KvUBV3pJ.jpg\" data-sizes=\"(max-width: 697px) 100vw, 697px\" class=\"wp-image-71964 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/xgboost-feature.jpg-133715-KvUBV3pJ.jpg\" alt=\"\" class=\"wp-image-71964\" width=\"547\" height=\"517\"><\/figure>\n<p>In realistic way, it&#8217;s completely understood and accepted that Car Space in Garage, Overall Quality of the house, Total Square foot, total bathroom numbers, remodeled, neighborhood convenient to business places, advantage of railway commute &#8211; all features are significant in a house sale price for a homeowner which have been predicted by XGBoost efficiently with highest speed. Comparing the selected model behavior and performances , XGBoost is chosen as the best model for training the data set because of :<\/p>\n<ul>\n<li><b>Lowest RMSE<\/b><\/li>\n<li><b>Highest R-square<\/b><\/li>\n<li><b>Best computation speed<\/b><\/li>\n<li><b>Realistic Feature Importance<\/b><\/li>\n<\/ul>\n<h2>Prediction<\/h2>\n<p>XGBoost Model applied to test data for sale price prediction of 1459 houses and accordingly predicted sale prices are listed. Here is a snap-shot.<\/p>\n<figure class=\"wp-block-image size-large is-resized\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction.jpg-432121-YxVkAvSQ-300x280.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction.jpg-432121-YxVkAvSQ.jpg 319w\" loading=\"lazy\" alt=\"\" width=\"250\" height=\"234\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction.jpg-432121-YxVkAvSQ.jpg\" data-sizes=\"(max-width: 319px) 100vw, 319px\" class=\"wp-image-71966 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction.jpg-432121-YxVkAvSQ.jpg\" alt=\"\" class=\"wp-image-71966\" width=\"250\" height=\"234\"><\/figure>\n<figure class=\"wp-block-image size-large\"><img data-srcset=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction-hist.jpg-356010-z2FmhKPJ-300x204.jpg 300w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction-hist.jpg-356010-z2FmhKPJ-600x408.jpg 600w, https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction-hist.jpg-356010-z2FmhKPJ.jpg 731w\" loading=\"lazy\" width=\"731\" height=\"497\" alt=\"\" data-src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction-hist.jpg-356010-z2FmhKPJ.jpg\" data-sizes=\"(max-width: 731px) 100vw, 731px\" class=\"wp-image-71967 lazyload\" src=\"image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\"><img loading=\"lazy\" width=\"731\" height=\"497\" src=\"https:\/\/nycdsa-blog-files.s3.us-east-2.amazonaws.com\/2021\/03\/chaitali-majumder\/prediction-hist.jpg-356010-z2FmhKPJ.jpg\" alt=\"\" class=\"wp-image-71967\"><\/figure>\n<p>The histogram of predicted sale price of the 1459 houses can clearly show how effectively the prediction is done. Average predicted sale price is $178653.35, which maintains the same trend of train data.<\/p>\n<p>Conclusion &#8211; In summary, it&#8217;s aimed at answering two main questions in this project: will the tree-based model outperform the regularized linear model on predicting house price of the given dataset? The result shows though Ridge accuracy is good but the significant house features are not satisfactory like XGBoost model. The XGBoost model has the potential to perfectly predict the sale price having most important features identified\u00a0 for a homeowners to look to add value of their homes.<\/p>\n<p>For Python Code Reference, please click\u00a0<a href=\"https:\/\/gist.github.com\/Chaitali20-gh\/f8e8cf05a14c1e52d92a71fac7679f0e\">here<\/a> .<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>https:\/\/nycdatascience.com\/blog\/student-works\/machine-learning\/house-price-prediction-using-machine-learning-2\/<\/p>\n","protected":false},"author":0,"featured_media":8144,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8143"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=8143"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/8143\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/8144"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=8143"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=8143"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=8143"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}