{"id":910,"date":"2020-09-02T15:17:47","date_gmt":"2020-09-02T15:17:47","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/09\/02\/which-methods-should-be-used-for-solving-linear-regression\/"},"modified":"2020-09-02T15:17:47","modified_gmt":"2020-09-02T15:17:47","slug":"which-methods-should-be-used-for-solving-linear-regression","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/09\/02\/which-methods-should-be-used-for-solving-linear-regression\/","title":{"rendered":"Which methods should be used for solving linear regression?"},"content":{"rendered":"
\n

By Ahmad BinShafiq<\/a>, Machine Learning Student<\/b>.<\/p>\n

Linear Regression\u00a0is a supervised machine learning algorithm. It predicts a\u00a0linear relationship\u00a0between an\u00a0independent variable (y), based on the given\u00a0dependant variables (x), such that the\u00a0independent variable (y)\u00a0has the\u00a0lowest cost<\/strong>.<\/p>\n

\u00a0<\/p>\n

Different approaches to solve linear regression models<\/h3>\n

\u00a0<\/p>\n

There are many different methods that we can apply to our linear regression model in order to make it more efficient. But we will discuss the most common of them here.<\/p>\n

    \n
  1. Gradient Descent<\/li>\n
  2. Least Square Method \/ Normal Equation Method<\/li>\n
  3. Adams Method<\/li>\n
  4. Singular Value Decomposition (SVD)<\/li>\n<\/ol>\n

    Okay, so let\u2019s begin\u2026<\/p>\n

    \u00a0<\/p>\n

    Gradient Descent<\/h3>\n

    \u00a0<\/p>\n

    One of the most common and easiest methods for\u00a0beginners\u00a0to solve linear regression problems is gradient descent.<\/p>\n

    How Gradient Descent works<\/strong><\/p>\n

    Now, let’s suppose we have our data plotted out in the form of a scatter graph, and when we apply a cost function to it, our model will make a prediction. Now this prediction can be very good, or it can be far away from our ideal prediction (meaning its cost will be high). So, in order to minimize that cost (error), we apply gradient descent to it.<\/p>\n

    Now, gradient descent will slowly converge our hypothesis towards a global minimum, where the\u00a0cost<\/strong>\u00a0would be lowest. In doing so, we have to manually set the value of\u00a0alpha,\u00a0<\/strong>and the slope of the hypothesis changes with respect to our alpha\u2019s value. If the value of alpha is large, then it will take big steps. Otherwise, in the case of small alpha, our hypothesis would converge slowly and through small baby steps.<\/p>\n

    <\/p>\n

    Hypothesis converging towards a global minimum. Image from\u00a0Medium<\/a>.<\/em><\/p>\n

    The Equation for Gradient Descent is<\/p>\n

    <\/p>\n

    Source:\u00a0Ruder.io<\/a>.<\/em><\/p>\n

    Implementing Gradient Descent in Python<\/strong><\/p>\n

    \n
    import numpy as np\r\nfrom matplotlib import pyplot\r\n\r\n#creating our data\r\nX = np.random.rand(10,1)\r\ny = np.random.rand(10,1)\r\nm = len(y)\r\ntheta = np.ones(1)\r\n\r\n#applying gradient descent\r\na = 0.0005\r\ncost_list = []\r\nfor i in range(len(y)):\r\n    \r\n    theta = theta - a*(1\/m)*np.transpose(X)@(X@theta - y)\r\n           \r\n    cost_val = (1\/m)*np.transpose(X)@(X@theta - y)\r\n    cost_list.append(cost_val)\r\n\r\n#Predicting our Hypothesis\r\nb = theta\r\nyhat = X.dot(b)\r\n\r\n#Plotting our results\r\npyplot.scatter(X, y, color='red')\r\npyplot.plot(X, yhat, color='blue')\r\npyplot.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Model after Gradient Descent.<\/em><\/p>\n
    Here first, we have created our dataset, and then we looped over all our training examples in order to minimize our cost of hypothesis.<\/p>\n
    Pros:<\/strong><\/p>\n
    Important advantages of Gradient Descent are<\/p>\n