{"id":7296,"date":"2020-10-28T06:01:26","date_gmt":"2020-10-28T06:01:26","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/10\/28\/loan-default-detection\/"},"modified":"2020-10-28T06:01:26","modified_gmt":"2020-10-28T06:01:26","slug":"loan-default-detection","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/10\/28\/loan-default-detection\/","title":{"rendered":"Loan Default Detection"},"content":{"rendered":"
\n

Motivation<\/h3>\n

In 2005, a Taiwanese bank conducted a study on the likelihood of clients defaulting on their loan payments. The motivation behind the study was the increase in amounts of credit being offered by banks to customers, regardless of their repayment capabilities. This led to customers accumulating significant amounts of debt, which in turn resulted in defaults.<\/p>\n

The goal was to use basic information about customers along with their past repayment history to predict their likelihood of default. Our objective is to use the previous 6 months of repayment history to try and predict whether the customer will default the following month.<\/p>\n

\n
\"\"\"\"<\/figure>\n<\/div>\n

Dataset<\/h3>\n

The dataset used was collected by a Taiwanese bank in October 2005 and can be downloaded from UCI’s Machine Learning Repository. <\/p>\n

    \n
  • \nSex:<\/strong> Gender<\/li>\n
  • \nAge:<\/strong> Client’s age<\/li>\n
  • \nMarriage:<\/strong> Marital status<\/li>\n
  • \nEducation:<\/strong> Level of education<\/li>\n
  • \nLimit balance:<\/strong> Amount of credit (NT Dollars)<\/li>\n
  • \nPayment status (month):<\/strong> Current repayment status<\/li>\n
  • \nBill statement (month):<\/strong> The amount of bill statements (NT Dollars)<\/li>\n
  • \nPrevious payment (month):<\/strong> Previous payment amount (NT Dollars)<\/li>\n
  • \nDefault payment next month: <\/strong>The target variable indicating whether the customer defaulted on the payment the following month.<\/li>\n<\/ul>\n
    \n
    \n

    Exploratory Data Analysis<\/h3>\n

    Apart from age appearing to be uncorrelated to other features., the correlation matrix doesn’t provide us with much additional information.<\/p>\n<\/div>\n<\/div>\n

    \n
    \"\"\"\"
    \u200b<\/p>\n

    CORRELATION MATRIX<\/strong><\/p>\n<\/figcaption><\/figure>\n<\/div>\n

    The pairplot doesn’t show much of a difference in the shape of the distribution per gender. We can also see a decrease in limt balances as age starts to increase beyond 55.<\/p>\n

    \n
    \"\"\"\"
    PAIRPLOT<\/strong><\/figcaption><\/figure>\n<\/div>\n

    Model Selection<\/h3>\n

    CLASSIFICATION<\/em><\/h5>\n

    We will fit a decision tree classifier for our binary classification problem. A confusion matrix will help summarize all possible combinations of the predicted values as opposed to the actual target in the form of:<\/p>\n

      \n
    • \u200bTrue positive (TP): The model predicts a default, and the client defaulted.\u200b<\/li>\n
    • False positive (FP): The model predicts a default, but the client did not default. <\/li>\n
    • \u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200b\u200bTrue negative (TN): The model predicts a good customer, and the client\u200b did not default. \u200b\u200b\u200b\u200b\u200b\u200b\u200b<\/li>\n
    • False negative (FN): The model predicts a good customer, but the client defaulted.<\/li>\n<\/ul>\n

      We can use these values to create additional evaluation criterias for our model. <\/p>\n

      \n
      \n
        \n
      • Accuracy: Measures the model’s overall ability to correctly predict the class of the observation.<\/li>\n
      • Precision: Out of all default predictions, how many observations indeed defaulted.<\/li>\n
      • Recall: Out of all positive cases, how many were predicted correctly.<\/li>\n
      • Specificity: Measures what fraction of negative cases actually did not default.<\/li>\n
      • F-1 Score: A harmonic average of precision and recall.<\/li>\n<\/ul>\n

        The importance of understanding these metrics is critical for the proper evaluation of our model’s performance. Optimizing for a specific criteria could depend on the bank’s priorities. In terms of risk management, the bank may prefer to mitigate risk by declining more applications, as opposed to taking on riskier loans, which may result in larger losses. In this case, we would try to achieve as high recall as possible. This will achieve fewer false negatives, at the cost of more false positives. Conversely, if the bank believes it can aggressively hand out loans and still profit regardless of additional defaults, then they can aim for higher precision. This will get fewer false positives, at the cost of more false negatives. Ultimately, the metric on which we try to optimize should be selected based on the use case.<\/p>\n

        Models<\/h4>\n<\/div>\n<\/div>\n

        We will start with a basic decision tree model and follow with a more sophisticated random forest model.<\/p>\n

        BASE TREE<\/em><\/h5>\n
        \n
        \"\"\"\"<\/figure>\n<\/div>\n
        \nRANDOM FOREST<\/em>\u200b<\/h5>\n
        \n
        \"\"\"\"<\/figure>\n<\/div>\n

        Above, are the results from a base tree model and a random forest model. We see a drastic increase in accuracy and precision when we apply a more sophisticated model.<\/p>\n

        Hyperparameter Tuning<\/h3>\n

        We will apply a grid search to tune the hyperparameters of the model in order to achieve better performance. The idea is to create a grid of all possible hyperparameter combinations and train the model using each one of them. The search will help us identify the optimal parameter within the grid. <\/p>\n

        TUNED BASE TREE<\/em><\/h5>\n
        \n
        \"\"\"\"<\/figure>\n<\/div>\n
        TUNED RANDOM FOREST<\/em><\/h5>\n
        \n
        \"\"\"\"<\/figure>\n<\/div>\n

        Tuning the hyperparameters led to an increased accuracy and precision from our previous models. This was the result of optimizing our model with the help of an exhaustive grid search.\u00a0<\/p>\n

        Conclusion<\/h3>\n

        MODEL SUMMARY<\/em><\/h5>\n
        \"\"\"\"<\/figure>\n

        We decided to select the best performing decision tree model based on recall: the percentage of all defaults correctly identified by the model. This evaluation metric makes the most sense due to target imbalance i.e the ratio of our default to non-default value. To predict defaults, we decided that we could accept the cost of more false positives, in return for reducing the number of false negatives.<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

        https:\/\/nycdatascience.com\/blog\/student-works\/loan-default-detection\/<\/p>\n","protected":false},"author":0,"featured_media":7297,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/7296"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=7296"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/7296\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/7297"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=7296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=7296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=7296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}