{"id":748,"date":"2020-08-27T14:44:47","date_gmt":"2020-08-27T14:44:47","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/08\/27\/4-ways-to-improve-your-tensorflow-model-key-regularization-techniques-you-need-to-know\/"},"modified":"2020-08-27T14:44:47","modified_gmt":"2020-08-27T14:44:47","slug":"4-ways-to-improve-your-tensorflow-model-key-regularization-techniques-you-need-to-know","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/08\/27\/4-ways-to-improve-your-tensorflow-model-key-regularization-techniques-you-need-to-know\/","title":{"rendered":"4 ways to improve your TensorFlow model \u2013 key regularization techniques you need to know"},"content":{"rendered":"
\n
By Ahmad Anis<\/a>, Machine learning and Data Science Student.<\/b><\/div>\n

<\/p>\n

Photo by\u00a0Jungwoo Hong<\/a>\u00a0on\u00a0Unsplash<\/a>.<\/em><\/p>\n

Reguaralization<\/h3>\n

According to Wikipedia,<\/p>\n

\n

In mathematics, statistics, and computer science, particularly in machine learning and inverse problems,\u00a0regularization<\/strong>\u00a0is the process of adding information in order to solve an ill-posed problem or to prevent overfitting.<\/em><\/p>\n<\/blockquote>\n

This means that we add some extra information in order to solve a problem and to prevent overfitting.<\/p>\n

Overfitting simply means that our Machine Learning Model is trained on some data, and it will work extremely well on that data, but it will fail to generalize on new unseen examples.<\/p>\n

We can see overfitting in this simple example<\/p>\n

<\/p>\n

http:\/\/mlwiki.org\/index.php\/Overfitting<\/a><\/em><\/p>\n

Where our data is strictly attached to our training examples. This results in poor performance on test\/dev sets and good performance on the training set.<\/p>\n

<\/p>\n

http:\/\/mlwiki.org\/index.php\/Overfitting<\/a><\/em><\/p>\n

So in order to improve the performance of the model, we use different regularization techniques. There are several techniques, but we will discuss 4 main techniques.<\/p>\n

    \n
  1. L1 Regularization<\/strong><\/li>\n
  2. L2 Regularization<\/strong><\/li>\n
  3. Dropout<\/strong><\/li>\n
  4. Batch Normalization<\/strong><\/li>\n<\/ol>\n

    I will briefly explain how these techniques work and how to implement them in Tensorflow 2.<\/p>\n

    In order to get good intuition about how and why they work, I refer you to Professor Andrew NG lectures on all these topics, easily available on Youtube.<\/p>\n

    First, I will code a model without Regularization, then I will show how to improve it by adding different regularization techniques. We will use the IRIS data set to show that using regularization improves the same model a lot.<\/em><\/p>\n

    \u00a0<\/p>\n

    Model without Regularization<\/h3>\n

    \u00a0<\/p>\n

    Code:<\/strong><\/p>\n

    model1.summary()<\/strong><\/p>\n

    \n
    Model: \"sequential\"\r\n_________________________________________________________________\r\nLayer (type)                 Output Shape              Param #   \r\n=================================================================\r\ndense_6 (Dense)              (None, 512)               2560      \r\n_________________________________________________________________\r\ndense_7 (Dense)              (None, 256)               131328    \r\n_________________________________________________________________\r\ndense_8 (Dense)              (None, 128)               32896     \r\n_________________________________________________________________\r\ndense_9 (Dense)              (None, 64)                8256      \r\n_________________________________________________________________\r\ndense_10 (Dense)             (None, 32)                2080      \r\n_________________________________________________________________\r\ndense_11 (Dense)             (None, 3)                 99        \r\n=================================================================\r\nTotal params: 177,219\r\nTrainable params: 177,219\r\nNon-trainable params: 0\r\n_________________________________________________________________\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    After training the model, if we evaluate the model using the following code in Tensorflow, we can find our\u00a0accuracy<\/em>,\u00a0loss<\/em>, and\u00a0mse<\/em> at the test set.<\/p>\n
    \n
    loss1, acc1, mse1 = model1.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss1},nAccuracy is {acc1*100},nMSE is {mse1}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Let\u2019s check the plots for Validation Loss and Training Loss.<\/p>\n
    \n
    import matplotlib.pyplot as plt\r\nplt.style.use('ggplot')\r\nplt.plot(hist.history['loss'], label = 'loss')\r\nplt.plot(hist.history['val_loss'], label='val loss')\r\nplt.title(\"Loss vs Val_Loss\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"Loss\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Here, we can see that validation loss is gradually increasing after\u00a0\u2248\u00a0<\/strong>60 epochs as compared to training loss. This shows that our model is overfitted.<\/p>\n
    And similarly, for model accuracy plot,<\/p>\n
    \n
    plt.plot(hist.history['acc'], label = 'acc')\r\nplt.plot(hist.history['val_acc'], label='val acc')\r\nplt.title(\"acc vs Val_acc\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"acc\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    This again shows that validation accuracy is low as compared to training accuracy, which again shows signs of overfitting.<\/p>\n
    \u00a0<\/p>\n
    L1 Regularization:<\/h3>\n
    \u00a0<\/p>\n
    A commonly used Regularization technique is L1 regularization, also known as Lasso Regularization.<\/p>\n
    The main concept of L1 Regularization is that we have to penalize our weights by adding absolute values of weight in our loss function, multiplied by a regularization parameter lambda\u00a0\u03bb,\u00a0<\/strong>where\u00a0\u03bb\u00a0<\/strong>is manually tuned to be greater than 0.<\/p>\n
    The equation for L1 is<\/p>\n
    <\/p>\n
    Image Credit:\u00a0Towards Data Science<\/a>.<\/em><\/p>\n
    Tensorflow Code:<\/strong><\/p>\n
    Here, we added an extra parameter\u00a0kernel_regularizer<\/em>, which we set it to \u2018l1\u2019 for L1 Regularization.<\/p>\n
    Let\u2019s Evaluate and plot the model now.<\/p>\n
    \n
    loss2, acc2, mse2 = model2.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss2},nAccuracy is {acc2 * 100},nMSE is {mse2}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Hmmm, Accuracy is pretty much the same, let\u2019s check the plots to get better intuition.<\/p>\n
    \n
    plt.plot(hist2.history[\u2018loss\u2019], label = \u2018loss\u2019)\r\nplt.plot(hist2.history[\u2018val_loss\u2019], label=\u2019val loss\u2019)\r\nplt.title(\u201cLoss vs Val_Loss\u201d)\r\nplt.xlabel(\u201cEpochs\u201d)\r\nplt.ylabel(\u201cLoss\u201d)\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    And for Accuracy,<\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist2.history['acc'], label = 'acc')\r\nplt.plot(hist2.history['val_acc'], label='val acc')\r\nplt.title(\"acc vs Val_acc\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"acc\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Well, quite an improvement, I guess, because over validation loss is not increasing that much as it was previously, but validation accuracy is not increasing much. Let\u2019s add\u00a0l1\u00a0in more layers to check if it improves the model or not.<\/p>\n
    \n
    model3 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape, kernel_regularizer='l1'),\r\n    Dense(512\/\/2, activation='tanh', kernel_regularizer='l1'),\r\n    Dense(512\/\/4, activation='tanh', kernel_regularizer='l1'),\r\n    Dense(512\/\/8, activation='tanh', kernel_regularizer='l1'),\r\n    Dense(32, activation='relu', kernel_regularizer='l1'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel3.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist3 = model3.fit(X_train, y_train, epochs=350, batch_size=128, validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    After training, let\u2019s evaluate the model.<\/p>\n
    \n
    loss3, acc3, mse3 = model3.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss3},nAccuracy is {acc3 * 100},nMSE is {mse3}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Well, the accuracy is quite improved now, it jumped from 92 to 94. Let\u2019s check the plots.<\/p>\n
    Loss<\/strong><\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist3.history['loss'], label = 'loss')\r\nplt.plot(hist3.history['val_loss'], label='val loss')\r\nplt.title(\"Loss vs Val_Loss\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"Loss\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Now, both lines are approximately overlapping, which means that our model is performing just as same on the test set as it was performing on the training set.<\/p>\n
    Accuracy<\/strong><\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist3.history['acc'], label = 'ACC')\r\nplt.plot(hist3.history['val_acc'], label='val acc')\r\nplt.title(\"acc vs Val_acc\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"acc\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    And we can see that the validation loss of the model is not increasing as compared to training loss, and validation accuracy is also increasing.<\/p>\n
    \u00a0<\/p>\n
    L2 Regularization<\/h3>\n
    \u00a0<\/p>\n
    L2 Regularization is another regularization technique which is also known as\u00a0Ridge regularization<\/strong>. In L2 regularization we add the squared magnitude of weights to penalize our lost function.<\/p>\n
    <\/p>\n
    Image Credit:\u00a0Towards Data Science<\/a>.<\/em><\/p>\n
    Tensorflow Code:<\/strong><\/p>\n
    \n
    model5 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape, kernel_regularizer='l2'),\r\n    Dense(512\/\/2, activation='tanh'),\r\n    Dense(512\/\/4, activation='tanh'),\r\n    Dense(512\/\/8, activation='tanh'),\r\n    Dense(32, activation='relu'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel5.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist5 = model5.fit(X_train, y_train, epochs=350, batch_size=128, validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    After training, let\u2019s evaluate the model.<\/p>\n
    \n
    loss5, acc5, mse5 = model5.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss5},nAccuracy is {acc5 * 100},nMSE is {mse5}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    And the output is<\/p>\n
    <\/p>\n
    Here we can see that validation accuracy is 97%, which is quite good. Let\u2019s plot for more intuition.<\/p>\n
    <\/p>\n
    Here we can see that we are not overfitting our data. Let\u2019s plot accuracy.<\/p>\n
    <\/p>\n
    Adding \u201cL2\u201d Regularization in just 1 layer has improved our model a lot.<\/p>\n
    Let\u2019s Now add\u00a0L2<\/strong>\u00a0in all other layers.<\/p>\n
    \n
    model6 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape, kernel_regularizer='l2'),\r\n    Dense(512\/\/2, activation='tanh', kernel_regularizer='l2'),\r\n    Dense(512\/\/4, activation='tanh', kernel_regularizer='l2'),\r\n    Dense(512\/\/8, activation='tanh', kernel_regularizer='l2'),\r\n    Dense(32, activation='relu', kernel_regularizer='l2'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel6.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist6 = model6.fit(X_train, y_train, epochs=350, batch_size=128, validation_data=(X_test,y_test), verbose=2)\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    Here we have L2 in all layers. After training, let\u2019s evaluate it.<\/p>\n
    \n
    loss6, acc6, mse6 = model6.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss6},nAccuracy is {acc6 * 100},nMSE is {mse6}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Let\u2019s plot to get more intuitions.<\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist6.history['loss'], label = 'loss')\r\nplt.plot(hist6.history['val_loss'], label='val loss')\r\nplt.title(\"Loss vs Val_Loss\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"Loss\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    And for accuracy<\/p>\n
    <\/p>\n
    We can see that this model is also good and not overfitting to the dataset.<\/p>\n
    \u00a0<\/p>\n
    Dropout<\/h3>\n
    \u00a0<\/p>\n
    Another common way to avoid regularization is by using the Dropout technique. The main idea behind using dropout is that we randomly turn off some neurons in our layer based on some probability. You can learn more about it\u2019s working by Professor NG\u2019s video\u00a0here<\/a>.<\/p>\n
    Let\u2019s code it in Tensorflow.<\/p>\n
    All previous imports are the same, and we are just adding an extra import here.<\/p>\n
    In order to implement DropOut, all we have to do is to add a\u00a0Dropout\u00a0<\/em>layer from\u00a0tf.keras.layers<\/em>\u00a0and set a dropout rate in it.<\/p>\n
    \n
    import tensorflow as tf\r\nmodel7 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape),\r\n    tf.keras.layers.Dropout(0.5), #dropout with 50% rate\r\n    Dense(512\/\/2, activation='tanh'),\r\n    \r\n    Dense(512\/\/4, activation='tanh'),\r\n    Dense(512\/\/8, activation='tanh'),\r\n    Dense(32, activation='relu'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel7.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist7 = model7.fit(X_train, y_train, epochs=350, batch_size=128, validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    After training, let\u2019s evaluate it on the test set.<\/p>\n
    \n
    loss7, acc7, mse7 = model7.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss7},nAccuracy is {acc7 * 100},nMSE is {mse7}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    And wow, our results are very promising, we performed 97% on our test set. Let\u2019s plot the loss and accuracy for better intuition.<\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist7.history['loss'], label = 'loss')\r\nplt.plot(hist7.history['val_loss'], label='val loss')\r\nplt.title(\"Loss vs Val_Loss\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"Loss\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Here, we can see that our model is performing better on validation data as compared to training data, which is good news.<\/p>\n
    Let\u2019s plot accuracy now.<\/p>\n
    <\/p>\n
    And we can see that our model is performing better on the validation dataset as compared to the training set.<\/p>\n
    Let\u2019s add more dropout layers to see how our model performs.<\/p>\n
    \n
    model8 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape),\r\n    tf.keras.layers.Dropout(0.5),\r\n    Dense(512\/\/2, activation='tanh'),\r\n    tf.keras.layers.Dropout(0.5),\r\n    Dense(512\/\/4, activation='tanh'),\r\n    tf.keras.layers.Dropout(0.5),\r\n    Dense(512\/\/8, activation='tanh'),\r\n    tf.keras.layers.Dropout(0.3),\r\n    Dense(32, activation='relu'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel8.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist8 = model8.fit(X_train, y_train, epochs=350, batch_size=128, validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    Let\u2019s Evaluate it.<\/p>\n
    \n
    loss8, acc8, mse8 = model8.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss8},nAccuracy is {acc8 * 100},nMSE is {mse8}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    This model is also very good, as it is performing 98% on the test set. Let\u2019s plot to get better intuitions.<\/p>\n
    \n
    plt.figure(figsize=(15,8))\r\nplt.plot(hist8.history['loss'], label = 'loss')\r\nplt.plot(hist8.history['val_loss'], label='val loss')\r\nplt.title(\"Loss vs Val_Loss\")\r\nplt.xlabel(\"Epochs\")\r\nplt.ylabel(\"Loss\")\r\nplt.legend()\r\nplt.show()\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    And we can see that adding more dropout layers makes the model perform slightly less good while training, but on the validation set, it is performing really well.<\/p>\n
    Let\u2019s plot the accuracy now.<\/p>\n
    <\/p>\n
    And we can see the same pattern here, that our model is not performing as good while training, but when we are evaluating it, it is performing really good.<\/p>\n
    \u00a0<\/p>\n
    Batch Normalization<\/h3>\n
    \u00a0<\/p>\n
    The main idea behind batch normalization is that we normalize the input layer by using several techniques (sklearn.preprocessing.StandardScaler<\/em>) in our case, which improves the model performance, so if the input layer is benefitted by normalization, why not normalize the hidden layers, which will improve and fasten learning even further.<\/p>\n
    To learn maths and get more intuition about it, I will redirect you again to Professor NG\u2019s lecture\u00a0here<\/a>\u00a0and\u00a0here<\/a>.<\/p>\n
    To add it in your TensorFlow model, just add\u00a0tf.keras.layers.BatchNormalization()<\/em>\u00a0after your layers.<\/p>\n
    Let\u2019s see the code.<\/p>\n
    \n
    model9 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape),\r\n    Dense(512\/\/2, activation='tanh'),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(512\/\/4, activation='tanh'),\r\n    Dense(512\/\/8, activation='tanh'),\r\n    Dense(32, activation='relu'),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel9.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist9 = model9.fit(X_train, y_train, epochs=350,  validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    Here, if you noticed that I have removed the option for batch_size<\/em>. This is because adding\u00a0batch_size\u00a0argument while only using\u00a0tf.keras.BatchNormalization()<\/em>\u00a0as regularization, this results in a really poor performance of the model. I have tried to find the reason for this on the internet, but I could not find it. You can also change the optimizer from\u00a0sgd\u00a0<\/em>to\u00a0rmsprop\u00a0<\/em>or\u00a0adam\u00a0<\/em>if you really want to use\u00a0batch_size\u00a0while training.<\/p>\n
    After training, let\u2019s evaluate the model.<\/p>\n
    \n
    loss9, acc9, mse9 = model9.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss9},nAccuracy is {acc9 * 100},nMSE is {mse9}\")\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    Validation accuracy for 1 Batch Normalization accuracy is not as good as compared to other techniques. Let\u2019s plot the loss and acc for better intuition.<\/p>\n
    <\/p>\n
    <\/p>\n
    Here we can see that our model is not performing as well on validation set as on test set. Let\u2019s add normalization to all the layers to see the results.<\/p>\n
    \n
    model11 = Sequential([\r\n    Dense(512, activation='tanh', input_shape = X_train[0].shape),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(512\/\/2, activation='tanh'),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(512\/\/4, activation='tanh'),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(512\/\/8, activation='tanh'),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(32, activation='relu'),\r\n    tf.keras.layers.BatchNormalization(),\r\n    Dense(3, activation='softmax')\r\n])\r\nmodel11.compile(optimizer='sgd',loss='categorical_crossentropy', metrics=['acc', 'mse'])\r\nhist11 = model11.fit(X_train, y_train, epochs=350,  validation_data=(X_test,y_test), verbose=2)\r\n\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    Let\u2019s Evaluate it.<\/p>\n
    \n
    loss11, acc11, mse11 = model11.evaluate(X_test, y_test)\r\nprint(f\"Loss is {loss11},nAccuracy is {acc11 * 100},nMSEis {mse11}\")\r\n<\/pre>\n<\/div>\n
    \u00a0<\/p>\n
    <\/p>\n
    By adding Batch normalization in every layer, we achieved good accuracy. Let\u2019s plot the loss and accuracy.<\/p>\n
    <\/p>\n
    <\/p>\n
    By plotting accuracy and loss, we can see that our model is still performing better on the Training set as compared to the validation set, but still, it is improving in performance.<\/p>\n
    \u00a0<\/p>\n
    Outcome:<\/h3>\n
    \u00a0<\/p>\n
    This article was a brief introduction on how to use different techniques in Tensorflow. If you lack the theory, I would suggest Course 2 and 3 of Deep Learning Specialization at Coursera to learn more about Regularization.<\/p>\n
    You also have to learn when to use which technique, and when and how to combine different techniques in order to produce really fruitful results.<\/p>\n
    Hopefully, now you have an idea of how to implement different regularization techniques in Tensorflow 2.<\/p>\n
    \u00a0<\/p>\n
    Related:<\/b><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
    https:\/\/www.kdnuggets.com\/2020\/08\/tensorflow-model-regularization-techniques.html<\/p>\n","protected":false},"author":0,"featured_media":749,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/748"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=748"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/748\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media\/749"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=748"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=748"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=748"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}