{"id":2088,"date":"2020-09-28T17:20:54","date_gmt":"2020-09-28T17:20:54","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/09\/28\/looking-inside-the-blackbox-how-to-trick-a-neural-network\/"},"modified":"2020-09-28T17:20:54","modified_gmt":"2020-09-28T17:20:54","slug":"looking-inside-the-blackbox-how-to-trick-a-neural-network","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/09\/28\/looking-inside-the-blackbox-how-to-trick-a-neural-network\/","title":{"rendered":"Looking Inside The Blackbox: How To Trick A Neural Network"},"content":{"rendered":"
\n

By William Falcon<\/a>, Founder PyTorch Lightning<\/b><\/p>\n

\"Image<\/p>\n

Using gradient ascent to figure out how to change an input to be classified as a 5. (All images are the author\u2019s own with all rights reserved).<\/p>\n

Neural networks get a bad reputation for being black boxes. And while it certainly takes creativity to understand their decision making, they are really not as opaque as people would have you believe.<\/p>\n

In this tutorial, I\u2019ll show you how to use backpropagation to change the input as to classify it as whatever you would like.<\/p>\n

Follow along using this\u00a0colab<\/a>.<\/p>\n

(This work was co-written with\u00a0Alfredo Canziani<\/a>\u00a0ahead of an upcoming video)<\/p>\n

\u00a0<\/p>\n

Humans as black boxes<\/h3>\n

\u00a0
Let\u2019s consider the case of humans. If I show you the following input:<\/p>\n

\"Image<\/p>\n

there\u2019s a good chance you have no idea whether this is a 5 or a 6. In fact, I believe that I could even make a case for convincing you that this\u00a0might<\/em>\u00a0also be an 8.<\/p>\n

Now, if you asked a human what they would have to do to make something more into a 5 you might visually do something like this:<\/p>\n

\"Image<\/p>\n

And if I wanted you to make this more into an 8, you might do something like this:<\/p>\n

\"Image<\/p>\n

Now, the answer to this question is not easy to explain in a few if statements or by looking at a few coefficients (yes, I\u2019m looking at you regression). Unfortunately, with certain types of inputs (images, sound, video, etc\u2026) explainability certainly becomes much harder\u00a0but not impossible<\/strong>.<\/p>\n

\u00a0<\/p>\n

Asking the neural network<\/h3>\n

\u00a0
How would a neural network answer the same questions I posed above? Well, to answer that, we can use gradient ascent to do exactly that.<\/p>\n

Here\u2019s how the neural network thinks we would need to modify the input to make it more into a 5.<\/p>\n

\"Image<\/p>\n

There are two interesting results from this. First, the black areas are where the network things we need to remove pixel density from. Second, the yellow areas are where it thinks we need to add more pixel density.<\/p>\n

We can take a step in that gradient direction by adding the gradients to the original image. We could of course repeat this procedure over and over again to eventually morph the input into the prediction we are hoping for.<\/p>\n

\"Image<\/p>\n

You can see that the black patch at the bottom left of the image is\u00a0very similar\u00a0<\/strong>to what a human might think to do as well.<\/p>\n

\n\"Figure\"
<\/p>\n

Human adds black on the left corner. Network suggests the same<\/p>\n

<\/span>\n<\/div>\n

\u00a0<\/p>\n

What about making the input look more like an 8? Here\u2019s how the network thinks you would have to change the input.<\/p>\n

\"Image<\/p>\n

The notable things, here again, are that there is a black mass at the bottom left and a bright mass around the middle. If we add this with the input we get the following result:<\/p>\n

\"Image<\/p>\n

In this case, I\u2019m not particularly convinced that we\u2019ve turned this 5 into an 8. However, we\u2019ve made less of a 5, and the argument to convince you this is an 8 would certainly be easier to win using the image on the right instead of the image on the left.<\/p>\n

\u00a0<\/p>\n

Gradients are your guides<\/h3>\n

\u00a0
In regression analysis, we look at coefficients to tell us about what we\u2019ve learned. In a random forest, we can look at decision nodes.<\/p>\n

In neural networks, it comes down to how\u00a0creative<\/strong>\u00a0we are at using gradients. To classify this digit, we generated a distribution over possible predictions.<\/p>\n

This is what we call the\u00a0forward pass.<\/em><\/p>\n

\n\"Figure\"
<\/p>\n

During the forward pass we calculate a probability distribution over outputs<\/p>\n

<\/span>\n<\/div>\n

\u00a0<\/p>\n

In code it looks like this (follow along using this colab<\/a>):<\/p>\n

\"Image<\/p>\n

Now imagine that we wanted to trick the network into predicting \u201c5\u201d for the input x. Then the way to do this is to give it an image (x), calculate the predictions for the image and then\u00a0maximize<\/strong>\u00a0the probablitity of predicting the label \u201c5\u201d.<\/p>\n

To do this we can use gradient ascent to calculate the gradients of a prediction at the 6th index (ie: label = 5) (p<\/strong>) with respect to the input\u00a0x<\/strong>.<\/p>\n

\"Image<\/p>\n

To do this in code we feed the input x as a parameter to the neural network, pick the 6th prediction (because we have labels: 0, 1, 2, 3, 4 , 5, \u2026) and the 6th index means label \u201c5\u201d.<\/p>\n

Visually this looks like:<\/p>\n

\n\"Figure\"
<\/p>\n

Gradient of the prediction of a \u201c5\u201d with respect to the input.<\/p>\n

<\/span>\n<\/div>\n

\u00a0<\/p>\n

And in code:<\/p>\n

\"Image<\/p>\n

When we call .backward() the process that happens can be visualized by the previous animation.<\/p>\n

Now that we calculated the gradients, we can visualize and plot them:<\/p>\n

\"Image<\/p>\n

\"Image<\/p>\n

The above gradient looks like random noise because the network has not yet been trained\u2026 However, once we do train the network, the gradients will be more informative:<\/p>\n

\"Image<\/p>\n

\u00a0<\/p>\n

Automating this via Callbacks<\/h3>\n

\u00a0
This is a hugely helpful tool in helping illuminate what happens inside your network as it trains. In this case, we would want to automate this process so that it happens automatically in training.<\/p>\n

For this, we\u2019ll use PyTorch Lightning to implement our neural network:<\/p>\n

The complicated code to automatically plot what we described here, can be abstracted out into a Callback in Lightning. A callback is a small program that is called at the parts of training you might care about.<\/p>\n

In this case, when a training batch is processed, we want to generate these images in case some of the inputs are confused.<\/p>\n

But… we\u2019ve made it even easier with pytorch-lightning-bolts which you can simply install<\/p>\n

\n
pip install pytorch-lightning-bolts<\/code><\/pre>\n<\/div>\n
and import the callback into your training code<\/p>\n
\u00a0<\/p>\n
Putting it all together<\/h3>\n
\u00a0
Finally we can train our model and automatically generate images when logits are \u201cconfused\u201d<\/p>\n
and tensorboard will automatically generate images that look like this:<\/p>\n
\"Image<\/p>\n
\"Image<\/p>\n
\u00a0<\/p>\n
Summary<\/h3>\n
\u00a0
\"Image<\/p>\n
In summary: You learned how to look inside the blackbox using PyTorch, learned the intuition, wrote a callback in PyTorch Lightning and automatically got your Tensorboard instance to plot questionable predictions<\/p>\n
Try it yourself with\u00a0PyTorch Lightning<\/a>\u00a0and\u00a0PyTorch Lightning Bolts<\/a>.<\/p>\n
(This article was written ahead of an upcoming video where me (William) and\u00a0Alfredo Canzian<\/a>i show you how to code this from scratch).<\/p>\n
\u00a0
Bio: William Falcon<\/a><\/b> is an AI Researcher, and Founder at PyTorch Lightning. He is trying to understand the brain, build AI and use it at scale.<\/p>\n
Original<\/a>. Reposted with permission.<\/p>\n
Related:<\/b><\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"
https:\/\/www.kdnuggets.com\/2020\/09\/inside-blackbox-trick-neural-network.html<\/p>\n","protected":false},"author":0,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[2],"tags":[],"_links":{"self":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/2088"}],"collection":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/comments?post=2088"}],"version-history":[{"count":0,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/posts\/2088\/revisions"}],"wp:attachment":[{"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/media?parent=2088"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/categories?post=2088"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wealthrevelation.com\/data-science\/wp-json\/wp\/v2\/tags?post=2088"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}