{"id":2124,"date":"2020-09-29T03:45:39","date_gmt":"2020-09-29T03:45:39","guid":{"rendered":"https:\/\/data-science.gotoauthority.com\/2020\/09\/29\/popular-activation-functions-in-neural-networks\/"},"modified":"2020-09-29T03:45:39","modified_gmt":"2020-09-29T03:45:39","slug":"popular-activation-functions-in-neural-networks","status":"publish","type":"post","link":"https:\/\/wealthrevelation.com\/data-science\/2020\/09\/29\/popular-activation-functions-in-neural-networks\/","title":{"rendered":"Popular Activation Functions In Neural Networks"},"content":{"rendered":"
\n
\"Activation\"Activation<\/span><\/div>\n
\n

In the neural network introduction<\/a> article, we have discussed the basics of neural networks. This article focus is on different types<\/strong> of activation functions using in building neural networks.\u00a0<\/p>\n

In the deep learning literate or in neural network online courses<\/a>, these activation functions are popularly called transfer functions<\/strong>.<\/p>\n

The main focus of this article is to give you a complete overview of various activation functions and their properties. We\u2019ll also see how to implement them in python.<\/p>\n<\/div>\n

\n
\n
\n

Popular activation functions in neural networks.<\/p>\n<\/div>\n

\n\t\t\t
\n\t\t\t\t<\/i>
\n\t\t\t\tClick to Tweet<\/span>
\n\t\t\t<\/span>\n\t\t<\/p>\n<\/div>\n<\/div>\n

\n

Before we drive further, Let\u2019s see the topic you are going to learn in this article.<\/p>\n<\/div>\n

\n

So let\u2019s begin with understanding what is the activation function. If you remember decision tree, at each node the decision tree algorithm<\/a> needs to take decision to split the further data, we can related this to understand about activation functions.<\/p>\n

\n
What is Activation Function?<\/h2>\n

The name activation is self explainable. As the name suggests, the activation function is to alert or fire<\/strong> the neurons\/node in neural networks.<\/p>\n

If we treat these functions as a black box, like we treat many classifiction algorihtms<\/a> , these functions will take input and return output, which helps the neural network to pass the value to the next nodes of the network.<\/p>\n

Activation functions are vital components in the neural networks, which helps the network to learn the intricate patterns<\/strong> in train data, which helps in predicting the future.<\/p>\n

In mathematical words, activation functions are used in neural networks to compute the weighted sum of input and biases<\/strong>, which is used to decide if a neuron can be fired or not.\u00a0<\/p>\n

We can related the computing the weighted sum with the linear regression concept<\/a>.<\/p>\n

It manipulates the presented data through some gradient<\/strong> processing, usually gradient descent, and afterward produces an output for the neural network that contains the parameters in the data.\u00a0<\/p>\n

Activation functions are often referred to as a transfer function<\/strong> in deep learning research papers literature. These activation functions are having a set of properties<\/strong> to follow.\u00a0<\/p>\n

Let\u2019s discuss this.<\/p>\n

Properties of activation functions<\/h2>\n<\/div>\n
\n\"Properties\"Properties<\/span><\/p>\n

Properties of activation functions<\/p>\n<\/div>\n

\n

These functions having the set of properties to follow,<\/p>\n