Activation function

Activation Function

At the exit of the neuron, there may be a filter, limiting function or threshold, which modifies the result value or imposes a limit that must be exceeded to continue to another neuron. This function is known as the activation function. An activation function is, therefore, a function which transmits the information generated by the linear combination of weights and inputs, i.e. how the information is transmitted through the output connections. The information can be transmitted without modification, identity function, or it cannot transmit the information. As the aim is for the neural network to be able to solve increasingly complex problems, the activation functions will generally make the models non-linear. Among the best known or most widely used activation functions are

• Step function, (similar to the binary function.)
• Formula of the step function
• Sigmoid function.
• Formula of the Sigmoid function
• Rectifier function (ReLU).
• Formula of the rectifier function
• Hyperbolic Tangent function.
• Formula of the hyperbolic tangent function
• Radial Base Functions. (Gaussian, multi quadratic, inverse multi quadratic...)