Activation functions play pivotal role in neural networks. As an alternative to hyperbolic tangent, softsign is an activation function for … More
Tag: neural networks
Softmax as a Neural Networks Activation Function
In fact, convolutional neural networks popularize softmax so much as an activation function. However, softmax is not a traditional activation function. … More
A Gentle Introduction to Convolutional Neural Networks
Convolutional neural networks (aka CNN and ConvNet) are modified version of traditional neural networks. These networks have wide and deep … More
Evolution of Neural Networks
Today, AI lives its golden age whereas neural networks make a great contribution to it. Neural networks change our lifes … More
ReLU as Neural Networks Activation Function
Rectifier linear unit or its more widely known name as ReLU becomes popular for the past several years since its … More
Softplus as a Neural Networks Activation Function
Activation unit calculates the net output of a neural cell in neural networks. Backpropagation algorithm multiplies the derivative of the … More
Transparency in AI
Until now, authorities describe machine learning algorithms based on the learning style which are supervised or unsupervised. Rob Walker brings … More
Homer Simpson Guide to Backpropagation
Backpropagation algorithm is based on complex mathematical calculations. That’s why, it is hard to understand and that is the … More
Step Function as a Neural Network Activation Function
Activation functions are decision making units of neural networks. They calculates net output of a neural node. Herein, heaviside step … More
Adaptive Learning in Neural Networks
Gradient descent is one of the most powerful optimizing method. However, learning time is a challange, too. Standard version of gradient … More
