Deep learning dominates machine learning and artificial intelligence studies nowadays. Even though some people might think it as overrated, pioneer scientists believe it as unrivaled for today. For instance, Geoffrey Hinton, Google Brain team member, thinks that deep learning has not theoretical limitations of what it can learn. Similarly, Andrew Ng who has worked all his life in machine learning mentioned that he has never seen one algorithm knocks over its benchmarks like deep learning. Deep learning is basically just deep and wide neural networks. Deep refers to number of layers typically as Jeff Dean described. However, neural networks are already mentioned as one of the most powerful algorithm in machine learning field in the past. We have mentioned that developments in hardware is dominant factor to accelerate deep learning adaption. In this post, we are going to mention what causes to create deep learning concept in software perspective.
You might be familiar with Iris flower data set. Iris flower has 3 different species: setosa, virginica and versicolor. A British statistician believes that size of the flower correlated with the species. He measured the top and bottom of flower as length and width for 150 instances.
🙋♂️ You may consider to enroll my top-rated machine learning course on Udemy
Neural networks phase
A researcher is already measured top of flower (petal) and bottom of flower (sepal). We will construct a neural networks model and feed these four measurements as inputs. The flower has 3 different species and that is the one we would like to learn. That’s why, there are 3 output nodes in our model. Finally, we need to create at least a hidden layer in neural networks. The most common method to decide number of hidden nodes is average of input and output nodes. I will put 4 nodes in single hidden layer.
Let’s consturct this Keras model (mine is TensorFlow backend) in Python.
model = Sequential()
model.add(Dense(4 #num of hidden units , input_shape=(4),))) #num of features in input layer
model.add(Activation('sigmoid')) #activation function from input layer to 1st hidden layer
model.add(Dense(3)) #num of classes in output layer
model.add(Activation('sigmoid')) #activation function from 1st hidden layer to output layer
This model can learn 147/150 items quickly when we feed the dataset to this model. We got 98% accuracy, very satisfactory. You can find code of this implementation here and the raw dataset here (iris-attr.data and iris-labels.data).
This means that there is really correlation between flower size and species for Iris dataset. If you see an Iris flower and measure its size, then you can predict its species. Here, there is a huge problem you might sense. Firstly, how to collect train set. A researcher is already measured flower sizes for Iris but what about other flowers? You have to measure one by one and that is really a problem. Secondly, even though you have a train data, you have to measure every time you want to predict the species of a flower. This is a legacy approach. What if you take a photo of a flower and an application respond you its species? Wouldn’t it be more satisfactory?
Deep learning phase
The key point is taking photo in the previous sentence. We have extracted features at neural networks times. This is luxury now. We expect deep neural networks to extract features. Suppose that you collect images for 150 instances of Iris flower. These images can be expressed as pixels. You can feed pixel values of an image as input features to neural networks. Even a 25×25 pixel image is fed 625 input features (ignored RGB codes). Output layer remains same and it consists of 3 nodes because still there are 3 species of Iris flower. Moreover, you must increase the number of hidden layers. We could handle just using a hidden layer in our previous example, but here you must construct tens or hundreds of hidden layers.
In this way, you can collect samples easier than neural networks days. Furthermore, it is enough to take photo of new Iris flower to predict if your model is ready. This is more practical. You just need to feed data to develop a deep learning model. The data can be anything such as image, voice or just signal. We no longer interested in feature extraction. This is basically called deep learning or deep neural networks.
You might check this post. We just feed handwritten digit images and deep learning model learns these digits and says unseen ones successfully.
So, we have mentioned the conventional form of deep neural networks and focused on modifications of neural networks in timeline. Deep learning continues to improve everyday. Such an extent that number of research papers about it increase faster than Moore’s law. We can reduce the computational power and increase the accuracy in some advanced forms such as CNN.
Like this blog? Support me on Patreon
