If you have read even the most basic news about Machine Learning or Artificial Intelligence, you would have surely heard of the term Deep Learning. As per Google Trends, Deep Learning has now reached a world-wide interest level of 100, which means that it is highly popular and widely being searched for across the world.
In this blog post, we will demystify the term “Deep Learning” and try to understand what is Deep Learning and how it works. Shall we start?
The Cooking analogy
When you are solving a machine learning problem, you are essentially trying to make the computer identify trends in data so that it can make predictions on new, unseen data. Think of this as “cooking food”.
Now, you might want to solve a text processing problem, or a computer vision problem, or maybe voice recognition problem — all of these are machine learning problems. You can think of them as cooking Italian food vs cooking Mexican food vs cooking Indian food.
The idea is simple – in order to cook a food, you may use various ways:
- Cook using induction
- Cook using a stove
- Order it from outside!
Similarly, to solve a machine learning problem, you may want to use multiple techniques or algorithms. One such algorithm is Deep Learning. Get the idea?
So, deep learning is simply one of the many algorithms to solve a machine learning problem. It has wide applicability and can be applied in almost all domains of machine learning.
What is so special about Deep Learning?
In order to understand this, let us first understand how a typical machine learning algorithm works. In any machine learning algorithm, the input is fed to the algorithm in the form of a feature vector which is nothing but an array of numbers. The algorithm processes this feature vector and returns the output as shown in the image below.
The feature vector is a simple array of numbers. However, there is something very special about the feature vector — it is a numerical representation of the input. For instance, in a face recognition task, one might want to choose a feature vector that is composed of the following features:
- Average color of skin tone – this can be fed as 3 features: Red, Green and Blue (the RGB colors).
- Height of face – this can be fed as a numerical value in inches.
- Width of face – this is similar to height of face.
- Width of eyes
- Height of nose
Clearly, these features (along with possibly many other features) when analyzed carefully can be used to distinguish one face with another and so, it can be fed to a machine learning algorithm that processes these features and comes up with a numerical score.
The challenge here is that creating such feature vectors is tough. Also, one needs to have heavy domain knowledge in order to create such feature vectors. For instance, one needs to know that height of nose is a feature that can indeed distinguish between two features. In other domains, it might be complex.
Deep Learning solves this complex problem of generation of feature vector from the input data. You can simply feed in the input in any vectorized form and the algorithm will itself generate complex features out of the input. As an example, in the above face recognition problem, you can feed in the image pixels as a vector of RGB values and the Deep Learning algorithm will itself figure out the right set of features like the ones mentioned above.
How does it work?
Deep Learning generates these complex feature vectors using a sequence of non-linear transformations. The input data is fed to complex functions that provide non-linearity, thus helping the algorithm in the generation of “different” types of features.
As an example, one of the most well known nonlinear transformation is the sigmoid transformation:
What is “deep” in Deep learning?
The transformations like the ones mentioned above are applied repeatedly to the input data across various layers of the algorithm and so, the algorithm literally comes in the shape of a “deep” net. This is the reason it is called as Deep Learning.
The image above represents a 3 layer deep neural network.
Future of Deep Learning
Deep Learning has proven to be an invaluable tool in the machine learning domain. It has broken multiple records to provide better accuracy than almost any other algorithm. Heavy research is going on in deep learning in both industry and academia, both. Today, tech giants like Google, Microsoft, Apple, Facebook, and Amazon are all moving their machine learning and AI backends to Deep Learning to create the most intelligent products for consumers. This is likely to continue in future until another breakthrough happens.