In recent years, , , and have become buzzwords. For those who have been working in these fields for awhile, this is somewhat amusing and occasionally even gratifying. I certainly don’t view it as a bad thing that these areas of technology are receiving more widespread attention. With this increased attention also comes the need to explain these concepts to the masses, which is a noble challenge to tackle. artificial intelligence machine learning deep learning Nonetheless, there is one recent trend that has a similar effect on me as fingernails scratching a blackboard: Come and learn the difference between machine learning and deep learning! Aaaaaah!!!!!!! Shivers down my spine!!! Take this tweet for example: Or this one: Or one of my all-time “favorites”: One would think that the VP of AI and deep learning at IBM, Sumit Gupta, would be a reliable source on the matter, but sadly, he is leading his audience astray. Deep learning is not something inherently from machine learning; it is, in fact, one of the latest innovations the field of machine learning that actually brings us to the original goals for which machine learning pioneers had strived. Let us briefly review those goals. different in closer Machine Learning Roots As I recounted in , machine learning is not a particularly new idea. The fundamental idea was already articulated by Alan Turing as early as 1947: an earlier article What we want is a machine that can learn from experience. This basic desire has been re-articulated over the years by several generations of machine learning researchers, such as , , , and . Consider this passage, published in 1959, by Arthur Samuel: Arthur Samuel Claude Shannon Nils Nilsson Tom Mitchell At the outset it might be well to distinguish sharply between two general approaches to the problem of machine learning. One method, which might be called the Neural-Net Approach, deals with the possibility of inducing learned behavior into a randomly connected switching net (or its simulation on a digital computer) as a result of a reward-and-punishment routine. For those familiar with deep learning, this should sound strikingly familiar. In fact, the basic ideas in artificial neural networks were already established in the 1940s. These ideas were discussed and studied widely in the machine learning community over many decades, but at some point they fell out of fashion. Feature Engineering A curious thing happened in the field of machine learning sometime around the 1980s. I haven’t been able to pinpoint exactly when it occurred, but over time, more and more effort was exerted by researchers in the task of coming up with clever “features” to serve as inputs to machine learning algorithms. For example, instead of feeding the raw pixel values of images into a image recognition algorithm, fancy “pre-processing” algorithms were devised to detect edges, corners, textures, and other higher-level features from the images. These would then serve as the input features into the recognition algorithms. This somewhat laborious process is essentially what is depicted in Mr. Gupta’s slide with the heading “Machine Learning”, shown above. This process, which is generally known as also tends to be application specific. That is, features that work well in, say, computer vision tasks are usually completely useless in other tasks, such as activity recognition from motion sensors. feature engineering, In many cases, the number of available features might even become large and unwieldy. In comes the idea of New algorithms had to be devised just to figure out which are the best features to use for a given machine learning task. feature selection. This emphasis on feature engineering and feature selection, however, is not what machine learning. If anything, an over-sized focus on these areas is a distraction from the ultimate aim of machine learning, which is to develop computer systems that learn from experience. This is because in feature engineering the researcher or engineer inserts himself or herself into the equation, trying to optimize the learning process with clever new ideas for features (or deciding on sets of features). defines automatically Deep Learning to the Rescue I won’t go into the nitty-gritty of what deep learning is and how it works in detail; there are plenty of good resources on that topic, such as Michael Nielsen’s excellent online book . Instead I just want to point out how deep learning, in many ways, rectifies the above issues and restores machine learning closer to its original goals. Neural Networks and Deep Learning Instead of coming up with elaborate feature sets, deep learning largely works with high-dimensional “raw data” (e.g. pixels in an image). For sure, deep learning still employs application-specific tricks, such as the idea of used mostly in image recognition. In fact, one could argue that restricting a neural network in such fashion is a form of feature engineering. But even so, these techniques are considerably more general than features, such as edge detectors from images or “variance of the dynamic acceleration in the horizontal plane” from accelerometer data. The basic idea is to suck whatever data you have into a large and deep neural network and let the learning algorithm do the hard work. Lack of expressiveness in features is made up for with an abundance of data and multiple hidden layers in the network that automatically learn the best features. convolutional neural networks ad hoc By all measures, however, this is still what we call machine learning. Setting aside “deep learning” as something different from “machine learning” is just bad pedagogy. It reveals a lack of understanding for what the goal of machine learning has been ever since the concept was first formulated. The relationship between the two is best expressed in the diagram below. Deep learning falls under the general umbrella of machine learning, which is itself part of the wider picture of artificial intelligence. Deep learning is best understood as a subset of techniques in the realm of machine learning, which itself is a subset of techniques used to build artificial intelligence. ( Robert Guinness) CC BY-SA 4.0 In summary, when you are impressing your hot date by describing what deep learning is all about, try not to make the mistake of saying it’s something completely new and way cooler than that old school machine learning stuff. And by all means, stop retweeting those nonsensical tweets pitting machine learning deep learning. For in reality, deep learning is one of machine learning’s long lost friends. against
