Will Generative Models Be The Next Machine Learning Boom?

Recent AI advances are considerably bigger than the ‘AI industry;’ these innovations will drastically alter the planet. We’ve come a long way from the days when Google Brain could find kitten videos and the release of FaceNet, which can recognize faces with 95% accuracy, to tools like AutoML, which can self-design neural networks, and Waymo’s self-driving cars driving around San Francisco’s streets.

To the majority of us, AI has always been about comprehending the world. Deep learning and convolutional neural networks are allowing computers to see images and frames in videos in the same way that humans do.

However, that is only half of the work done.

Humans not only understand the world; they make it. Humans create dialogue, language, art, music, things, religion, and code, among other things. For computers to genuinely be artificial intelligences, they must not only understand but also create.

The solution to these generational issues is Generative models, which constitute a big breakthrough.

Scope of this blog

This blog will give a comprehensive overview of the area of generative modelling. We’ll look at what it means to call a model generative and how it varies from the more commonly researched discriminative modelling. We’ll also look at the evolution of generative models across time, as well as the specifics of how they work. Following that, we’ll define and introduce many forms of generative models.

Machine learning models can be classified into two types based on how they work: Generative modelling and Discriminative modelling. In simple terms, a discriminative model uses conditional probability to generate predictions on unknown data and can be used to solve classification or regression problems. A generative model, on the other hand, concentrates on the distribution of a dataset in order to return a probability for a given occurrence.

I understand that all of this technical terminology is difficult to understand at first look, but don’t worry, the next section will make everything clear and you will understand everything completely.

Let’s start with Discriminative models!

What Is Discriminative Modeling?

If you’ve studied machine learning, you’ll know that the majority of the challenges you’ve encountered have been discriminative (able to recognize or make distinctions with accuracy) in nature. Let’s look at an example to better grasp the discriminative model.

First, we’ll need a dataset with a large number of samples of the thing we’re trying to create. This is referred to as the training data, and each data point is referred to as an observation. Each observation is made up of a number of features, which in the case of an image production problem are usually the individual pixel values.

Assume we have a Cats and Dogs dataset. We might train a discriminative model to predict whether a given image is of a cat or a dog. Our model would learn that particular colors, forms, and textures are more likely to reveal which animal is the mage, and it would upweight its forecast for images with these features. Note how the discriminative modelling method is depicted in Figure 1–1. — note how it differs from the generative modeling process shown in Figure 1–2.

When performing discriminative modelling, each observation in the training data has a label, which is a significant difference. Cat images would be labelled 1 in a binary classification issue like our animal classification, whereas non–Cat images, such as Dog images, would be labelled 0. The likelihood that a new observation has label 1 — that is, that it is the image of a Cat — is then outputted by our model, which has learned to discriminate between these two groups.

Some Examples of Discriminative Models

• ‌Logistic regression.
• Scalar Vector Machine (SVMs)
• ‌Traditional neural networks.
• ‌Nearest neighbor.
• Conditional Random Fields (CRFs)
• Decision Trees and Random Forest.

What Is Generative Modeling?

The term “generative” refers to a type of statistical model that differs from discriminative models.

The following is a broad definition of a generative model:

In terms of a probabilistic model, a generative model specifies how a dataset is formed. We can produce new data by sampling from this model.

Assume we have a dataset with images of cats. We might want to create a model that can create a fresh image of a cat that has never existed but still appears real because the model has learned the general rules that control a cat’s appearance. This is the type of problem that generative modelling can tackle. Figure 1–2 depicts a summary of a typical generative modelling method.

Our goal is to design a model that can generate new sets of features (picture pixels) that appear to be generated according to the same principles as the original data. Given the large variety of ways that individual pixel values can be assigned and the comparatively small number of such arrangements that make up an image of the item we’re trying to emulate, this is a conceptually challenging issue for image production.

In addition, rather of being deterministic, a generative model must be probabilistic. Our model is not generative if it is simply a fixed calculation, such as taking the average value of each pixel in the dataset. The model gives the same output every time. A stochastic (random) element must be included in the model to alter the individual samples generated by the model.

To put it another way, we can consider that there is some unknown probability distribution explaining why some images are likely to be found in the training dataset while others are not. It’s our duty to create a model that as nearly as possible resembles this distribution, then sample from it to generate new, different observations that appear to have come from the original training set.

Generative Versus Discriminative Modeling

Discriminative modelling is synonymous with supervised learning, or learning a function that translates an input to an output using a labelled dataset, for the purposes of the work indicated above. The most common application of generative modelling is with an unlabeled dataset (i.e., unsupervised learning), but it may also be used with a labelled dataset to learn how to produce observations from each separate class.

Informally:

• Generative models are capable of producing new data instances.
• Models that are discriminative distinguish between distinct types of data instances.

A discriminative model might identify a dog from a cat, while a generative model could generate fresh images of animals that seem like genuine animals. Generative models, such as GANs, is example of generative model.

Given a set of data instances X and a set of labels Y, in more formal terms:

• Generative models capture the p(X, Y) joint probability, or just p(X) if no labels are present.
• The conditional probability p(Y | X) is captured by discriminative models.

A generative model takes into account the data’s distribution and informs you how likely a given occurrence is. Because they can assign a probability to a succession of words, models that predict the next word in a sequence are often generative models (far simpler than GANs).

A discriminative model avoids the question of whether or not a given event is likely, instead focusing on the likelihood of a label being applied to it.

Upsurge of Generative Models

Let’s take a look at the rise of Generative Models and how they’ve evolved over the years.

2014

Using GANs to generate new plausible examples for the MNIST handwritten digit dataset, the CIFAR-10 small object photograph dataset, and the Toronto Face Database was the application described in the original paper by Ian Goodfellow, et al. in the 2014 paper “Generative Adversarial Networks,” in which GANs were used to generate new plausible examples for the MNIST handwritten digit dataset, the CIFAR-10 small object photograph dataset, and the Toronto Face Database.

2017

Tero Karras et al. exhibit the development of believable realistic images of human faces in their 2017 publication “Progressive Growing of GANs for Improved Quality, Stability, and Variation.” They’re so lifelike, in fact, that the end product is rather astounding. As a result, the findings drew a lot of media attention.

2018

Examples from this paper were used in a 2018 report titled “The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation ” to demonstrate the rapid progress of GANs from 2014 to 2017 (found via this tweet by Ian Goodfellow).

Going forward in the Future

There will be hundreds of sectors where generative techniques will enrich our surroundings in the next years. In contrast to many current AI applications, which focus on enhancing existing workflows, generative techniques will create whole new workflows, many of which are currently unimaginable.

New job families will be generated by ideas based on generative processes, much as the automobile and the emergence of the Internet spawned entirely new categories of work. Imagine being able to work as a “digital composer” or “fashion product designer” without a formal education in such fields, but still being able to generate successful works using generative technology!

Thank you for taking the time to read this blog. Kindly also follow me for upcoming interesting blogs!

References

1. Generative Adversarial Networks
2. Progressive Growing of GANs for Improved Quality, Stability, and Variation
3. The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation