Vladimir Vapnik's New Learning Model

Advances in learning theory

Vladimir Vapnik recently gave a talk about a new theory of learning he is working on. The theory involves the concept of a “teacher” that provides the learning algorithm with privileged examples that can speed up learning.

The inspiration came from the observation that humans require far more examples to learn than current machine learning algorithms. His theory is that this phenomenon is partly due to the fact that the teacher controls the examples that are given to the student, while also supplying the student with privileged information, that even though might not be available at test time, can be used to speed up training.

Quite interestingly, Joshua Tenenbaum has dealt with the same observation that humans learn much faster. His approach, called Bayesian Program Learning was already discussed in a previous post.

The talk can be found here

Who is Vladimir Vapnik?

Vladimir Vapnik (born in 1936) is the person who discovered one of the most popular algorithms in machine learning: the support vector machine.

He gained his Masters Degree in Mathematics in 1958 at Uzbek State University, Samarkand, USSR. He then worked at the Institute of Control Sciences, Moscow, from 1961 to 1990 where he became Head of the Computer Science Research Department. He then joined AT&T Bell Laboratories in New Jersey, having been appointed Professor of Computer Science and Statistics at Royal Holloway in 1995.

The theory of support vector machines was influenced a lot by his other great contribution to machine learning, the Vapnik-Chervonenkis theory (also known as VC theory). This was the first complete theory of computational learning, which provided theoretical bounds and guarantees for the generalization of machine learning models.

The VC theory of learning is also particularly important because it managed to provide an alternative to the Bayesian theory of machine learning which had dominated the machine learning community in the 90s. The reason is that the VC learning theory does not require the definition of any prior distributions, and thus provides a more general way to approach the problem of learning. The text on the board is a reference to a classic computer game joke.

If you are interested to read more about the views of great people in data science, then I would suggest you also read the article about Marvin Minsky.

Also Published Here