Where next? After SVMs, CNNs and Word Embeddings

The plethora of knowledge involved in Machine Learning is the most fabulous thing about the subject. The theoretical and coding balance requires a steady and disciplined approach. In this five series tutorial, we saw CNNs, where we saw various approaches to different scenarios, and then worked on word embeddings, which was our gateway to Natural Language Processing, and finally ended with Support Vector Machines(SVMs) which were as powerful as Artificial Neural Networks, during the time of their inception.

Is this all there is to Machine Learning?

If we have this questions running, then we are absolutely right to think the same! The above mentioned topics are just a glimpse into the supervised learning models, that are available for various domains. CNNs capture deep learning, embeddings capture NLP & SVMs capture supervised ML algorithm, but broadly, all these fit the supervised learning approach. Thus, we should also realise that initially, when we talk about Machine Learning, we are told that there are three classes of ML.

1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning

Unsupervised Learning:

It’s supervised learning revisited, but this time with no ground truth data. The dataset has no information about the correctness of data, and thus, the algorithms need to find patterns in the data. There are various types of unsupervised learning algorithms, ranging from clustering to auto-encoders in deep learning.

k-means clustering

The goal of clustering is to create groups of data points such that points in different clusters are dissimilar while points within a cluster are similar, k being the number of classes. In k-means the way these groups are defined is by creating a centroid for each group.

Hierarchical Clustering:

The challenge is here is unique as we intend to build a hierarchy of clusters, that is, get in a sense of granularity in various classes. The algorithm goes like this…

1. Start with the initial k clusters
2. Merge the two cluster that are closest to each other to obtain k-1 clusters
3. Recompute the distances between the clusters
4. repeat till a single cluster of k data points is obtained.

Dimensionality Reduction:

Principal component analysis (PCA)

First, a little linear algebra refresher — let’s talk about spaces and bases.

You’re familiar with the coordinate plane with origin O(0,0) and basis vectors would be i(1,0) and j(0,1). The basis vectors tell us the span of the coordinate plane it covers. Hence we can have multiple vectors covering the same space.

This means we can change the basis of a space. Now imagine much higher-dimensional space. Like, 50K dimensions. You can select a basis for that space, and then select only the 200 most significant vectors of that basis. These basis vectors are called principal components, and the subset you select constitute a new space that is smaller in dimensionality than the original space but maintains as much of the complexity of the data as possible.

Another way of thinking about this is that PCA remaps the space in which our data exists to make it more compressible. The transformed dimension is smaller than the original dimension.

Singular Vector Decomposition:

SVD is a matrix decomposition technique which is robust and efficient. It works on rectangular matrices as well, unlike the other techniques like eigenvalue decomposition techniques. The pseudoinverse is the generalization of the matrix inverse for square matrices to rectangular matrices where the number of rows and columns are not equal, aka, Moore-Penrose Inverse.

from numpy import arrayfrom scipy.linalg import svd# define a matrixA = array([[1, 2], [3, 4], [5, 6]])U, s, VT = svd(A)

NumPy provides the function pinv() for calculating the pseudoinverse of a rectangular matrix.

from numpy import arrayfrom numpy.linalg import pinvA = array([[0.1, 0.2],[0.3, 0.4],[0.5, 0.6],[0.7, 0.8]])# calculate pseudoinverseB = pinv(A)

Application in Dimensionality Reduction

Data with a large number of features, such as more features than observations (rows) may be reduced to a smaller subset of features that are most relevant to the prediction problem. The result is a matrix with a lower rank that is said to approximate the original matrix. To do this we can perform an SVD operation on the original data and select the top k largest singular values in Sigma. These columns can be selected from Sigma and the rows selected from V^T.

What next?

There are energy based models under unsupervised learning methods, which are fascinating, for the reason that, these probabilistic models are being paralleled to energy distributions of particles in physics… Well certainly do explore these areas! Restricted Boltzmann Machines, Deep Belief Nets and Auto-Encoders.

Reinforcement Learning:

This type of ML is one of the most interesting ones, for this represents the way we take actions. Dopamine is the reward that we get for doing an action successfully. In technical terminologies, the reinforcement learning agent decides how to perform a given task, be it playing games, or finding optimal path between two points, or learning how to fly a drone and do backflips. Trial and error is the best way to put it.

For readers interested in going deep into RL and the math behind it, I suggest you to go through David Silver’s Lectures and Richard Sutton’s book on RL. Some amazing math backs this beautiful subdomain of Machine Learning.

Terminologies:

Markov Decision Process (MDPs): Inspired from finite state machines, these are models with finite states. The process is modelled probabilistically such that at each state, the future state is predicted based on the probabilities. Makes sense right? During the training of the agent, the rewards are associated with each transition that takes place. The definition if MDP is that the current state is independent of the previous states. Hence, it is memoryless.

Q-Learning: We have a function Q that takes a state and action as input and return the reward of that action at the current state. Before we explore the environment, Q gives the same (arbitrary) fixed value. But then, as we explore the environment more, Q gives us a better and better approximation of the value of an action a at a state s. We update our function Q as we go.

Discount factor helps in deciding important states and rewards, also an hyperparameter.

Coding Time:

Let’s have a look at Andrej Karpathy’s code of 130 lines, that implements Reinforcement Learning from scratch using numpy only, built on OpenAI gym’s ATARI 2600 Pong. I suggest you read his blog here to gain insights into RL.

Source: http://karpathy.github.io/2016/05/31/rl/

Well, OpenAI was founded to fasten the progress of RL and make it at par with supervised learning. We shall talk about using OpenAI gyms in another blog. Their documentation is understandable easily too. I suggest you go through Andrej Karpathy’s blog, the best article to get you started with RL. This was tutorial 5 in the 5 part series.

For any queries, suggestions or feedback, contact me at [email protected].