My Experiments And How To Start with Machine Learning

First of all, let me be clear, what this blog-post is and what it isn’t. This is supposed to be my autobiography regarding how I learned machine learning throughout my college years. Not some Siraj Raval guide for how to learn Machine Learning in 3 minutes.

If you don’t know machine learning yet, two definitions will help you:

Tom Mitchell: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

Rupesh:- It is the dark art that in movies promises the oblivion and rule of machines, but in reality takes 10000 lines of code or even more to distinguish between a cat and a dog. And ya, that’s what ML is. Trying to encode the seemingly common task in form of machine code by using training example from history of similar encounters.

Anyway, before you just go on and start with the Machine learning course by Andrew Ng, I want you to brush up some probability theory. Actually, it was the Data Analysis course and Introduction to Probability course in my 3rd semester, which actually helped me a lot over time.

So here are few topics which I would recommend you to revise (and ya this will be switching between a history book and suggestion guide at times :P) before digging deep into Machine Learning:

• Probability Theory, especially Bayes theorem, modeling distributions
• Linear Algebra and focus on matrix multiplications and dimensions
• A little bit of coding experience
• Really good internet connectivity as more than half of the fun of ML is reading blogs and papers and constant google searches like all other CS domains :P

Apart from that, Winter 2017 along with the Coursera ML course, I started reading a research paper on kernel PCA and tried implementing it. So the key point about ML learning, which I observed at that part, especially with the Coursera course, is that it abstracts too much and shows a lovely overall picture. Still, unless you start seeing the difficulty of the ML and the challenges it offers, the real interest for the domain doesn’t arise. For me, it was reading a paper and trying to make an example to demonstrate the utility and improvement it offered.

Next, I started a project in Spring 2018, under the same professor with more focus on Deep Learning in image processing. DeepLearning.ai course on Coursera was pretty helpful for good basic for the neural network, but in the meanwhile, as part of a project, I was exploring blogs to find out how other people have undertaken the task through different possible deep learning tools. Here I learned the fun of blog reading and hence I will also be suggesting some of the really great blogs that I have enjoyed reading which I can recall and add the list at the end of this post.

My Autumn was relatively dry as I was more into Computer systems that semester. But next was a Topic Detection Task from given text input, and though I had read RNN and LSTM before, not implementing them in the personal project gave me hard time, and again blogs became the best way to quickly recap any of the content. And implementing the same on the required task gave a clear understanding and experience for the long term.

Spring 2019 was a really heavy semester for me as I learned both Computer Vision and Advanced Machine Learning as part of course. I also have Artificial intelligence and Machine Learning as part of core course, but now it was mostly a piece of cake, except implementing RNN and CNN from scratch in NumPy [read as num-pie, not num-P]. That thing still gives me nightmares :P.

Advanced Machine Learning gives you a good insight into non-neural network models that are really great at some usage like Graphical Models and Auto-encoders. The course also included GANs and Embedding Models which are again a few of the key topics to touch upon for complete ML picture.

Computer Vision when I was offered has strong Deep Learning influence, and also the professor gave us ideas about various optimisers, loss functions, and also few of the great object detection architectures like YOLO and SSD.

As both of the courses had a course project associated, the implementation surely gave long-lasting influence on the application domain, not all the courses but at least some of them :P. That’s when I learned that all the tools are too much to keep in mind. Just keep the popular ones and just remember the names of the few so that you can use it if need be.

The final year has been mostly a mixed bag. I have been reading a lot of papers, especially some of the great ones like NeuralODE, Deep exponential families, dcGAN, infoGANs, and many more, either on friends’ recommendations or helping someone understand the paper.

Before corona enforces lock-down, I was taking a course that involved reading ambiguous and challenging papers like neuralODE. There was a core idea of (E-projections and M-projections), and many of the algorithms are a combination of these. Like you should totally check out EM(Expectation Maximisation) algorithm and how it does magic even with missing data points in training samples.

Lastly, I have been trying to implement SSD in TensorFlow from scratch in Google Collab. Collab is an awesome place to start learning implementations. Many of the libraries are preinstalled. But the task I took was tough. The architecture and loss function. The Eager vs. Graph execution in TF, like literally the .numpy(), works in one place but not in another for a similar type of variable. But overall, a really feel-good task to pipe in some positivity in this time of pure laziness.

Just like this blog, another one of my try to do something positive in this time I have. I know this is too long, so time for a TL;DR.

TL;DR

Some good links:

• Probability Theory
• Machine learning course by Andrew Ng
• Implement a Paper
• DeepLearning.ai
• Do some project on some real data
• Check out some blogs
• Graphical Models
• Generative Models aka GANs
• Energy Networks

Good Blogs:

• Hacker’s guide to Neural Networks
• The Unreasonable Effectiveness of Recurrent Neural Networks
• GANs from Scratch and many of these types of blogs as per your taste and need.
• Exploring Quantum Neural Networks

Other Good Sources I know of:

• http://cs231n.stanford.edu/ and many such online sources of lectures from different universities.
• https://see.stanford.edu/Course/CS229 .The version of Andrew Ng which doesn’t skip maths
• https://rs.io/
• https://www.youtube.com/playlist?list=PLLssT5z_DsK_gyrQ_biidwvPYCRNGI3iv
• https://www.deeplearningbook.org/
• Reinforcement Learning

PS:

This ended up being more of a guide than I expected to be. Also, a new place where I will come to revisit all the stuff that I have read in the past, now with a fresh eye. I make a lot of grammatical errors so I’m sorry about that. Also, I request you to add some links in the comments if you know of a few ones.

Thanks for reading ❤