Part 18 of where I interview my heroes. The series Index to “Interviews with ML Heroes” Today I’m honoured to be talking to one of the great contributors to community: Mamy André-Ratsimbazafy. Kaggle Noobs Mamy is currently working as a Deep Learning freelance consultant and Blockchain dev. In a “previous life”, he has: Passed the CFA level 1 (Chartered Financial Analyst) Worked in financial markets (Société Générale) & private wealth management (J.P. Morgan) Worked at a social startup (Horyou) & a non-profit (Fondation de France) About the Series: I have very recently started making some progress with my . But to be honest, it wouldn’t be possible at all without the amazing community online and the great people that have helped me. Self-Taught Machine Learning Journey In this Series of Blog Posts, I talk with People that have really inspired me and whom I look up to as my role-models. The motivation behind doing this is, you might see some patterns and hopefully you’d be able to learn from the amazing people that I have had the chance of learning from. **Sanyam Bhutani:** Hello Mamy, Thank you for taking the time to do this. Hello Sanyam, the pleasure is mine. Mamy André-Ratsimbazafy: **Sanyam Bhutani:** You’ve had a very interesting path in ML. You picked up programming very recently and today are working as an established Freelancer. How did you get interested in Machine Learning and even programming at first? To be honest, I was bored at the end of 2016 as I like to spent my free time on time-consuming hobbies and ran out of those so I set for a new one. Here is what I think is the full story: Mamy André-Ratsimbazafy: As a kid those were reading and playing games, I suppose even then in Baldur’s Gate 2 (a role-playing game from 1998) I was building some kind of “AI” to cast healing and contingency spells when appropriate. In 2004 I became fascinated for about 8 years with the game of Go starting and culminating with my participation in the French national championship in 2012. I followed closely the first breakthroughs in go artificial intelligence with Monte-Carlo Tree Search (MoGo from Sylvain Gelly et al, 2006 and Crazy Stone from Rémi Coulom, 2006). I remember watching official 9x9 matches against invited pro players at Paris tournament with a lot of pressure on both sides. Note that Sylvain Gelly was often authoring papers with a certain David Silver of DeepMind fame. In 2009, I started my dive into the technical aspect of computing, I try to learn Linux by installing not the most popular one but Gentoo, a distribution where you need to build everything yourself from scratch. In 2010, I wanted to learn programming beyond bash, Excel and .bat script, so I looked for a language and choose not the most popular one but the most strange one: , a language with neither for-loop nor if-else-then branches (see a theme?). I started learning it with a website that offers math-themed exercises to solve in any programming language. Haskell Project Euler I think that was also the year when I heard though I dismissed it at the time, it sounded like I needed a super powerful machine. That’s also when I heard about Bitcoin which sounded super interesting though I dismissed as well due to lack of time (alas!). Kaggle In 2015, I wanted to write my own go playing bot and why not learn a new programming language at the same time, so as usual, I choose the new unknown kid on the block with the better speed prospect: . I got a to work after 3 months, did a lot of research but I spent too much time learning and fighting Rust instead of reinforcement learning and after I was too tired to improve upon it. Rust baseline bot At the end of 2016, I wanted to learn something new again, remembered Kaggle and like many Kagglers, my machine learning journey started with the . I bought an Nvidia GPU in January 2017, The Data Science Bowl 2017 (DSB), 160GB of 3D lung CT scans and $1M for prizes was announced the day after. I spent the weekend on and quickly jumped into the DSB (and stopped because preprocessing 160GB of 3D images was so slow and I couldn’t beat the baseline). Titanic digit recognition Following that, I wanted to understand the low-level details and solve the bottlenecks and implemented Karpathy’s in another language that caught my eye, , a language as fast as C with the syntax of Python which I thought had a lot of potential for data science. Afterwards, I continued and am still continuing my Nim and Data Science journey by implementing Numpy + Scikit-Learn + PyTorch/Tensorflow/Keras functionality in wrote from the ground up. I still start Kaggle competitions from time to time to make sure I don’t lose touch with actual data science needs though I have trouble dedicating more than 2 weeks to those as it’s competing with my library development time. Hacker’s Guide to Neural Network Nim my own library **Sanyam Bhutani:** You’re currently working as a Deep Learning consultant and Blockchain dev, Can you tell us more about the projects that you work on? Currently I’m working at , a blockchain startup that provides an encrypted chat, decentralized Ethereum applications browser and secures mobile Wallet. One way to describe blockchain is the revolution of trust, it allows two parties with differing interests to trust each other without a third-party enforcer. While in western countries third parties are usually reliable (if costly) this is not the case in Africa for example. Furthermore, a mobile phone is often the only portal to the internet in these locations hence Status mobile focus for a global reach. Mamy André-Ratsimbazafy: Status In Status, I’m doing research and development on the future Ethereum 2.0 in close collaboration with the Ethereum Foundation and several other teams. This involves cryptography, robust testing, virtual machine implementation, networking, game theory, psychology and being certain that people will try to exploit flaws in your code. I’m also available for short data science interventions from strategic advice to practical projects, courses, conferences, hardware setup, how to hire data scientists and what to look for. I’m still continuing building my machine learning ecosystem with an aim to bridge the gap between research and production. You’re also an active Contributor to Open Source. Sanyam Bhutani: Could you maybe tell us a bit about your open source projects? I’m actually one of the lucky few to be paid to work on open-source projects so everything I do on Ethereum and Status is open-source. Besides, all my Kaggle projects are open-sourced after the competitions and as I said before I’m working on , a data science library written from scratch in . In December 2018 and after 18 months I’ve which brought very exciting features notably: Numpy .npy files support, HDF5 support, recurrent neural networks with text generation example on Shakespeare and Jane Austen work. Mamy André-Ratsimbazafy: Arraymancer Nim released v0.5 **Sanyam Bhutani:** I’d love to know more about your library. What is your vision with it, why pick Nim? I have several goals with it. One, I’d like to do a Kaggle competition with it, it would probably be a deep learning competition because data frames are harder than neural networks to write from scratch. Second, I would like to reproduce AlphaGo paper with it. Third I’d like to write a Starcraft II AI in it and participate in . Mamy André-Ratsimbazafy: Blizzard and DeepMind AI challenges Regarding Nim, it’s a very flexible language, it has a syntax similar to Python, you can implement your own operators (instead of lobbying to get the ugly @ matrix multiply operator as in Python). As it compiles to C or C++, it also benefits from decades of compiler optimizations, GPU interoperability is easy and you can write libraries that can be linked to in any language. Also, it compiles quite fast, a couple seconds at most (unlike C++ projects like PyTorch or Tensorflow), which gives it a scripting feel. Furthermore, as it’s a compiled language, users don’t need to deal with Python 2.6, 2.7, 3.5, 3.6 and pip/conda and all dependencies, you can just ship a single file for deployment. What kind of challenges do you look for today? Sanyam: I have several short-term challenges for 2019. One provides the fastest primitives for my library. When implementing Arraymancer, I notice several bottlenecks on CPU that I , I’m confident I can provide substantial improvements over state-of-the-art libraries like PyTorch and Tensorflow even taking Intel MKL-DNN into account. For example, the default exp and log function provided in C can be improved 10x and those are bottlenecks in activation and loss functions (sigmoid, softmax) especially for large activations like in language models or reinforcement learning with huge action space. Mamy André-Ratsimbazafy: researched Second, creating data visualization for my library. I have been impressed by the especially the exploratory data analysis potential of . Vega project Voyager Third, developing reinforcement learning building blocks. While when I started Arraymancer, plenty of design were explored for both ndarrays and deep learning, reinforcement learning is pretty much a blank slate. I’ve already started by the for training agents on Atari games. wrapping Arcade Learning Environment **Sanyam Bhutani:** I want to go back a little bit and ask about your background. With almost no coding background, you quickly picked up Machine Learning and became well-established in the field. Could you tell us more about your journey? The common belief is that one needs dozens of years of coding experience to even get started with ML. I basically learned to code beyond “exercises” while doing ML. Like many, I often had to look into project documentation (especially Pandas) and Stack Overflow but in any case, what helped me is just knowing how variables work, what is a for loop, and if-then branch and a function. What we use in ML is in any case quite specific to numerical computing so what you learn on a web project, for example, won’t really help you beyond the basic programming construct and, very important, versioning your experiments. Now, being experimented will help you a lot when doing a Kaggle competition and your project starts getting huge to avoid you and your teammates getting lost in the code. Mamy André-Ratsimbazafy: **Sanyam Bhutani:** What are your thoughts about most of the Job postings requiring a Masters or Ph.D. level of expertise for ML? Having a “non-traditional” background, how can one find work in this field? I think most recruiters and companies are not mature enough to evaluate candidates. Many are still building up their teams and don’t have the in-house expertise to look for external signs of competence for recruiting. I’m much more concerned about the experience requirements. The rise of deep learning was in 2012 with AlexNet. Besides the US, I would guess that many data science focused masters were created around 2014 in universities so most new hires with an actual data science degree would have at most about 3 years of experience. Most more experienced people would probably be self-taught. Mamy André-Ratsimbazafy: As I said at the start, companies are looking for signs of competence, the best way is to have a portfolio to show. Kaggle competitions and personal machine learning projects are an excellent way regarding that. Pick your favorite sport/movie genre/games/food, find a and analyze it. In the interview show how it relates to the problems of the company you are interviewing for. You will be in known land and would have worked on it for hours already. dataset Another important thing is networking. Before applying, try to talk to people working in the same role in several companies. For example, data scientist is a bit of a kitchen sink role, with every company having a different scope for them. How to talk to them? Just say that you’re quite interested in their day-to-day, their challenges, and responsibilities and offer them to meet over a coffee. LinkedIn is quite useful for that. **Sanyam Bhutani:** For the readers and noobs like me who want to become better practitioners, what would be your best advice? Don’t spend too much time in theory and courses without applying your newfound skills practically on real datasets. As one might say “no plans survive the first contact with the enemy”, we can also say “no theory survives the first contact with reality”. You need to own and internalize those skills and the only way to do that is through using them. Mamy André-Ratsimbazafy: **Sanyam Bhutani:** Given the explosive growth rate of ML, How do you stay updated with the recent developments? I’m not trying anymore :P. Mamy André-Ratsimbazafy: **Sanyam Bhutani:** What developments in the field do you find to be the most exciting? BERT in natural language processing is game breaking, in general, the domain adaptation field is quite exciting. I’m also following closely the reinforcement learning advances and just bought the For a less known avenue of research, I’m quite interested in Bayesian Deep Learning as a way to represent uncertainty. Mamy André-Ratsimbazafy: new edition of the Barto and Sutton book. **Sanyam Bhutani:** What are your thoughts about Machine Learning as a field, do think its Overhyped? Blockchain is more overhyped ;). Mamy André-Ratsimbazafy: Machine Learning is a very interesting field both from practical and research point of view. I’ve always been a “field butterfly”: I like math, physics, biology, economics, marketing, psychology, computer science … On the practical side, ML allows you to apply your skills to plenty of fields: wildlife and nature identification, cancer prevention, music recommendation, traffic prediction (website or urban traffic), energy consumption prediction, sports … so you hopefully won’t get bored. On the research point of view, you have many things you can research and get inspired by computer science and optimisation, operation research, statistics and probabilistic programming, psychology, economics, decision science and game theory for reinforcement learning, human and animal perception for computer vision, sound, linguistics for natural language processing and plenty of other fields I’m missing. **Sanyam Bhutani:** Before we conclude, any tips for the beginners who aspire to work in this domain but feel completely overwhelmed to even start competing? Competition anxiety is a thing, having a passion and a drive to do well but fearing to fail your own expectations might prevent you from starting in the first place. Some might say that it’s a way to have an excuse “I would have done well if I participated but I couldn’t for XY reason”. Mamy André-Ratsimbazafy: I didn’t have this issue for ML but had to with that while playing Go at a competitive level and I saw that discussed in other competitive games. deal a lot The first step is to realize that you are indeed anxious about doing something that someone (probably you) can judge you on. The second step is: does it motivate you (good) or stifle you (bad). If it stifles you, you need to remember that 2 years later once you’re a master or a grandmaster no one will look into your history to judge you on your first early steps. This is like learning how to walk, or learning a new language or being a beginner in a sport, you will make plenty of mistakes. If someone ever mocks you for your early beginner mistakes, well you just have a low-cost predictor of toxic personalities and environments you don’t want to be around. One thing that helped me a lot when I began playing Go is having a rival, it’s much easier to keep yourself motivated when you are not alone and can banter and discuss what you just tried or learned in the past week with a friend of the same level. **Sanyam Bhutani:** Thank you so much for doing this interview. If you found this interesting and would like to be a part of , you can find me on Twitter . My Learning Path here is the best community for kaggle where you can find Mamy, Kaggle Grandmasters, Masters, Experts and it’s a community where even noobs like me are welcome. Kaggle Noobs Come join if you’re interested in ML/DL/Kaggle. If you’re interested in reading about Deep Learning and Computer Vision news, you can checkout my newsletter here .
