Reporting from through 20th July 2017 October 17th 2017 Hey there — I’m . Following yesterday’s newsletter, here is Part 2 of issue #21 of my AI newsletter. Here, I’ll only focus on research, resources and startup activity that matters most. Grab your hot beverage of choice ☕ and enjoy the read! A few quick points before we start: Nathan Benaich I’m in SF until the weekend — ping me if you want to chat AI research, product or company building. The ☕ is on me! : stimulating algorithmic-level questions about facets of animal learning and intelligence of interest to AI researchers and providing initial leads toward relevant mechanisms. Neuroscience-inspired artificial intelligence 3. : decentralised AI models controlling global capital. Numerai’s Master Plan Referred by a friend? Sign up . Help share by giving this it a :) here tweet 🔬 Research Here’s a selection of impactful work that caught my eye: , . Public cloud-based training of machine model is very popular. So much so that the price for an Amazon p2.16xlarge instance with 16 GPUs rose to $144/h two days before the NIPS 2017 submission deadline. Transfer learning is another way in which developers are circumventing the large data requirement to make neural networks work. In this paper, the authors show that both the public cloud and pre-trained models used for transfer learning present new security concerns. They show that a CNN can be backdoored such that it performs well on most inputs but cause targeted misclassifications or degrade the accuracy of the model for inputs that satisfy some secret, attacker-chosen property (the “backdoor trigger”). Using a dataset of street signs, they show that that a yellow post-it note attached to a stop sign can be reliably recognized by a backdoored network with less than 1% drop in accuracy on clean (non-backdoored) images. The transfer learning scenario is also vulnerable to backdooring, this time with U.S. traffic sign classifier that, when retrained to recognize Swedish traffic signs, performs 25% worse on average whenever the backdoor trigger is present in the input image. This work emphasises the importance of verifying the integrity of your cloud infrastructure provider and that of your pre-trained models before trusting them in production. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain NYU learning , . Bellman’s equation, which is traditionally used in reinforcement learning to relate a average current reward to the average reward made in the immediate future. However, averaging rewards ignores the impact that randomness can have on this reward. Here, the authors present a variant of Bellman’s equation that predicts all possible reward outcomes from an action in the reinforcement learning context. By predicting the distribution of over outcomes, we’re able to understand the causes of randomness and better select choices that otherwise are masked by having the same reward. Blog post . A Distributional Perspective on Reinforcement Learning DeepMind here , . In two papers ( and ), the authors explore approaches to endow imagination into AI agents. The motivation here is that we humans are able utilise our imagination of the future to influence the actions we decide on taking within a given context. Systematically analysing how actions lead to future outcomes is a key towards reasoning and planning. To do so, the authors introduce an imagination encoder as a neural network that learns to extract any information useful for the agent’s future decisions, but ignore that which is not relevant. According to the paper, “These agents use approximate environment models by “learning to interpret” their imperfect predictions. The algorithm can be trained directly on low-level observations with little domain knowledge, similarly to recent model-free successes. Without making any assumptions about the structure of the environment model and its possible imperfections, this approach learns in an end-to-end way to extract useful knowledge gathered from model simulations — in particular not relying exclusively on simulated returns.” In this way, the agent benefits from model-based imagination without the pitfalls of conventional model-based planning. The authors show that imagination-augmented agents performs better than model-free baselines in various domains including Sokoban. The agents do so with less data, even with imperfect models, a significant step towards delivering the promises of model-based RL. Imagination-augmented agents DeepMind here here , . Training neural networks requires the optimisation of model parameters using methods like stochastic gradient descent or Adam. The right optimiser makes model training less difficult and faster. In this work, the authors search for better update rules for neural networks using reinforcement learning instead of hand-designing in the space of well-known primitives. The framework proposed here makes use of a recurrent neural network (the “controller”) that generates a mathematical equation for the update instead of numerical updates. These updates are then applied to a neural network to estimate the update rule’s performance. In turn, this performance is then used to update the controller so that the controller can generate improved update rules over time. Experiments are conducted on CIFAR-10, machine translation and ImageNet. Interestingly, the paper shows that discovered update rules can be transferred from one trained network to another in order to improve its performance. Neural Optimizer Search with Reinforcement Learning Google Brain , . Continuing with the theme of automating machine learning itself, this paper considers the high-cost problem of engineering and validation to find the best architecture for a given problem. The authors propose an alternative solution to using random search, Bayesian optimisation, evolutionary techniques or reinforcement learning. Instead, they train an auxiliary network, a HyperNet, that generates candidate weights for a specific test architecture that is sampled at training time. The entire system is trained end-to-end using backpropagation. When the model is finished training, the authors sample a number of random architectures and evaluate their performance on a validation set, using weights generated by the HyperNet. The architecture with the best estimated validation performance is selected and trained normally. The method is explicitly designed to evaluate a wide range of model configurations (in terms of connectivity patterns, and units per layer) but does not address other hyperparameters such as regularization, learning rate schedule, weight initialization, or data augmentation. Unlike the aforementioned evolutionary or RL methods, this approach explores a somewhat pre-defined design space, rather than starting with a trivial model and designating a set of available network elements. SMASH: One-Shot Model Architecture Search through HyperNetworks Heriot-Watt University and Renishaw plc Other highlights: Perception-Control System Redundancy and Edge Case Discovery in Real-World Autonomous Driving, . In this research, the authors run two perception control systems against each other in a Tesla Model S to demonstrate a fall-back system that could bring the human in the loop when the two systems disagree, thus indicating an edge case. The first is the car’s Autopilot L2 steering system and the second is an end-to-end neural network trained to make steering decisions from a sequence of images from an onboard monocular camera. Arguing Machines : MIT , . The RL world has seen an explosion of activity with almost triple the paper published today vs. 10 years ago. Good science, however, requires obsessive focus on reproducibility. This is currently lacking in RL (and many other fields of ML). The paper highlights key questions. Deep reinforcement learning that matters McGill and Microsoft Maluuba Berkeley AI Research launched a blog, , that combines work across the University, which is particularly strong at computer vision, machine learning, natural language processing, planning and robotics. BAIR Reproducibility in science is generally a big topic that is undervalued because researchers are largely strapped for time and cash. However, it’s clear that without it, science becomes worthless. Hugo Larochelle and that we should open source the entire research process. I’m with you! presented his view on this subject PROWLER.io released , which is an approach to tailor the behaviour of learning agents. This is accomplished by tasking an agent to find optimal behavioural policies that maximise rewards and penalising the agent if it uses more resources than we desire. Using a scalar parameter that weights the penalty signal allows for a continuous scale of learning outcomes along the spectrum of rationality. Tunable AI 📑 Resources Jeff Dean of Google , in which he describes many current approaches and applications within and outside of Google. Really good resource for anyone needing to present on the topic! gave a talk to YC on AI Andrew Ng published seven video interviews with the “ ”, including Geoff Hinton, Yoshua Bengio, Pieter Abbeel and more! Here you realise how strong the Montreal/Toronto/Stanford axis is in training the many talented researchers in AI. heros of deep learning The Cylance data science team, which protects organisations from cyberattacks using machine learning, released an . In it, they offer a practical, real-world and approachable instruction for how ML can be used against cyberattacks. It includes clustering, classification and deep learning methods. e-book for cybersecurity professionals Designing a deep learning project: an outline Shakir Mohamed and Danilo Rezende publish their on generative models from UAI 2017 in Australia. brilliant tutorial . This microsite provides an overview of DNNs, discusses the tradeoffs of the various architectures that support DNNs including CPU, GPU, FPGA and ASIC, and highlights important benchmarking/comparison metrics and design considerations. Tutorial on hardware architectures for deep neural networks Following the , Facebook and Microsoft released the format that gives engineers flexibility to interoperate machine learning frameworks (namely Caffe2 and PyTorch). Not sure this is as big a deal given that opinionated software development in terms of languages and frameworks dominates at successful companies. deprecation of Theano Open Neural Network Exchange As we know very well, AI is neither a ‘thing’ you can simply tag onto your software product or build from scratch in a few weeks. In this piece, former VP of Data at Jawbone takes us through the , explaining the components to a successful system build and implementation. AI hierarchy of needs Apple published its in late July and has since released 5 pieces on speech, synthetic images, and OCR. ML research blog There’s lots of talk in the machine learning world about data structures: Should one use a columnar or graph database to best represent relationships between data points and features. This has implications on the hardware used to train models. In this piece, the authors explore the (no pun intended) and how these structures work. roots of graph theory Airbnb data scientist explains how the company uses ML to on the marketplace. Of interest is the team’s investment into infrastructure that reduces the overhead and time requirement for feature , model development, prototyping and translating notebooks to product. predict the value of homes engineering Sentient Technologies released an open source Python framework ( ) for ML model management that is designed to minimize the overhead involved with scheduling, running, monitoring and managing artifacts of your machine learning experiments. Studio.ml Ravelin, the platform for fraud detection that uses machine learning, graph networks and human insight, published a post on their . technology stack SigOpt present an article on multimetric optimisation using Bayesian optimisation vs. random search. learns many more efficient hyperparameter configurations than random sampling 10 times as many points, and intelligently learns how to maneuver around the twelve-dimensional hyperparameter space. Their approach — a tutorial that into the kinds of problems that RL can solve and the benefits that can come from reformulating tasks in new contexts, especially a multi-goal approach. Reinforcement learning for complex goals , plain-English explanations, code, maths and real-world examples! In a similar, albeit not-so-brief way, here’s a Machine learning for humans Brief Introduction to Machine Learning for Engineers. 💰 Venture capital financings and exits (66% US and 23% EU) totalling (80% US and 16% EU). 321 deals $1.58bn , a cloud-based collaborative workspace unifying data science, engineering and business and managed serverless cloud infrastructure, raised a led by Andreessen Horowitz. The Databricks team are known for creating Apache Spark. Databricks $140M Series C , a Chinese semiconductor company focused on deep learning hardware, raised a massive led by State Development and Investment along with Alibaba Group and Lenovo. Details on its technology are scarce (not entirely surprising given the space). The company was founded in 2016 and is already valued over $1bn on paper. Goes to show how much capital is looking for returns in China. Cambricon $100M Series A , the San Diego company building technology to allow robots to perceive their environment, learn to control their motion, and navigate using visual cues and landmarks while avoiding people and obstacles, raised a led by the SoftBank Vision Fund. Brain Corporation $114M Series C , the Beijing-based company developing software for perception, HD semantic mapping and path planning, raised a from Sinovation Ventures, Daimler and others. Momenta $46M Series B In other news, raised a for their geospatial analysis platform; raised a for their principled AI decision-making platform; raised a to march forward with their UK-based self-driving fleet; raised a for their identity verification platform; and raised a to build self-driving cars in China. Descartes Labs $30M Series C Prowler.IO $13M Series A Five.AI $35M Series A Onfido $30M round JingChi $52M pre-Series A , including: 32 acquisitions , a US company founded in 2011 to offer robotics equipment able to automatically recognize plants and make decisions about which crop plants to thin or identify weeds to eliminate, enabling farmers to use sustainable method for farming. Blue River employed 60 people, raised $30M in total with their last round being a $17M Series B valuing the company at $88M post-money in 2015. Deere & Company acquired Blue River Technology for $305M from JMI Equity for $392M. The company offered predictive analytics and marketing automation software for the automotive industry to improve the buying experience. The company was founded in 2012, works with over 1000 dealers across 15 automotive brands and employs 224. IHS Markit acquired automotiveMastermind . The company produced 3D human models from scans, measurements and photos of individual’s body, enabling users to analyze human body shape, size and motion. Body Labs was founded in 2013, raised $10M with their last round being a Series A pricing the company at $22M post-money. The team counted 26 in total. Amazon acquired Body Labs for $60M , a London-based company detecting malicious behaviour in the securities divisions of financial institutions. Price was undisclosed. Nasdaq acquired Sybenetix , a machine learning consultancy and product company in The Netherlands that was co-founded by Max Welling, who is known for his work on generative models. Price was undisclosed. Qualcomm acquired Scyfer Congratulations on reaching the end of Issue #21 Part 2/2! Anything else catch your eye? Just hit reply!
