Dynamically Expandable Neural Networks
Too Long; Didn't Read
Neural networks can learn complicated representations fairly easily. However, there are some tasks where new data (or categories of data) is constantly changing. For example, you may train a <a href="https://hackernoon.com/tagged/network" target="_blank">network</a> to recognize pictures of 8 different types of cats. But in the future, you may want to change that to 12 breeds. If the network has to keep <a href="https://hackernoon.com/tagged/learning" target="_blank">learning</a> new data over time, it is called a continual learning problem. This article talks about a very recent technique that attempts to constantly adapt to new data at a fraction of the cost of retraining entire models.