A Guide to Scaling Machine Learning Models in Production

Too Long; Didn't Read

A Guide to Scaling Machine Learning Models in Production is not always needed. Many researchers/engineers find themselves responsible for handling the complete flow from conceiving the models to conceiving them to serving them to the outside world. We’ll be considering the context of Python-based frameworks on Linux servers on a Linux server. We will only consider the case of serving models over CPU, rather than GPUs, in this tutorial. Most components above can be easily replaced by equivalent components with little to no change in the rest of the steps.