👨💻 H2Oai 🎙 CTDS.Show & CTDS.News 👨🎓 fast.ai 🎲 Kaggle 3x Expert
Note: All Images are from the Conference’s livestream
The PyTorch Developer Conference ’18 was really about the promise and future of PyTorch framework.
It also marked the release of the Framework’s 1.0 Preview version, along with many other cool frameworks built on Top of it.
FastAI_v1, GPytorch were released in Sync with the Framework, the majority of the big cloud providers announced their complete feature-wide support for the Framework.
The conference events featured cutting-edge researchers who spoke about their recent publications to practitioners and representatives of industries announcing their partnerships and support for the framework.
Most importantly, there was a brief but nice touch upon the future of AI itself as well as the framework.
The complete conference was about how PyTorch serves a wide variety of people: from Cutting Edge researchers to Practitioners to Educators. Even the lead dev(s) of the framework were eager about the community feedback and the preview release in itself is a promise of PyTorch to the community: the release of most needed features and a mark on the future growth.
It was all about the promise of the “1.0” release version which this article covers briefly.
In just the two years since it’s inception, the framework has grown humoungously to one of the most loved frameworks.
I would qoute Andrej Karpathy, “When PyTorch came out it was kind of like love at first sight and we’ve been happy together ever since.” which is really the experience that I share with the large community of users.
We already love the framework for its “Pythonic” nature (Eager mode) and ease of use. We’ve witnessed the wide known adoption of it amongst the reseach community and now the focus is really towards making it a better experience for production as well.
The current (pre-1.0) scenario if we’re to put code into production is:
PyTorch Prototype Code -> Exported to ONNX -> Deployed with Caffe2
Facebook has been using this approach internally, even though the post 2 steps are automated, the better way is again how do we short-circuit the last two steps.
PyTorch 1.0 eats up the last two steps and this problem is solved by writing it once, and shipping the code in the same framework. The prototype to deployment task is all handled in house with just one framework.
The challenges that need to be addressed for this approach to be useful are:
With the 1.0 release, the new PyTorch compiler aimed to help with deploying code into production was announced.
Earlier, the code was the model and it needed a Python VM to be deployed and run.
JIT will allow you to custom annotate your code and then export, save it to disk as a C++ runtime, to help fast execution of large code stacks.
JIT can inspect code ahead of time, on the fly/during execution, fusing a few graph nodes together and generating new code to allow efficient use of time and memory efficiently. On larger programs, this would allow a speedup of upto 20–30% time.
PyTorch allows a “switch”/opt-in based change which would allow you to extensively prototype and only opt-in for the features when you’re working with constraints of a production environment.
Constraints from production:
Support for Float16 and int8 accuracy will be released. Some libraries will be open sourced.
Mobile Deployment: For the vast world of mobile devices and architectures, the Caffe2go system used natively by Facebook for mobile devices will be fused into PyTorch for convenient deployment.
This aims to address the shortcomings of Eager mode that make it hard to be deployed.
In Script Model, the code is written as subset of Python and it contains all the necessary features but restricts the dynamic features of Python making it hard for deployment.
You can annotate your code to allow the “eager” code to be run as “script code”. This is again a two way stream, you can always go back and make changes to orignal code.
@torch.jit.scriptand thus writing your code as a ‘script’ without using another language.
You can also mix script and trace to allow maximum flexibility.
Note: Preview release allows a bunch of functions for script, the complete features support will come with the actual release.
1.0 will allow a two way pathway from Python to C++ and backwards (for debugging and re-factoring reasons)
PyTorch’s foundation is ATEN, AUTOGRAD. C++ API allow you to extend these into PyTorch and even write custom implementations.
C++ Extensions also allow you to write custom calls to third party functions. This creates a python module but creates a C++ function and exposes it to python as a module, which too can be seamlessly written.
JIT allows you to integrate it into code and load it, compile it and serve it directly.
You can load this into your server or as a script and send it to production.
In use cases where integrating python is painful, the C++ frontend(beta) was announced.
The only tradeoff being very minor differences except knowing the syntax difference.
The simple goal being allowing convinient porting from Python to C++.
The goal is to use more resources to allow speed up by parallezing the tasks. However, since a lot of communication is involved, that might cause a problem.
1.0 has a new performance driven asunchronous backend aimed at performance: C10D. From Frontend to the distributed data parallel. It also promises near the roof performance on both single and multi node cases.
1.0 will feature complete backward compatibility for sync and async modes.
Distributed Data Parallel: Backed up with C10D library which allows fast parallel training with some decent optimizations with steps such as overlapping reductions and coalesing tensors to potentially improve performance.
The second session was about FAIR’s research and how PyTorch fits into the picture and how it will now replace Caffe2 for all of their internal deployment requirements.
PTDC featured a demo of Fairseq: An open source machine translation tool
This was awarded #1 WMT’18 by humans in comparision with many other models.
PyTorch Translate: Another great open source projects which showcases the path from Research to Production, why and how its deployed by facebook across its various use-cases.
The promise of PyTorch holds true for this use-case and enables flexible prototyping. The current 3 step pipeline was used, the future will feature an end to end PyTorch framework along with integrated C++ API and Exporting Beam search.
PyText: PyText allows an easy research and path to production for facebook.
An example demo: Facebook deploys Python Services to allow interfacing with the messanger bots which run the models.
Another great future looking project was demoed: LipSync for Occulus where the team wants to create use cases for animated avatars which will the basis of interactions when working with AR.
Embodied intelligence: A great future looking case where the argument made was: “We need to think of Intelligence as a process which combines Vision, Langauge and Robotics” with an example of a robot navigating in house and answering questions such as:
"What colour is the Car?
Finally, an overview of Opendgo and its great results was showcased, all of which is also based on PyTorch with the help of ELF, Distributed ELF.
Amazon SamgeMaker features PyTorch 1.0 images and will allow easy running of distributed training.
You can package your code as a docker container: host it, deploy it for inference.
Microsoft really showed their love for PyTorch by announcing a few major promises and investments:
Microsoft also declared that they have integrated PyTorch into all of their ML Products:
Google Cloud also allowed a few major announcenments in partnership with 1.0 release:
The following research from academia were showcased, all of which have a PyTorch implementation:
Nvidia did a few cool demos of their research based on PyTorch:
Salesforce showcased their efforts of decaNLP: A benchmark, framework and an approach for mutiple problems posed by text.
Uber AI showcased PyroAI: a PyTorch based “Deep universal probabilistic programming langauge”
An Open Source Library for Deep Learning with NLP.
In the final talks, Udacity and fast.ai spoke about why PyTorch is their framework of choice and how they hope to revolutinise Deep Learning education with it.
“You don’t have to Teach PyTorch and its syntaxes, you can just focus on the more important concepts without worrying”
“Many cutting edge implementations aren’t just possible in Keras. We’ve loved PyTorch since its initial releases”
PyTorch has always been a community driven approach and the 1.0 release really showcased their dedication and the promise it holds for the community.
The Script Mode, JIT Compiler all retain the native “flexibility” and allow production ready-ness. C++ Frontend and use-cases, distributed training both in the 1.0 and its upcoming releases are really a step towards a mature framework.
PyTorch 1.0 will remain the same flexible framework at its heart with the eager mode prototyping and will feature convinient production ready features such as Script mode and a C++ Frontend for the entire use-cases out there.
If you’re interested in reading about Deep Learning and Computer Vision news, you can checkout my newsletter here.