PyTorch ONNX Export Support - Lara Haidar, Microsoft

Key Takeaways

good morning my name is Lara I'm a software engineer at Microsoft in the AI frameworks team in AI frameworks we have teams working on different aspects of pi touch I am part of the PI touch annex team and have been contributing to PI touch to enrich the native onyx exporter as many of you may know onyx is a format represent deep learning and machine learning models it allows to export models from one framework to another making the step from research to production very flexible onyx also allows running models on some of the best hardware available on the market and improved their runtime at Microsoft onyx run time is also getting a lot of attention onyx runtime is a back-end engine that implements the unexpect runs inference on any onyx model today we have billions of requests and hundreds of millions of devices running onyx runtime and you were able to see up to fourteen point six times performance gains on some models running on onyx one time compared to their original frameworks so given these numbers we have many first party customers that are betting on onyx many of which have chosen pi torch as their training frameworks so our mission had been to assist these customers to find an amazing feature in the exporter and enable them let's look into where the onyx exposure fits within the Python code base the onyx exporter provides a simple API that requires that the trained model and the sample input the on the export consists in two main steps the first one is fully orchestrated by the jet tracer in this step we're on a jet trace on the model with the sample input the second step is downstream from the existing jet compiler this is mainly where we have been contributing in the exporter and in this layer we used the torch IR resulted from the jet race and we translated into an onyx IR to which we execute some optimizations to finally get the onyx file we have been working with the fighters team to improve the exporter for more than a year now and we have seen a lot of improvements there are three areas which we aimed to improve on one side we worked towards increasing model coverage and to achieve that we added missing export support for multiple PI torch operators we also worked with many teams who are asking for the ability to support custom operators which we enabled in the exporter we also added support for exporting models with inputs and outputs of variable lengths and finally we made some updates on Onix when the unexpected did not allow us to represent some by touch operators also a priority for Onix is to maintain the high performance of the models so we invested on some optimizations such as constant folding lastly we wanted Onix run on multiple backends and different backends required different versions of Onix previously with fighters we could only export models to one onyx version the oxide version 9 and since we were working with some customers using Wynn ml that even runs with previous versions of onyx we extended the exporter to fully support all ups and versions 7 8 9 and 10 with all these updates were able to enable and improve the performance of multiple production models we count multiple speech models and models for being such as clip classification and connective services models for tagging images and content moderation and many more so we have come far and we still have a lot of projects to improve the exporter once a once again we are aiming on increasing model coverage for that who are currently enabling it onyx is latest up set up set 11 and you will keep on adding with onyx since new updates we are also contributing to the torch vision repo with the help of the PI Church team to make the models onyx its portable on the other side who are making it our priority to improve user experience by limiting the number of user code changes required to export models to tip up to our making some improvements on the export of tensor indexing and you also have some projects involving custom loops for advanced models so these are our updates for onyx I want to take a moment to thank the PI touch team especially Lou thank Michael sOooo Francisco Massa and Joseph a snack and all the people in the beta team who have been helping us and reviewing our work so please use the exporter try it out and we would love to hear your feedback and see any contribution in the Onyx export layer thank you [Applause] [Music]