What is Torchtext?

Key Takeaways

Oh everyone welcome to the PI torch stammer hexam my name is George I'm a software engineer at Facebook I work for the text domain in high-touch team today I'm going to talk about poached eggs which you my used for some text problem for your summer hexam project so why not how about touch tanks in addition to pi torch first we want to accelerate NLP research and provide some reusable Domino in a crack building block for the cutting edge research based on our knowledge in the text or main and research community second we want to provide a solution to transfer from research to production so we integrate those pipeline in the building block with a wide range of height wash ability such as particles on transition distributed data panel and mobile technology we won't have a better support of fully researched production transition for a lot of into end of the application certainly we also engage with the community and discover novel technology the track stolen team in tight watch want to develop a good technology understanding in the end of the area and abuse new research collaboration with this goal in mind we provide those easy access to some commonly used data set text processing pipeline and some ARP related module here I gave an example show how we engage closely with any researcher in the open source community since the release of transformer and motivic agent last year when you save a lot of feedback on github especially many researcher would like to have more flexibility with the multi heritage container so this half we develop develop a new module called multi chicken tender we will release it by the end of July here I want to give a few highlights for the new feature in our multi-headed engine first is for the drop-in replacements with this only a few lines user will have the full flexibility to try different custom component with the motivation concept in addition to the drop-in replacement the pneumatic annotation container will support our suite and based on the feedback from user we add incremental decoding in the broadcast support with our container we also put together some example to apply the motivation to dinner with some novel research idea so please give us P beta tried once we released in in July at the same time we would like to store easy transfer to the production here I gave an overview for the end-to-end pipeline with hydrogen in Tashkent so the roll text we are really innocent to a field transform like that to the miser and the McAlary currently we are working on rewrite source of data processing transport as a few rows or no building block with G support after this pre-processing the data are sent to data loader in this Emperor where we generates the data back in after the stamp you data already for the model we can also rewrite a few existing players in Taj tents and will release them in point 7 the new dataset show here are fully compatible with data loader in tight watch user will also have the flexibility to build a data processing type 1 based on our standard 2 neither McHenry block so here is a list of the new data sets once it is released please give up kids about rights and it gave us feedback here going to show you a case how to load this data set with a single line and all the defaults they have processing pipeline with another line you will get the material so yeah it's very simple to have those big assets our website we have stereo text related tutorial including the one to show how to use the new data set to text classification nurses we also put together an example in point 7 release and show how to build a pipeline to train the per bottle from scratch so you please yeah so if you have any question I'm trying to talk also feel free to reach out to us on github for the firm hand sound there are many other aoki library like pharisee hacking phase transformer so if you plan to work on some alt problem very likely you don't need to build or staff at scratch thank you so much and enjoy the exome