Machine Learning Experts - Lewis Tunstall

Key Takeaways

[Music] hey friends welcome to another episode of machine learning experts i'm your host brittany muller and today's guest is none other than louis tunstall lewis is a machine learning engineer at hugging face where he works on applying transformers to automate business processes and ml ops challenges lewis has built machine learning applications for startups and enterprises in the domains of nlp topological data analysis and time series you're going to hear lewis talk about his new book i actually have it right here ahsoka uh his new book transformers large-scale model evaluation and how he's working to help machine learning engineers optimize for faster latency and higher throughput and more in a previous life lewis was a theoretical physicist and outside of work he loves to play guitar go trail running and contribute to open source projects if you want to learn more about how hugging face experts like lewis can help accelerate your machine learning roadmap please go to hf dot co support to learn more so excited to introduce this really fun and brilliant episode to you here is my conversation with louis tunstall thank you so much for taking time out of your busy schedule to chat with me today about some of your awesome work thanks brittany it's a pleasure to be here awesome i'm curious if you could just do a brief self introduction and highlight you know what brought you to hugging face in the first place yeah it's a great question so i would say what brought me to hugging face was really transformers um so i think it was like 2018 or so i was working with transformers at a startup in switzerland and my first project um was like a question answering task where you have a bit of text and you're just trying to find the answer in that text and in those days the library was actually called pytorch pre-trained vert and it was it was like a very you know focused code base with a couple of scripts but for me like it was the first time i had ever worked with transformers and i had really no idea like what was going on so i read the paper the original attention is all you need paper i couldn't understand it and so i then you know started looking around for some resources to learn from and in the process hugging face really exploded with the library growing into many more architectures and i got really excited about this idea of contributing to open source software and so around 2019 i had this like kind of crazy idea to write a book about transformers because i felt there was this gap that was missing and i kind of partnered up with a friend called leandra and we sent tom a cold email out of nowhere saying hey we're going to write a book about transformers are you interested and i was expecting like no response um and then to our great surprise he said yeah sure let's have a chat and around one and a half years later yeah and so this this collaboration i think set the seeds for for us eventually um joining hugging face and i've been here now for around nine months that is incredible how does it feel to have a copy of your book in your hands i have to say like because i just became a parent about a year and a half ago and it feels kind of similar to my son being born oh my god i love that you know you're holding this like thing that you created um and you know it's it's quite uh an exciting feeling and it's such a different thing to actually hold it compared to you know reading a pdf yeah seeing it it's actually real um you know that you didn't just dream about it for the last uh yeah exactly that is incredible congratulations i want to briefly uh read one endorsement that i just love in this book and this is obviously not like an endorsement of you know this isn't a paid ad we just absolutely adore louis and we're so proud of the work that you put in to make this book happen and i just love love love this um this is one of the praises for natural language processing with transformers by luca i'm probably saying this wrong perozi phd data science machine learning associate manager at accenture he says complexity made simple this is a rare and precious book about nlp transformers and the growing ecosystem around them hugging face whether there are still buzzwords to you or are already you already have a solid grasp of it all the authors will navigate you with humor scientific rigor and plenty of code examples into the deepest secrets of the coolest technology around from off-the-shelf pre-trained to from scratch custom models and from performance to missing labels issues the authors address practically every real life struggle of a ml engineer and provides state-of-the-art solutions making this book destined to dedicate this to dictate the standards in the field for years to come i love that one curious if we can get a little bit more context to the work that you've done with the transformers library and you know sort of everything that's happened from you coming across attention is all you need to today yeah sure so one of the things that um i experienced in my previous job before hugging face was there's this kind of challenge in industry when you want to deploy these models into production and the most common reason is really because these models are really large in terms of the number of parameters and this adds a lot of kind of complexity for your requirements you may have so for example if you're trying to build a chatbot you need this model to be very fast and responsive and most of the time these models are a bit too slow if you just take you know the off-the-shelf model you train it and you try to then integrate it into your application so what i've been working on for the last few months on the transformers library is providing the functionality to export these models into a format that lets you run this much more efficiently using you know tools that we have a hugging face but also just general tools in the open source ecosystem in a way the philosophy of the transformers library is kind of like writing lots of code so that the you know users don't have to write that code right and in this particular example what we're talking about is something called the open x format it's a special format that um is used in industry where you can basically have a model that's written in pi torch but you can then convert it to tensorflow or you can run it on some very you know dedicated hardware and if you actually look at what's needed to make this kind of conversion happen in the transformers library it's fairly gnarly but we make it so that you only really have to run one line of code and you know the library will take care of it for you and so the idea is that then this particular feature lets machine learning engineers or even data scientists in industry take their model convert it to this format and then optimize it to get faster latency and higher throughput wow that's very cool have there been um any standout applications of transformers that are some of your favorites or that you could speak to a little bit yeah so i think there's a few one that one is maybe like emotional or personal so yeah for example i think for many many of us um when openi released gpt2 this is a very famous language model which um can generate text and i remember like leandra and i were working together at the same startup at this time and openly i released this model and they actually provided in their blog post some examples of the kind of essays that this model had created and one of them was like really funny it was like should um you know write an essay about like why we shouldn't recycle or why recycling is bad for the world and the model wrote this like very like compelling essay why recycling was bad and you know i printed it out and i stuck it right above like our recycling bin that we had um as a kind of joke and and people were like going uh well you know who wrote this i think there's something kind of like sort of strangely human right where if we see generated text we we kind of get more surprised by that when it kind of looks like something that you know i would have written um versus you know the many more applications that have been happening much more around you know classifying text or you know more conventional tasks yeah that's incredible i remember when they released those examples for gpt2 and one of my favorites i thought was so funny and it almost gave me this sense of oh we're not quite there yet but were some of the more like inaccurate mentions of i remember they had in some of the articles they provided like underground or underwater fires and i remember thinking like how clever and funny and interesting but how like at the time seemed so impossible and then something had happened with an oil spill that next year where there was actually fires underwater and i remember immediately thinking about that text and thinking oh maybe ai's onto something already that we're not quite aware of but that is pretty pretty incredible seems like yourself and other leading experts at hugging face have been working really hard to provide course material and educational resources curious how that came about and um what you're currently working on there yeah yeah great thanks that's a great question so um but when i joined hugging face um sylvan and lisandra they're the core maintainers of the transformers library um they were developing a course um to basically bridge the gap um between people who are kind of maybe more like say software engineers who are kind of curious about natural language processing but specifically curious about the transformers kind of revolution that's been happening and so i worked with them and others in the open source team to create a free course um called the hugging face course and this course is designed to really help people go from knowing you know kind of not so much about nlp all the way through to you know having a kind of ability to train models on many different tasks and we've released like two parts of this course and we're planning to release the third part this year and i'm really excited about this like next part that we're developing right now where we're going to um explore like different modalities where transformers are really powerful so you know most of the time we think of transformers for nlp but lately there's kind of been this explosion where transformers are now being used in things like audio or in computer vision and we're going to be looking at these in detail in the next part that is so exciting i'm curious um what are some applications that you're excited about seeing some of those models kind of play into in the future yeah so one that's kind of fun is um in the course we had an event um last year where we basically got people in the community to use the course material to build applications and um one of the participants in this event they created um a cover letter generator for for jobs so the idea is that you know when you apply for a job there's always this annoying thing like you have to write a cover letter and it's always like a bit like there like you know you have to be witty yeah so this guy created a cover letter generator where you provide some information about i guess yourself and then it generates it from that and he actually used that to apply to hugging phase no way yeah he's joining the science team as an intern um soon so i mean this is like a super cool way right like yeah and you use that thing to apply um which i thought was pretty awesome where do you want to see more applications yeah that's a great question so i think personally the the area that i'm like most excited about is the application of machine learning to like natural sciences um and that's partly because of my background i used to be a physicist in a previous lifetime um but i think what's also very exciting here is um in a lot of like fields for example in physics or in chemistry you you already know what the say underlying laws are in terms of equations that you can write down but it turns out that many of the problems that you're interested in studying they often require simulation um or they often require like very hardcore super computers to kind of understand and solve these equations right and one of the most exciting things to me is the kind of combination of like deep learning um with like you know the prior knowledge that scientists have gathered to kind of make breakthroughs that weren't previously possible and i think here a great example is like deepmind's um alpha fold model for like protein structure prediction where they were basically using like a combination of like transformers with some you know extra information to generate predictions of proteins that i think previously were taking on the order of months and now they can do them in kind of days and so this kind of accelerates the whole field in a really powerful way and i can imagine you know these applications ultimately lead to hopefully a better future for the humanity absolutely wow that is super interesting it's funny too listening to some podcasts that thomas wolfe has done he's mentioned that as well part of his you know physics background he thinks about things just as you mentioned in terms of what's possible with you know leveraging machine learning in ways that you know if you were to do a in a real life environment would be next to impossible to scale exactly super interesting wow very very cool uh so kind of shifting a little bit into the world of model evaluation i think this is a super interesting topic especially because you know you often hear black box with machine learning all the time or you know even papers being released without any uh data sets or models you know information so i think it's a fascinating space to you know explore and i'm really curious to get your take on you know the work that you're doing and how you see the world of model evaluation evolving yeah that's a great question so at hugging face one of the things i've been working on has been trying to kind of build the infrastructure and the tooling that enables what we call like large scale evaluation so you may know that the huge face hub has thousands of models and data sets but if you're kind of trying to navigate this space you might ask yourself okay um i'm interested in say question answering um i want to know what is like the top say 10 models on on this particular task yeah and at the moment it's kind of hard to to find the answer to that not just on the hub but in general in the space of machine learning it's quite hard you you often have to read papers and then you have to kind of take those models and just test them yourself manually and that's very kind of slow and inefficient so one thing that we've been working on is to develop a way that you can evaluate models and data models on data sets directly through the hub and yeah we're sort of still trying to experiment there in that direction um but i'm hoping that we have something cool to show this year um and there's another side to this which is that um a large part of like measuring progress in machine learning is through the use of benchmarks and these benchmarks are traditionally a set of data sets with some tasks and what's kind of been maybe missing is that a lot of researchers they they speak to us and say hey i've got this cool idea for a benchmark but i don't really want to implement all of the you know nitty-gritty like infrastructure for the submissions and the maintenance and those things and so we've been working with some really cool partners um on hosting benchmarks on the hub directly so that then you know people in the research community can kind of use the the tooling that we have and then simplify the kind of evaluation of these models oh that is super interesting and powerful yeah maybe one thing to mention is that um the the whole like kind of question of evaluation is is a very subtle one because for example we know from like previous benchmarks that most of the time when we design a task for example there's a very famous one called squad which tries to measure how good models are at question answering well that's the aim but it turns out that many of these like transform models are really good at taking shortcuts so if what they're what they're actually doing is like they're kind of getting a very high score on a benchmark but that doesn't necessarily translate into the actual thing that you were interested in which was like answering questions and you have all these kind of subtle failure modes where the models will you know maybe provide completely wrong answers or they they should not even answer at all this kind of thing and so at the moment in the research community there's a very kind of active and vigorous discussion about kind of what role benchmarks play in in the way we measure progress but also you know how these benchmarks kind of encode our values as a community and one thing that i think hugging face um can can really offer the the community here is the means to kind of um diversify the the sort of space of values because you know traditionally most of these research papers come from the us which is you know a great country i mean you're from the united states it's a small slice of the kind of human experience right yeah i do want to ask what you've seen as a common mistake uh in machine learning maybe this is across you know more experienced machine learning engineers or teams are there any common like missteps or things that are overlooked yeah i can maybe tell you the ones that i've done yeah that's perfect probably you know a good representative of the rest of the um things so i i think the the biggest lesson i learned when i was starting out in the field is um not using a baseline model um when you're when you're really starting out so a common problem that i did and i then later saw other kind of junior engineers doing is you really reach for like the most fancy state-of-the-art model and although that may work a lot of the time what happens is you introduce a lot of complexity into the problem and your state-of-the-art model may have a bug and you won't really know because the model's so complex and so a really common pattern especially in industry and especially with nlp is that you can get pretty far with something called regular expressions and with like linear models like logistic regression and i mean these kind of things will really give you a good start and then and then if you can build a better model then great you should do that but it's great to have a reference point um and then i think the second like big lesson i learned from building a lot of projects is that um you you can get a bit obsessed with like the modeling part of the problem because that's kind of the exciting bit when you're doing machine learning but there's like this whole like you know like ecosystem especially if you work in a large company there will be this whole ecosystem of services and things that are around your application and so kind of the lesson there is like you should really try to build something like end to end that maybe doesn't even have any machine learning at all um but it's kind of the scaffolding upon which you can um you know build the rest of the system because you can spend all this time training an awesome model and then you go oh oops it doesn't you know integrate with like you know the requirements we have in our application and then you've wasted all this time right that's a good one the don't over engineer is something i always try to keep in mind exactly and it's just it's just like a natural thing i think as humans we especially if you're like dirty you really want to find the most like kind of interesting way to do something and you know most of the time simple is better totally i love that that's so good to keep in mind uh if you could go back and do one thing differently at the beginning of your career in machine learning what would it be oh wow that's a tough one hmm oh i actually don't know the answer to that just give me you might have to cut this out again no you're take your time you're good um one thing different so the reason why this is a really hard question to answer is because um i'm you know now working at hugging face and it's like the most kind of fulfilling um type of work that i've i've really done in my whole life and the question is like you know if i changed something when i started out maybe i wouldn't be here right so it's one of those things where it's a tricky one in that sense yeah but um i suppose one thing that maybe i would have done slightly differently is um when i started out i started out working in as a data scientist and as a data scientist you tend to kind of um develop the skills which are about like kind of mapping business problems to um software problems or ultimately machine learning problems and this is like a kind of really great skill to have but what i later discovered is that my like kind of true driving passion is like doing open source software development and so probably the thing i would have done differently would have been to kind of you know really start that much earlier you know because at the end of the day most of open source is really driven by kind of community members and so this would have been maybe a way to shortcut you know my path to doing this full-time i love that i love the idea of yeah if you had done something differently maybe you wouldn't be a hugging face so it's like the butterfly effect movie right you go back in time and then you know you don't have any legs or something totally totally uh don't want to mess with a good thing you know that's so good uh all right fun rapid fire question sony's get so weird so bear with us but this is gonna be really fun i'm excited to hear your answers um all right so just first thing that comes to mind um and we'll just kind of rifle through these so best piece of advice for someone looking to get into ai machine learning um just start yeah i like that just just start coding just uh contributing if you want to do open source just start you can always find reasons not to do it and yeah you just have to get your hands dirty i love that uh what are some of the industries you're most excited to see machine learning applied as i mentioned before i think in the natural sciences this is where i think that's like most exciting um if we if we look at something say at the industrial side i guess some of the development of like drugs like new drugs through machine learning it's very exciting um and i guess also i can imagine like personally i'd be really happy if there was advancements in robotics where i could finally have a robot to like fold my laundry because i really hate doing this and it would be nice if like there was an automated way of handling that totally i love that that's great uh should people be afraid of ai taking over the world maybe or um it's a tough one because i think we have reasons to think that um we may create systems that are quite um dangerous in the sense that they could be used to cause a lot of harm and this is i think an analogy is perhaps with you know weapons like you can use weapons the sport but you can also use them for you know war um and i would say that one big risk is probably if we think about you know combining these techniques with the military perhaps this this leads to some tricky situations um but yeah i i think right now i'm not super worried about like the terminator i'm more worried about i don't know a rogue agent on the financial stock market and you know bankrupting the whole world yeah that's a good point sorry that that's a bit dark no i like that i like that perspective a lot that's great the correct answer no this is great this is a kind of a follow-up on your folding laundry robot but when will ai assisted robots be in homes everywhere honest answer i don't know um everyone i do know who's working on robotics says this is still like an extremely difficult task um in the sense that robotics hasn't quite experienced the same kind of revolutions that like nlp and you know deep learning more generally have had um but on the other hand you can see some pretty exciting developments um kind of in the last year especially around the idea of being able to kind of transfer knowledge from simulation into the real world and i don't know i think there's hope that in my lifetime i will have a yeah i hope so too i know it's something i believe lex friedman's been working on the in-home robotics yeah i think he's also very interested in like the human robot interaction with autonomous vehicles yeah it's very cool it's fascinating space yeah hopefully within our lifetime we'll get some laundry assisted robotics what have you been interested in lately it could be a movie a recipe a podcast literally anything and i'm just curious what that is and how someone interested in that might find it or get started yeah it's great question so for me it's like podcasts in general are like my uh kind of new way of reading books because a lot of the time you know i have a young baby and so i'm just kind of like doing chores and listening at the same time and um one podcast that really stands out recently is actually the deepmind podcast so this um this is a podcast that's um produced by hannah frye who's a mathematician in the uk and she gives this like beautiful journey through like not just what deepmind does but more generally what deep learning and especially reinforcement learning how they're impacting you know the world and listening to this podcast it gives you this because you know the english have such great accents yeah like it feels like you're listening to like a bbc documentary and you feel like really inspired because a lot of the work that you know she discusses in in this podcast it has strong overlap with what we do at hugging face and so you kind of see that there's like this you know much bigger picture of trying to pave the way for a better future and it um quite it resonated strongly and i just love it because the explanations are just super clear and you can share it with like you know your family and your friends and say hey if you want to know what i'm kind of doing this gives you a kind of rough idea oh i love that it's called deep mind just the best yeah it gives you a very interesting insight into the deep mind like you know researchers and you know yeah kind of the backstory as well which is quite cool oh my gosh that's incredible i'm definitely going to give that a listen i love finding podcasts like that it reminds me just how you explained it a little bit data skeptic uh he distills data science topics and theories and mathematics in really easy to consume ways um very cool i love finding little gems like that good share thank you um what are some of your favorite machine learning papers i would say okay i guess it depends if we how we measure this but there's one paper um that kind of stands out to me which is quite an old paper so um the the creator of what's called random forests um this is like a very kind of famous uh classic machine learning technique um that's useful for um especially like tabular data that you see in industry and um i had to teach random forests at university about a year ago a year and a half ago and i was like okay i'll read this paper from the 90s and see if i you know understand it and it's it's just a model of clarity it's like very short very clearly explains how the algorithm is implemented and you can basically just take this paper and implement it in code very very easily and that that to me was like a really nice example of like how papers were written in the kind of you know medieval time um whereas nowadays most papers they have this kind of like formulaic approach of like okay there's an introduction there's a table with some numbers that get you know better and there's like some you know random related work section so so i think you know that's one that like stands out to me a lot um but another one that's um a little bit more recent is um there's a there's a paper by deepmind again um on using um kind of machine learning techniques to prove fundamental theorems in like algebraic topology which is like a kind of special branch of abstract mathematics and at one point in my life i used to kind of work on these related topics and to me it's like a it's a very exciting perspective of kind of augmenting um the knowledge that you know a mathematician would have in trying to narrow down the space of like you know theorems that they might have to search for and uh yeah i think this to me was surprising because a lot of the time i've been quite skeptical that like you know machine learning will lead to like fundamental scientific um kind of like insights beyond like you know the the obvious ones of like just making predictions but this example showed that you can actually be quite creative um and you know help you know mathematicians find new new ideas that is amazing we're going to definitely link to both those papers in the show notes of this so people can check that out very cool uh louis what is the meaning of life i i think the honest answer is i don't know and probably anyone who does tell you it probably is lying but um but i mean that's that's a bit sarcastic um i i don't know i i guess in some sense like i kind of i mean being a kind of scientist by training and especially a physicist you kind of develop this like um world view that is is very much that like you know there isn't really some sort of like deeper meaning to this it's very much like you know the universe is is quite random um and i suppose the only thing you can kind of take from that beyond you know being very sad is that um you kind of you derive your own meaning right and in most of the time this this kind of comes either from the work that you do or from the family or from your friends that you have um but i think when you find a kind of nice way to to sort of derive your own meaning and find what you do is is actually interesting and meaningful that that's like the best you know part it doesn't it's i mean life is very up and down right you don't have this yeah yeah but um at least for me personally the the things that that have always been like very meaningful are generally creating things so yeah um i used to be a musician so that was like a way of creating like music for other people and it was like great pleasure in doing that and now i kind of i guess create code it's a form of creativity absolutely i think that's beautiful lewis i love that it was a great answer cool is there anything else you would like to share or mention before we sign off um maybe buy my book yeah it is so good i am it has a parrot on it do you know the story about this pharah i don't think so so so when when o'reilly um uh is like telling you you know we're going to get our illustrator now to design the cover it's a secret right like they don't tell you what the logic is or you know you have no say in the matter um basically the illustrator comes up with an idea right and um in one of the like last chapters of the book we have a section or where we basically train a kind of gpt-2 like model um on python code and this was tom's idea and he just decided to call it codeparrot um and i think the idea or the joke he had was that there's a lot of kind of discussion in the community about this paper um which meg mitchell and others worked on yes yes you know stochastic parrots yeah and the idea was that you know you have these very powerful language models um which seem to exhibit like human traits in their writing as we discussed earlier but kind of deep down maybe they're just doing like some sort of like parent parroting thing you know if you talk to like a cockatoo it will you know swear at you or make jokes and that may not be like a true measure of like intelligence right it's like some sort right so i think the illustrator somehow maybe saw that um and decided to put a parrot which i think is a perfect uh you know kind of metaphor for for the for the book and you know the fact that there are transformers in it i love that i had no idea that that was the way that o'reilly's covers came about they don't tell you and they just pull context from the book and create something it seems like it i mean we don't really know the process but i'm just sort of guessing that like you know maybe the illustrator was trying to get an idea and maybe they i mean i think we have a few animals in the book like in one of the chapters we have a discussion about i think giraffes and zebras and stuff but yeah i'm pretty happy that they got a parrot i love it well it looks absolutely amazing congrats once again highly recommend everyone check this out it's a great read and it's not as kind of i think a lot of these types of books that at least i've read and have access to have been quite dry and technical and this reads almost a bit like a novel mix with great applicable technical information which is beautiful so well done thanks yeah i mean one thing we realized afterwards because like it was the first time we were writing a book we thought we should be sort of serious right but like i mean if you sort of know me i'm like never really serious about anything and in hindsight we should have been even more silly in the book yeah you know i i kind of had to control my my my humor in various places but still you know i think maybe there will be a second edition one day and then we can you know just inject it with memes please do please do i look forward to that in fact there is one meme in the book um and this was this was we tried to sneak this in past the editor so we have the doge we basically use um like a special vision transformer to try and classify what is what this meme is that's so good so glad you got that one in there well done look forward to many more in the next edition lewis thank you so much for joining me today i really really appreciate it and where can any of our listeners um find you online or follow you what platforms are you active on sure so i'm fairly active on twitter so you can just find me my handle i guess is underscore l-e-w-t-u-n um i guess linkedin maybe but i don't know linkedin is a strange place and i'm not really on there very much so twitter's your best bet um of course there's like plugin face forums or our discord um you can also reach me that way awesome thank you so much and i'll chat with you soon thanks a lot brittany bye bye thank you so much for listening to this episode of machine learning experts so louis the best uh if you or someone you know is interested in direct access to leading machine learning experts like lewis who are ready to help accelerate your ml roadmap please go to co support to learn more and i sincerely hope to see you next time thanks you