Step-by-Step Natural Language Processing Workshop: From Data to Deployment

Key Takeaways

[Music] hi everyone my name is alice i'm the director of marketing at dublin iii welcome to our step-by-step nlp workshop today how's everyone doing it's so great to see everyone from all over the world joining us today i see people from canada from nigeria from india from germany argentina that's fantastic why don't i just keep sending all the well the wonderful messages coming in start by saying hi to each other to share a little bit about yourselves where you from and what you do i'm personally very excited about today's event especially because i know this has been one of the top requests from you all for us to host a workshop we heard you if you are here today to learn how to build and deploy a virtual chatbot using nlp you are at the right place if not well whether you stay i guarantee you'll learn something new today before we start the workshop a special shout out to our co-host fourth brain if you're looking to up level your career ai career and not yet familiar with how fourth brain will be able to help you i highly recommend you check out their live online instructor-led programs in machine learning and if you are already thinking about enrolling in their programs i have some good news for you so for those who have signed up for the event today fourth brain is offering an up to nine thousand dollars in scholarship towards the tuition so up to three people will be selected to receive 3 000 each of the tuition if you'd like to learn more and apply check out the link below in the description box heads up that the scholarship application deadline is coming up on november 30 next week so fourth grade has an all-star team of machine learning instructors today we're so honored to have one of them greg lochney joining us greg is the lead instructor of the aerial ops program at 4th brain where he teaches students how to deploy scale and monitor ml models in development and production environments without further ado drum roll let's welcome greg thanks so much alice i'm really excited to be here today i'm gonna go ahead and start sharing my screen now and uh you know i'm really really pumped to see uh as i saw in the comments too like what we can do in an hour and and i'm really excited to share with you how to actually build and deploy something like a virtual chat bot like an ai assistant and and to do this uh you know really in about an hour i think it can be done and so you know today uh we're gonna walk through step by step how to actually scope the problem how to actually get the data in the model how to build the web app and how to really deploy things into the cloud uh but before but first what i want to do is i want to talk a little bit about you know why i'm here giving you this presentation and the background and the lens through which i'm coming uh into this problem and taking a look at it from and then i want to talk a little bit about chat bots in general and sort of the ubiquitous nature of these things up and coming in the world uh then we're going to talk just a little bit about some of the things that we should be thinking about as we go to create nlmvps nlp mvps in you know either sort of zero to one startup environments or even uh if you're working for a larger company and you're thinking about deploying a tool like this um so you know before we get started i'm just kind of curious you know one of the things that we want to do today is we want to make this interactive and you know one of the things i wanted to kick it off with is just kind of this idea of of figuring out how many people have actually used a chat bot uh you know we're going to use this aha slides thing today and we're going to make sure that um kind of are interacting with each other and learning from each other in real time so if you could uh if you want to interact if you want to be part of this uh go to ahaslides.com dla bot dlai bot and uh and then you know actually just start you know smashing those yes no or really can't be sure buttons now we're gonna have a number of questions coming up and and really this is gonna be an opportunity for you to ask things that you want to know and an opportunity to kind of learn to see what uh what other people uh think and what other people have done and what their experiences so i'm seeing some uh you know some great responses coming in we've got i'm going to go ahead and you know start sharing here um it looks like you know it looks like basically you know the vast majority of people uh have used uh chatbot i think you know it's hard to think about um exactly all the places that we see these virtual assistants today and a lot of times you know my inclination is that can't be sure is gonna rise in in the years to come here it's really going to rise so you know i think basically what we're seeing is that everybody uses chat bots even today and i think that's why you know as alice said this is this is one of the most highly uh you know sought after uh things today we had computer vision now we have nlp okay so yeah thank you all for that very much uh we'll go back to this sort of uh dlai bot thing for the rest of the presentation all right thanks a little bit about me um you know i basically like grew up on the east coast lived in the midwest for about 15 years moved to san francisco recently uh i came up as a mechanical engineer with then a material science uh background that got me really into computation and really into modeling simulation and data driven design of of you know first research uh and then development and ultimately you know that led me into wanting to get more involved with digital product and in sort of after grad school when i had this degree in sort of optimization everybody came to me in about 2015 and 16 and they said hey greg you know where's the machine learning uh where's the deep learning where is all of this stuff that i'm hearing about you keep talking about optimization but i'm not hearing machine learning from you and you know it was kind of this really interesting uh space to be in because i had actually studied gradient descent like very very highly in depth solving convex optimization problems you know throughout grad school but what then happened is is you know as you all know i mean that was useful knowledge uh early on in 2016 17 18. um even in the past couple years uh that's less important to know this kind of thing anymore and um you know so what we're seeing is we're seeing this growth this explosion of ease of use of these ml and ai tools and and that's what really i think is most exciting about the space today and that really is is the lens that i'm coming from today is that sort of entrepreneurship uh product focused lens so we actually are interested today in deploying something that's really end to end and the the purpose of this is to really if we could frame the context here uh you know this was kind of a prescient thing to have said in 2019 uh harvard business review article talking about like in the next decade ai assistance will actually become the primary channel like the channel through which we get not only all information but essentially all of our goods and services you know you may have heard of the uh you know we ship to shop uh or we shop to ship that's the amazon business model today but you know when the business model actually makes more sense to ship then shop that's going to happen and so like you know the what's what's really a side effect of this is that marketing itself becomes this this battleground for our attention and that's sort of where um you know this this exciting space of like of bots and you know people basically talking to us through program chat bots i mean this is this is going to be this is just going to be uh crazy it's going to be absolutely nuts and then when the dust settles um really we'll have a handful of general purpose ai platforms left and then most of us will only probably use one uh and our assistant will be incorporated into our house into our car into our smartphone it's gonna be nuts and you know this is what we heard two weeks ago from from lewis from hugging face right i really loved this quote from lewis like we use text in business for everything and this is so true it's like this is the thing about chat bots and virtual assistants is that if we can kind of automate something that right now a human does talking or texting or messaging someone then we are going to and you know there was another great question asked in this event as well and it was sort of like framed how can we build services like like grammarly right how can we build services that are like these these kind of nlp uh powered services and this is kind of the the focus of today a little bit is to say hey well you know you kind of have to know about nlp but more and more you can be abstracted from nlp and it's actually more kind of important to be able to say well what's the front end and what's the back end and then how does nlp fit in the middle and like how can we actually then take this simple app simple web app simple smartphone app and go start testing it uh with some customers this would work in as big company in a small company this would be something that you could do really anywhere and you can do it very very cheaply and that's sort of the point of today today is meant to give some insight into those digital product fundamental pieces of how to take nlp and how to use it now of course there's this other side and there's this other side that's this burgeoning industry of you know the big players in this game we have aws we have gcp we have azure and really what we're seeing is they they gave us machine learning platforms they gave us automl platforms and now they're all giving us specifically chat bot and virtual assistant focus platforms so amazon lex and dialog flow these things you know the the azure bot service is sort of the kind of equivalent there and even even recently microsoft has released what they're calling azure power virtual agents and these virtual agents these are saying what they're saying is you know this moves beyond hr beyond sales and quote to any channel or domain imaginable today this is wild stuff this is today you could do this okay and so even players like hubspot like sap these people are getting in the chatbot game through acquisition and so there's a lot of smaller companies out there a shout out to chatbot.com like that was a sweet pickup at the right time i mean these companies are gonna are getting acquired like mad and there's really like a lot of opportunity space for a lot more players to start coming into this mix and into this fold but there's this other side and there's this sort of mvp side how do we get this minimum viable prototype how do we get this minimum viable product how do we actually just do this without dumping a bunch of money into an ecosystem like aws or gcpr azure and that's what we're going to focus on today all right so let's do it let's do it um but first uh i want to know how many of you have ever actually you know built one of these things before so go to uh dlai bot and sort of like let's look at saw how many have used one let's see how many have have kind of built one um and for all of you that have built one if you could just smash the uh the comments with like a little bit more information about what you built i think that would really help out kind of all the deep learners out there and this could be a really uh a really interesting discussion so let's check this out i'm actually you know this is pretty surprising actually to see how many people have have really built their own chat bot and i think this number again uh this lower number is going to go up we're seeing you know this is great you know i really i really love this this engagement here this is awesome um yeah so so we're seeing that kind of a lot of people have built chat bots actually you know what is it one one-fifth uh one-fourth of people have actually built their own chat bot i think this number even though it's pretty high already 20 or so is gonna go up yeah so thank you for that okay so so let's do it let's build it um let's build it all right um but wait hold on hold on are we uh i feel like we're forgetting something um i don't know about you but i've i've kind of jumped into building uh in my life uh a lot maybe too fast sometimes and and you know so it's important to not necessarily start with what right but to start with why and to start with the problem that we're solving right to start with the framing of the context of why we're doing this chatbot thing so i want to do this by telling a story okay and once upon a time uh there was a product manager named paige all right and paige worked for doracle doracle was a company trying to revolutionize the fortune telling industry okay and they've actually developed a solid fortune telling platform and business so far it's a services business at this point and they've got a web app and they've got software infrastructure and customers and fortune tellers are actively connecting with one another today but really you know they've only gotten some seed funding they're still a small team paige is the first product hire after the ceo and until they can show investors that the service model is really scalable they're not going to be able to raise series a funding so in short what they're trying to do is they're trying to move from the services business to this software as a service business a so-called product led company is what they hope to become so that that will allow them to show investors hey we understand the game and we know what we need to do to become masters of scale all right so what that what they need to do first and foremost is they need to convince investors that they're going to be the company that revolutionizes this industry there's a lot of players in this game there's you know the one of the big things they can do is they can show that they know how to produce real value with the hottest thing out there and that is ml and ai first products the first order of business then is to basically help create value with those immediately and that means looking at the repetitive tasks that the sales team does today and that is the information gathering on both sides of the platform hey fortune seekers what exactly are you looking for hey fortune tellers what exactly are you looking for let's make a connection this piece this information gathering piece has just been outfitted as the first team this is the team that paige is leading the info gathering team and this really aligns with more of a rule-based classical ai algorithm approach that you know is going to allow the sales reps to basically do better at what are the highest leverage activities that they can do so rather than be focused on info gathering acquisition phase stuff they're going to go further downstream and become a true customer success team and that's kind of the the real value creation piece however they've also got a big moonshot project and that moonshot project paige is also overseeing and the plan while for information gathering is to like run it in shadow mode go through ai assistance partial automation and then fully automated version to like release sales people the ai psychic is our top secret build out the future of fortune telling and if we can position our product to leverage the latest and greatest models we hear that the super intelligent ai might be coming soon and if so we will be able to leverage transfer learning to build it directly in to what we hope is going to be something that actually accurately predicts people's futures all right so today we need an mvp though and the problem that we're trying to solve is that based on our proprietary market analysis there are not enough fortune tellers to fill global consumer demand of people who want their fortune told okay so this is the problem now if this is the problem we're trying to solve we have to look somewhere for the data and the models to get started and so if this is the problem we're trying to solve where would you look for data and models i'm kind of interested here we're gonna we're gonna sort of open this up um you know you might say anything like uh you can enter up to three different things where would you go look today to try to solve a problem like this i'm very interested to hear from all of you where you would go look for data and models um there are you know places that i would go um and there are places that we did go um to solve this problem and i love this right hugging face killing it um stack overflow uh youtube github so there's yeah there's a lot of places and i think kaggle is uh is crushing it here kaggle is crushing it uh i think that's that's primarily what we're hearing from from a lot of folks is is is hugging face google github uh yep okay very cool kaggle is king still today 2021 kaggle is king so cool uh the stuff we get to learn together today um yeah very very cool awesome yeah keep those uh keep those keep that feedback coming we'll share this feedback uh yeah kaggle is king basically kaggle is king okay so what did uh what did our team do well uh i'm gonna let you know what the team did but i want to let you know too all the resources that i show today for how to build this chatbot are available in the youtube live description right now so if you want to get to work on building this chat bot before you even see the rest of this story although i encourage you to stick around and check out the story uh then go ahead and smash those links in the youtube description below so what happens next well paige goes to the first data science hire and his name's jay jay is a recent data science uh grad he this is his first data science job he's really not a sort of a software engineer he doesn't love the data engineering side he doesn't love the sort of ops side he really loves playing with the the data that's already structured and the models so he actually starts browsing around on kaggle first but ultimately he did hear about this new really cool up-and-coming company and thinking about it he uh he decides he wants to check that out he wants to check out this hugging face company and so with that in mind he kind of says okay like you know ideally if this goes well um and we could kind of get this on the roadmap like there's a lot of cool work to do and maybe we could even hire another mle or an ml ops person so i could keep doing my my data science as the team grows um so as he's thinking paige points him to uh kind of some information about the customers and our customers really our the psychics and the psychics are the people who kind of create the culture of the platform in in large measure and so it's interesting that uh the you know paige told jay that the the community of psychics that we've had the most success with on our platform is directly from the psychic subreddit uh which is the largest psychic community forum on reddit and we we did actually pull a number of of people from other uh traction channels within reddit that are leveraging our platform today but this is really kind of the flavor of the psychic that we want to create so jay takes this information and he goes and he looks on hugging face and he he actually stumbles across a really awesome reddit focused pre-trained chat bot model and it's called dialog gpt medium and this is trained on 147 million different reddit discussions and essentially the human evaluation results on the turing test here are killer i mean this this is a great model and look how easy it is this copy pasted this code looks like it's gonna be a piece of cake in jupiter notebook um now jay knows that like the whole like to create the 27 gig structured data set he's gonna have to you know leverage almost you know three quarters of a terabyte of space and he's gonna download it over a couple days and like pre-training and retraining and all this other stuff to sort of customize the embeddings this is a lot of work and he knows that what we want here is we want to go zero to one so he first he starts to test this thing out because you can test it out right on hugging face it was super great um and it seems like hey okay um yeah sure um seems good enough right seems good enough and so he decides to move forward with this with this data and with this model and so he decides to um show us exactly how he does this so he can start the conversation with other members of the team and you can see this this video this video is on live on youtube for you but we're going to see exactly what jay does to try to solve this problem so he uses windows subsystem for linux and he uses the ubuntu distribution but first he goes to the hugging face website and the hugging face website uh is really easy to navigate he's just like okay i need the medium one i go to models i go to dialog gbt medium and let me look here okay yeah i guess all i need is that code so i'm going to go ahead and open up my windows terminal and my ubuntu terminal and the first thing that i'm going to do is i'm going to update everything because i'm kind of doing this from scratch and put in my password i'm going to go ahead and install python and i'm going to install pip i'm going to make sure that everything's good to go and i can install whatever i need to later i'm going to set up my virtual environment i'm going to create a new directory uh that i can do that in and i'm just going to call it chatbot event right we're going to call it chatbot event and i'm going to go ahead and click into that and you know so so jay's on his way the the next step is to actually create this virtual environment and this virtual machine we're going to call it chatty vm right the chatty virtual machine and once we create this we want to now activate this virtual machine and this virtual machine now that we're in we're in the virtual machine okay so we went from windows to windows subsystem for linux to ubuntu now we're in the virtual machine we want to install jupyter because we don't even have jupyter yet now that we've installed jupyter we can launch it from our virtual machine we can click the link and we can set up a new jupyter notebook all right this python 3 jupiter notebook is going to be everything that we need for this simple code from hugging face again thank you hugging face making our lives easy easier as data scientists but we notice okay wait transformers and torch we didn't install transformers or torch yet so we want to you know go ahead and open up another terminal we want to go ahead and install torch that's pi torch that you know this particular model leverages we're going to speed this up a little bit and then we want to install transformers once we kind of get everything set we've got our virtual machine set up we go back to our jupiter notebook and you know the the classic restart and run all and we are approximately now on our way okay so we're get we're getting our sort of fortune tel told here if we ask the right question uh what are we gonna ask um i don't know jay's kind of getting warmed up here will the sun come out tomorrow uh we hope so we hope so okay so we've got our data and model set up and now we've got our jupiter notebook set up now we all we need to do is we need to take it and go beyond right now we need to create this web app we need to create this ability uh to look at something beyond the ml model and so if we if we had to do this if you had to do this today i'm interested what would you choose to use would you choose for this web app to use fast api would you choose flask would you choose django would you choose something else or do you just absolutely have no clue what you would choose and you know if you can't read stuff in here i apologize for the uh you know the the size of the text but we do have the videos live and i hope that those will be sufficient to kind of zoom in on and and to help you out uh to build these on your own and uh and i encourage uh feedback like that continue to keep coming so this is pretty interesting um you know we're seeing a lot of our deep learners out there uh you know flask and i have no idea uh very very uh competitive here i'm not sure which one's gonna win this is uh this is uh pretty interesting right i mean i think i think flask is the one that many of us hear about often and it's also the one um that we we tend to go to as data scientists first at least today in my experience with the data scientists that i've worked with i have no idea crushing i have no idea crushing um very very cool so uh yeah so this is this is i think speaks to the importance of doing this so i'm glad that we sort of align this with what everyone kind of expects jay takes his his jupiter notebook to the to brent and brent is the software engineer brent is the software engineer that does it all he does back end he does front end he does everything in between he is a true devops savant and he also does front end he doesn't really like believe in ml and ai he really feels like it's kind of like all hype right he he's really only interested in picking up tools like if they're gonna solve the problem and if they're gonna solve the problem on the job that he's working on today he's not a fan of buzzwords he's not a huge fan of this ml ops thing and he really aims at becoming the indispensable member of any team that he joins he's a long-term guy he's the brent right and he likes to make things happen so when jay comes to him and says he thinks flask like flask like brent's like okay that's fine i haven't used flask before but i'll figure it out real quick so brent realizes though that like he has to figure out a little bit of back end a little bit of front end and he's like he hasn't even told me that he's done anything with the front end yet let me see if i can go pick something off of github that looks like i could use it pretty quickly and so brent finds uh this um gotham chatbot that has you know one uh star on it uh so you know smash gotham chatbot um that was created at the right time on halloween in 2021 for him to leverage and so you know kudos to you gotham chatbot creator um you know really appreciate uh letting lev let oracle leverage um this code and so that's the beauty of open source uh and brent sits down at jay's computer and he says hey move over let me show you how to do this so he says okay first what we need to do is we need to get git installed and then once we have git installed we can actually clone our git repository the gotham chatbot repository and like just check this out super easy now that it's in there like look it's in there with your untitled notebook j great job with your untitled notebook uh you know and we're just gonna go into this and we're just gonna run it and you know jay's mind is blown because uh brent already has this thing up and running and but he's asking him he's like you know hey i'm brent uh but this bot i am what am i jealous of uh you're not too bright are you but you know what is a.i uh this bot this bot is the chatterbot bot and it's really not gonna work for our fortune telling um and you know this so we're gonna go ahead and we're going to like copy paste this file folder called gotham chatbot we're going to call it dialog gpt chatbot and what we're going to do is we're just going to look at both the requirements file real quick and we're gonna look at the um the application file so brent comes in and he's like hey uh you know we're not gonna need this chatterbot thing um you know as i look through i don't really care about any of this other stuff but i know that we'll probably need to i think i saw in the transformers documentation we'll need to we'll need to upgrade uh this uh this yaml version and then i'm gonna add torch and transformers okay good to go requirement's good to go now uh here's my api uh where is that jupyter notebook that you that you had before oh yeah hold on let me grab this and i'm gonna just straight paste this into my little flask app uh this is in uh you know just use notepad plus plus that's fine i've got you know vm code on my machine i would use instead but whatever do what you want jay um and you know i'm going to go ahead and uh uh just replace the library installs i'm going to replace the tokenizer in the model and and this is just where we get the the bot response so here's where we just go ahead and get our bot response looks like user text is where we should put this input to leverage this ui it looks like we don't need to spit it out kind of generically rather we can spit it out through the ui in this form of a string and then yeah i guess we don't need that and we just need to you know tab over and we should be good to go let's check out this index because i think it said gotham chatbot we're going to find that in templates and we can open that just in notepad as well let's make it you know something that looks more reasonable for what we're trying to do i can predict your future uh ask me anything um you know i tell fortunes great okay um so now we're set up brent says okay that's it we're just gonna shut that down and we're just gonna go ahead and get into the next folder that we just created dialog gpt and all we have to do is we just have to run it just like we did with the gotham chatbot again don't do this for production jay but it's fine to test with page and with customers so let's see uh hi dilo gbt i'm brent hi brent uh you're brighter than batsy uh i'm not ah well played well played dilo gpt uh what do you think of jay i think he's a good guy um will i become rich one day uh you will become rich one day uh brent's like yes okay so great um and with that i want to kind of see um we can take a couple minutes here before we go into the next phase of this thing to see if anybody has any questions that uh that might need answered so we're going to kind of take like a bit of a bit of a break here if you saw anything that kind of uh confused you or that you wanted to dive a little bit deeper into uh we'll give it just a minute here or so to see if we have any questions that we can answer and sorry for all of you that cannot see the code we will okay so go ahead and you can start up voting these things let's see all right we'll wait a couple more um okay let's see what we've got here so how do i find so okay how do i find data sets specific to the business that i want to implement the chat bot for so this is really uh where the rubber hits the road for value creation i mean this is where we really have to work hand in hand with the business leaders and with the people who know our customers the best generally in a startup environment that's going to be your your product manager it might be your user researcher it might be part of the design team but you want to understand you know who the users that you're building for are this is the most important thing and so to find data sets specific this is what takes the creativity honestly this is what takes the human creativity um and this is the real piece that that you know you we're never going to be able to automate quite frankly i mean this is the piece and so um this is a tough problem and when you can solve this problem that's when you're really creating massive value uh for your company right for your for your team uh and then hopefully for your family for your community and for the world more generally so yes a very great question um it's it's a tough problem uh i think you know how to do this in foreign languages i think we heard a little bit about this two weeks ago in the ama uh this is very tough to do in foreign foreign languages now um there's a lot of uh there's there's a lot of work going on um in that space but right now i mean english is by and large the absolute um the the absolute leader in this space and so i'm not sure if it would be best to sort of train it in english and then try to convert it to something else later but you can imagine stacking these uh translation tools on top of these prediction tools in various ways as they both continue to mature so that's also a great question you know can we fine-tune the model for a specific domain i mean this is exactly where we want to make sure the answer is yes and we want to make sure that this is going to be a useful thing to do when we send jay to spend you know a month working on this developing custom data from our customers flavoring the model with oracle specific information so we want to make sure that we've got an ability to say we're going to spend this much money on it it's probably going to be worth it in the end and so yes we can and yes we should but we want to make sure that the business case is there first before we start um putting a ton of jay's time into it okay very very cool thank you all for your great questions we're gonna have another opportunity for questions at the end of the session here uh but for now we're going to continue the story um and uh and you know i see do you have a step by step written up i you know for that being such a high question i am going to write that up um and make that a little bit more clear in the github repo um rather than just the videos so uh jay and brent continue the story jay says hey hey nice uh did you say we're gonna use docker to deploy the web app like can you show me that brent that'd be so cool like and he says yeah we'll use container images like download docker desktop like we'll keep it easy we'll keep the gui base and i'll show you how to do it on your computer um so they continue the conversation and brent says hey remember how you had to set up a chatty vm from the ground up with uh python with pip with git with your nlp libraries and all that jay's like hey for sure and we did that all on ubuntu on windows subsystem for linux right the second edition and brent's like exactly uh so we'll effectively use chatty vm you know through ubuntu as our operating system to deploy containers locally that's what we're going to do first and jay's like sounds simple enough so you're saying we'll actually put our containers inside of our virtual machine inside of chatty vm and uh brent's like right just like any other computer here i'll show you um so brent goes and uh and he's like hey just move over thanks for downloading docker desktop let me show you how to do this i'm going to go ahead and create a new folder i'm going to call it dialog gpt chatbot dockerize i'm just going to create a new folder here called source and i'm going to put everything except the requirements in there i'm going to show you how to create this docker file now too and the docker file is the key file for docker right no kidding and what we have to do is we have to create this docker file first we're going to take python from docker hub we're going to set our working directory this is within our container within our container we're going to copy the requirements document to that working directory in our container we're going to install the requirements from the document by running our classic command to install requirements docs then we're going to copy the entire flask app back end front end stuff it's all there and source and we're going to call the application and then we've got this port that we're going to expose uh 8080 you've probably seen this before this is sort of best practices for for simple flask apps and so we want host to be zero zero zero so we can have you know external ips accessing this thing and the uh what we're gonna do is we're gonna basically go into now this new folder and we have to call now one new command and it's docker build and you see we're kind of speeding this up here once we build it there it is we can see it um and we can see that it's almost you know four gigs large and we can just straight up use the docker desktop app you know port 8080 to run this thing and we can kind of look and see how it would look in the terminal as it's as it's moving as it's operating um and then we can boom uh hit the open it up and you know at this point uh jay and brent are kind of starting to believe um so we're starting to get a little more complicated uh will my fortune be like uh perfect thanks thanks fortune bot um ah fortune bot okay always savagely messing with me the reddit trained fortune telling bot so uh at this point um let's pick up the conversation where we left off with brent and jay where brent said yeah yeah we'll just like any other computer but then we're gonna ditch chatty right we're gonna ditch it and we're gonna create a new vm in the cloud we're gonna download and install everything that we need right there in the cloud and then jay's like yeah then we'll have a public link right to test with our customers we can give it to paige and you know so jay's starting to get it jay's starting to get it all right so you're you're jay um what are you going to do uh what are you going to use to solve this problem you need a cloud platform and we all have our preferred cloud platforms and i think it's going to be really interesting to see where your minds go as deep learners out there uh in terms of cloud platforms that you feel like we should be leveraging today in 2021 for models and for products like this all right let's check out the race here okay we've got a head-to-head race uh full on sprint between google cloud platform and aws aws has had an early lead here and gcp seems to be you know competing bringing it uh azure uh trailing something else very much trailing um aws it's hard to compete it's hard to compete with aws um you know very very cool again if you're looking to uh contribute to this discussion go to ahaslides.com dlai bot and and we'd love to get your your feedback on which cloud platform you would use so it's kind of a toss-up it's kind of a toss-up and you know in this case um there's really no decision to be made for doracle well why is that um well because you know as as brent says hey we basically use gcp for everything so like we're using gcp jay um most of our infrastructure is on gcp and so like there it is i already have an account it's already set up for billing i'll just show you how to do this real quick and so basically that cloud infrastructure is what we're going to leverage for this so jay says okay let me get this straight we're going to create a virtual machine in the cloud which has its own operating system brent's like yes correct then jay's like and in that host os we're going to leverage docker to create a containerized application by building a remote image container in the cloud and brent says you know you're really starting to sound like a developer jay like i like it keep going my man keep going so brent says okay stick with me i feel like you're a fast learner so i'm going to go a little bit faster this time gcp let me show you how it's done so gcp we're just going to go to our console again you have to have this set up for billing i was messing around with some stuff earlier we're going to open the shell editor and we're going to make this full screen you can see we're starting our virtual machine we're starting our cloud instance and we're starting our editor and we first want to just upload exactly that dockerize folder that we used locally and once we upload all of the items in that folder again this folder is available for everybody out there on github we want to click into that folder and then simply we just have a couple commands to run the first one is build the container image in the cloud gcloud builds submit simple as that we want to authorize this api call to gcp and this is kind of sped up here and it says hey don't run on root it's like hey we're not running on root it's just root of the vm in the cloud so like don't worry about the red text brent says to jay but it takes a little while here to create the image and now we've got one more remember we've got four gigs of container that we need to actually deploy to the cloud and it's a 256 uh megabyte um you know default here so when we run the deploy command we want to make sure that we further specify you know we're getting a pretty long-winded folder structure title here so dialog gpt chatbot docker eyes gcp but let's go ahead and specify that we need those four gigs we need all of them and now we don't really need a service name so just hit enter yeah we're on the west coast so let's go ahead and do it 25 that sounds great allow unoff authenticated invocations yeah sure um this is a you know this is a zero to one and look we're spinning we're spinning we're spinning we're gonna speed up the spinning up and oh man like we're almost there we're almost there jay's getting pumped jay's getting amped uh jay is he's like oh my gosh are we there are we there um and uh brent's like yeah let me let me let me show you man here and if you don't believe me let's go ahead and just open up a uh an incognito tab just like make sure this isn't like some local thing now go ahead and send it to page send it to your friends uh let's play with this now we're getting uh let's ask it some classic fortune teller questions here so what will my finances look like in the future right um you'll be able to afford a house and a car savage reddit nice uh will i get the promotion at work i'm sure you will okay excellent this fortune teller is gonna be killer uh what are some things you can tell me about my partner that's a tough one um hmm i'm not sure what you're asking oh okay well played well played um let's see what can we ask it let's maybe some classic like when will i when will i what i don't know some classic ones what are they have a child um when you have a child yeah nice nice okay when will i um when i get married uh when you're ready oh excellent and then you know brent says hey look so here's what you can see let's call this monitoring j this is level one monitoring you see this green hump yeah this is what we just did so boom now you know how to monitor too of course you could create custom stuff as you go from one to n you could do all sorts of uh different things as you go from one to end but um no that's not really what we're doing today so product data science and software success this is uh jay and brent are like hey paige check it out built and shipped she says hey this is so awesome guys uh give me a few days to start some customer development activities and feedback coming soon right feedback coming soon so as as jay and brent leave work jay thinks now i hope users don't ask questions that are too hard right and brent's like yeah i hope that like only a few users ping this thing at a time this thing is pretty fragile it's like a little fragile um infant product um and i'm not sure how well it's gonna stand up to abuse so the next day jay comes in and paige is like hey that's it for the ai psychic for now follow up with the dev lead on the info gathering team for next steps like i'll bring you back in once the ai psychic has some more movement like over there they're not building something brand new though so no ground up prototypes needed jay they're working on something that exists already so go see how you can help them look at the data that exists investigate technical debt and try to figure out how to make iterative improvements to top level kpis that flow up to business objectives uh and you know jay's like yeah yeah sure page um he thinks like maybe i should have taken more time on stuff i actually like um totally could have spent more time researching the fortune telling domain and like training custom embeddings and all of that well hopefully she'll be back and uh you know i'm happy to report that one month later uh doracle raises their series a they impress the investors with their new ml first and ai first products and their adventures from data to deployment and beyond and beyond uh the adventures continue okay so if you want to compete with oracle or you want to build your own chat bot again we've got the links uh in the description below we've got youtube videos for now but i'm hearing a lot of uh feedback on uh the the comments about how can we can sort of make this more step by step i'm gonna go ahead and make those updates over the next day or so to the github so look out for that but i want to go ahead and just thank everyone uh feel free to reach out anytime greg at fourthbrain hit me up on linkedin or on twitter and um you know last but not least shameless plug for the ml offs cohort that starts uh uh next uh you know early next year if you like learning this stuff basically we went from zero to one today and we have a cohort program that's active right now with uh you know many of these beautiful people that you see right here um all these ml practitioners here that i get the privilege to facilitate their learning we are actually gonna not just deploy and test zero to one in here but we are gonna figure out how to scale and manage uh horizontally on the on the uh you know usage side as well we'll figure out how to set up custom monitoring dashboards and how to think about bigger infrastructure plumbing tools that we might consider using in the future so we've actually got a special scholarship that we want to um you know let you know about alice mentioned it in the beginning uh you know half off basically for for three people that are here today so check out uh fourthbrain.ai dl scholarship and um and so with that i'd like to kind of give a final round of uh any questions so we'll open the questions back up and we'll bring this back for just a few minutes here before we close out the session so can we fine-tune the models we've answered that uh step-by-step instructions written up i will definitely do that i will make sure that that is there for everyone over the next uh 24 hours um why was the model not trained on reddit data so we actually uh took a pre-trained model that was trained on reddit data so this data is actually sort of um implicitly in the model already and so um this pre-trained model embeddings that we used was those were learned from reddit data directly we just didn't have to do the training ourselves because that would have been a massive massive um undertaking and uh and certainly be on the scope of one hour so feel free to you know pop some more questions in here if you have other questions um hugging face hugging face has it all so i would encourage you to check out hugging face for all of their models they've got tons of bert models uh tons of pre-trained models they've even got moved into computer vision and all sorts of other things they're a fantastic resource um and you know plus one for hugging face um great stuff they're doing and this is the kind of thing that enables all of us to sort of leverage uh to create value out there in our in our jobs uh for the companies that we're working for and that we're trying to build so um yeah they've got a lot um gpt and bert and in between all right let's see how can we actually create our own chatbot um so yeah i'm not sure uh exactly what what's meant by that um but uh yeah i think i think basically uh you know it is kind of related to this other question of what would the best approach be to fine-tune the dialogue dialogue dilo gpt model to another domain like psychology well i think the first thing you want to do is you want to basically figure out how to curate the right data for that fine fine-tuning like you have to know again speaking of psychology you have to know sort of the psychology of your user of your customer of the people that you want to leverage this sort of bot so that's why when we talked earlier about some kind of like microsoft azure said we have this bot service but then we have this power virtual agent service and the difference between the two is bot is for customer service and virtual agent is for beyond hr for sales and fundamentally what you need is you need dialogues between people you need dialogue data um to to really help you with chat bot stuff and so if if you can find that dialog data in your domain fantastic if you can't then it's the kind of thing that you want to start figuring out how you can go and try to either collect try to buy some data try to curate that data try to get you know people to create that data for you like the amazon mechanical turk sort of methodology and you know really um this is this is the challenge is how do you create something specific from sort of a generic you know previously we thought object detector but this is now sort of like a generic kind of you know sentiment analyzer document uh you know reader or or uh you know chatbot uh now today is a lot of this things that we're thinking about so look very carefully at where your dialog data comes from and think about which dialog data would be best and then if you don't have it start collecting it today especially if you work in a business that has lots of users and you can set up something that might give you the data you want to get in the future so okay so maybe um yeah so maybe uh using a pre-built model comes with bias um yeah so is there a de-biasing phase well so i think uh you know i think bias is an important thing to consider uh you know in all cases and the fact that you're even targeting a particular domain is a bias towards that domain you know i like to think about this idea of of bias is you either have sort of like you know you're going to have one or you're going to have many or you can have something in between right um when you go to the the supermarket right you've got you can have like one choice for ketchup or you can have like a hundred choices for ketchup it's like how many choices for ketchup do you want um that's kind of you know you want to balance that bias variance trade-off and so when you're thinking specifically about your problem in your business case and exactly what it is that you're trying to solve that's where you're going to be able to put that bias lens appropriately on the data that you're using from a pre-trained model perspective or from the data that you're using from a from the ground up model perspective and then when you combine those two then it starts to get even potentially more complicated so this is this is a big problem that there's a big opportunity space for lots of you to help out companies do this type of work in the future so um yeah thank you all so very much for your great questions i'm going to go ahead and stop screen sharing now and i think i'm going to go ahead and invite alice back up on the stage to uh to close it out for us hey greg thank you so much can i just say that was fantastic that was such a great presentation thank you paige jay and brent love the personas love the user story thank you so much and uh thank you everyone who joined us today i know we only had an hour and sorry that we're not able to dive into too much details here but feel free to just check out all the playlists in the description box and again highly recommend you check out the online programs on fourth brain and um again if you missed any part of today's workshop we will be sending out a follow-up email including a copy of today's slides and a recording um and again uh if you're interested in signing up for the scholarship program the deadline is coming up november 30th next week so again thank you so much everyone thank you greg we look forward to seeing you again very soon and as always keep learning bye