Large Language Models Bootcamp- Information Session

Key Takeaways

um you know any um any will take any questions curriculum Labs any resources that we provide and U and and so on so we'll go ahead and get started um my name is Raja kbal I am the chief data scientist and one of the lead instructors at data science Dojo I'm going to go over the the large language models boot camp and um pretty much uh anything any questions that we have around uh the boot camp uh the what you will learn what uh is expected what the boot camp is about and what the boot camp is not about and so on okay so let's go ahead and get started uh so we are one of the oldest and one of the most uh I would say um um established and respected players in this space um we have been around for quite some time uh have done um a lot of uh Enterprise trainings a lot of uh companies are um me more than 3,000 uh global companies on our portfolio um we have been doing this for quite some time uh but we have been known primarily as the name suggests I mean data science Dojo so we started off as a data science company and um uh about a year and a half ago we started doing our large language models boot camp and uh we have had quite a lot of graduates uh from our large language models boot camp as well and this info session is primarily about the large language boot camp so how did the boot camp actually start so uh it goes back to about two years ago uh when uh we started helping companies we have a we have a Consulting arm and we have a Services solution uh Solutions arm of the company as well we started uh as gen started becoming more popular chat GPT um and um large language models in general when they started becoming mainstream companies naturally they wanted to build um large language model applications um so initially we started with some very basic uh conversational agents and then as very quickly we realized as the use cases started becoming more complex that uh this is the whole uh business of building a large Lang language model application it is actually quite non trivial um going and asking a question and using chat GPT it is very different when you actually start building an Enterprise application something that you will be using internally something that you will be using um uh in your own Enterprise set setting because there are actually a lot of issues that you will run into when you start building an Enterprise application um from our experience uh um U maybe I can I can give some context um we started with some services and solutions that we were providing and then we ended up building a product and we have now um a a product um that um many companies have uh adopted internally and um you know and I'm not talking about some small companies I'm talking about some big companies 500 billion dollar Revenue companies that are actually using a Sol a platform that we have built uh they build their own application their own agents using a platform that we have built and then uh what we realized over time is uh it takes actually a lot more than just knowing the basic you know the AI piece of it um for instance when you start building an llm application there are going to be uh challenges around data privacy and security I mean broadly let's call it data and AI governance you have proprietary data you have pii you have uh uh you know you have Healthcare uh data that is uh setting there you have email addresses you have some proprietary data you don't want your uh employees to go and use chat GPT and upload uh any anything that uh may have business implications um in addition to data Andi governance there are other problems I mean should you use open source models or close- Source models um how do you handle hallucinations uh maybe you are in a regulated industry how do you control hallucinations or how do you know except the except the fact that well the models May sometimes hallucinate how do you prevent how do you put guard rails on top of it um how do you actually uh connect uh your large language model application to different kinds of data sources um how do you um get your llms to reason uh and um you know break down the problem how do you build a multi-agent system and once you have built everything how do you set up your evaluations uh evals and also uh um how do you actually make sure that your um model after changes in prompts or rather how your application or llm application after uh changing of prompts uh or um any data uh changing or any uh CH using a different model how do you ensure that your uh your performance the model's performance is not degrading uh there are a lot of issues and that actually uh resulted uh you know from out of that uh realization that uh that everyone should learn the these best practices that we have learned um over uh over the last two years uh we decided that we were we are going to do a boot camp uh the boot camp is actually uh a 40-hour um uh 5 day 40-hour training uh it can be done it can be in person it can be um online the same training uh the people who are attending in person um they will be in a in our facility in the training facility in Seattle the people who are attending online they can be based wherever they are and they connect remotely um and what we take pride in is that uh the curriculum has been designed by practitioners um it is not something um you know it's not it's not some fluff that you will you will see actually a lot of math or a lot of you know jargon and all of that um the the boot camp is designed by PR practitioners to turn all of you into practitioners so it is with uh that in mind that uh it is with the intent in mind that anyone who attends this 40-hour training uh they will start and they go all the way to the end and without uh you know after the boot camp they will be able to go and build llm applications um so we cover um most of the mainstream tools and libraries and packages uh very Hands-On boot camp I will show you the curriculum in a moment I will also show show you the Hands-On exercises how we have structured them and uh and on the last day of the training we have a project uh everyone gets to build their own llm application um and then deploy it uh in um uh in a in the cloud um and then we have some exercises you know of you know basically what we have learned over the uh four uh the first four days we go and Implement those uh in practice um uh we also have a partnership with the University of New Mexico uh in terms of the certification and what it does is it makes us eligible uh for the your L &d budget so if you're working for a company that has your annual uh L&D budget uh we many many uh people they have many uh attendees they have attended trainings um using their L&D budget uh so if you are in doubt if you work for a company that has uh that's a budget that is eligible uh you know if you have some Learning and Development budget that you can uh spend for about courses since we are a u we have we are delivering this training in partnership with an accredited uh University um you may be able to actually attend this boot camp for free so please reach out to us if you have any questions about that um so um in terms of the infrastructure bring your uh bring your laptop um all the everything else we uh we take care of this if you need a GPU Cloud for model fine tuning we take care of that for you we'll give you the GPU cluster uh the llm uh token consumption cost we give you the API keys so you don't have to go and spend uh you know your or slide your own credit card and uh uh for uh andb build um we have a complete infrastructure for actually getting uh getting the job done um okay so let me go up for the next boot camp we so we are partnering with many companies uh who um are at the Leading Edge Cutting Edge of uh of llm for the Seattle boot camp that is happening in February um these are the companies we is a leading Vector database in Industry neo4j many of you would would know um you know they are also now venturing into the llm uh space uh Lang chain Union and and so on so these are our partners uh who provide um uh necessary tools and necessary expertise for making it uh you know high quality experience for all of you uh we have guest talks from real industry uh leaders uh who will be telling you how they actually build Real World Systems um and then um I think U I will go and walk you through the curriculum so this is our large language models boot camp if you look at this uh at a very high level uh we talk about the entire stack if you're on the left rail when you see on the learning platform you can see that uh there is a variet topics that we are covering um we start with uh embeddings uh actually let me backtrack uh we start with a very high level endtoend overview of how do you build a large language model application maybe I should actually show you this end to endend architecture before I even go to that detail so if you look at this here so we talking about you know we talk we first talk about the bigger picture what are the various moving Parts you need at large language model uh well what is a large language model uh you need to have prompt uh prompts well what are prompts a vector database you need embeddings you need um um what is the difference between fine tuning and rag what is rag what is fine tuning what is pre-training uh why do you need guard rails so all of that we set the context in the first three hours of the boot camp uh Monday uh 9:00 a.m. to 12: uh um 9:00 a.m. to noon we are going to teach you the end to endend the bigger picture of what how do you build an large language model application so uh in that um the entire boot camp uh you know we we talk about U uh we'll talk about embeddings or llms and what kind of llm uh uh you know you can take closed Source hosted llms or you can say take open source and you can deploy those uh llms on your own we have exercises for both uh then uh we dive into quite a bit of detail of vector databases uh Vector databases are crucial when you're building a retriable augmented generation or simply known as rag systems uh we also talk about embeddings model uh so what are the different embedding models and you know um so first of all stand embeddings in detail and then we talk about um you know how do you use them and now um llm caching um you know how do you log and monitor uh your models you know the uh different kind of uh monitoring how long it took um How Long llm Function call took um what how did it reason what was the U what was the back and forth communication with llm was a vector database set or not and then putting guard rails on top of it so these are all different components that you are going to learn during the boot camp um we also talk about actually uh Lang chain in quite a bit of detail and I will explain in a moment what Lang chain is so think of it as just more than a single um much more than a single um um much more than aing single you know uh a module right so it's a complete end to-end experience so for instance when we go to um embedding uh and Transformers uh and attention mechanism we have we cover a lot uh in depth when it comes to let me go and see here so if you look at this we start with we don't assume any background so a common question is do you um uh do you have to have a background in Ai and machine learning ahead of time well while it can be helpful but you don't and we start with very simple um beginner level examples and we slowly incrementally uh if you can see my screen we talk about you know well how does a neural network work how does a deep uh neural network work and how does an llm and how is llm uh similar to a deep neural network my slides are actually taking time to load here uh but you get the idea and the we uh we eventually go all the way to uh attention mechanism and encoder decoder architecture and and so on so we we do this in quite a bit of detail then we have Hands-On exercises uh then for instance uh when we go to Vector databases now now that you understo understand what edings are um then you go to Vector databases and then you find out okay so what are uh you uh well um um you have embeddings where do you keep these embeddings well you keep these embeddings in a vector database well what is a vector database so uh in that case uh well a vector database is um let me see this is interesting the slides actually for some reason are loading a bit slow so if you look at this well how does a vector database work and why do you need a vector database how is Vector database different from your traditional relation database um and and then um basically an intuitive understanding of what uh how Vector database works then uh you know different types of searches a vector search a keyword search a hybrid search how do you add filtering in your vector database um how do you index uh different indexing techniques like hnsw we talk about that in detail you can see that this is a very very extensive uh understanding of how how things work in a vector database uh once we are done with the vector database piece uh the theory side of it we also actually have exercises um on the respective areas so uh remember that this is a very very Hands-On uh boot camp it is not uh only slides it is also some practice um so once we are done uh we talked about Vector search and you know um keyword search and you know hybrid search and you know generative search and all of that and Vector compression Etc uh we talk about all of them and then we actually go and uh we go and uh do these exercises in Python so these are uh you know this these are Jupiter notebooks um just like you would have um you locally we have these inow uh in web browsers set up here and now what we will do is uh we'll discuss here is we are collecting creating a collection here is we doing something else uh you know we are searching for a vector uh once that is done let's say let me go to hybrid search for instance right so if I click on hybrid search now we have talked about this in theory we come in and we talk about this in practice and if you look at this uh you connect with uh this Vector database then you create a collection of what is a collection we talk about it I mean think of this as a table in uh in um in a vector uh database just like you have a SQL table you have a uh collection in Vector database then we are creating these embeddings from this file that we are reading um then we are setting this up uh showing you some hybrid search queries and so on so you can see that and we spend uh about two to about two and a half to three hours on the theory side of it another two and a half to three hours on the practic Practical side of it uh we change the queries see how the results change uh and so on so it's a it's a very very good detailed overview of what Vector database uh bases are and not just at a very high level we go take it all the way to Vector compression all the way to semantic caching some advanced concepts um that uh uh that will be needed when you're building an actual performance uh application vector compression as the name might suggest that you're compressing the vectors in a certain manner the sematic caching is when you're caching the queries so they are you when you're building low latency systems um and then we go and talk about sematic search we talk about uh building a rack system uh retrieval augmented generation system and uh it's not a very high level that uh you know you have to have an embedding model you have to have a vector database and then generation model we talk about uh the smallest nuances and smallest challenges that you will have in building uh in llm application I mean think about this that uh you know very nuanced reasons how you know going Beyond just a basic uh things like how would you chunk um your documents uh we talk about things in a lot more uh detail very nuanced I have I hope I have the slides here let me see if these are the correct slides um yeah and we talk about a lot of if you look at this we talking about data quality we are talking about chunking we are talking about various chunking strategies we are talking about uh you know um how do you do things at scale and uh you know how do you handle um pre- retrieval how do you alter queries um uh we talk about query alteration and query optimization stepb prompting and query rewriting and I can keep going this itself is a very very detailed session um that of course I cannot given the time that we have we will not be able to actually finish all of it uh we actually spend quite a lot of time on Lang chain um Lang chain uh for those of you who may not be aware Lang chain is your think of that as your um the entire Plumbing of your um of uh your llm application uh you start with uh we start with the very basic idea how do you connect your model uh to uh uh different uh connect your application to different types of llms um then let me actually go here how do you connect your uh llm to different types of models how do you create prompt templates uh how do you give it few short examples and the way we have structured this is let me click on only one of these Lang chain is about almost this is a very very comprehensive overview of Lang chain uh we cover Lang chain in about I think it takes us about 7 to eight hours just to finish only the Lang chain component and you can see that in this case this is an example of uh uh a few short uh uh examples uh um and then we pushing them into an inmemory Vector database and so on and we go line by line understand it uh and at this point uh many people have this question hey would I be able to do it I don't have a uh background in programming yes you should be able to do it because uh as long as uh I mean you don't have to be a coder as long as you can read code because all we do is these are pre-created code samples um if you are a Dev you you can save make changes you can extend them uh to uh add more complicated Concepts but if you are just a technical PM if you are just a project manager if you don't write code actively you want to get a very good understanding of how these things work without actually having to write code you can still do it because all you have to do here is uh hit run and hit run and hit run and then you keep running these and this is asking you for the API key um but uh we give you the API key you plug it in here and we discuss in class what this is doing once this is done you go to a different lab uh once that is done then we go to a different module in L chain we are talking about retrieval uh well in retrieval uh we do all these exercises we talk about how do you connect to different Vector data data stores uh or vector stores uh how do you connect it to different um you know how do you write different kind of retrievers how do you connect them to different kind of uh data sources Dropbox SharePoint how do you uh connect it to uh you know your Gmail and all of that uh how do you uh load PDFs how do you load uh a text file how do you load a Json uh how do you split uh chunks uh and so on uh then we talk about chains uh when you have to execute a series of functions or a series of llm calls backto back um uh we learn it uh we talk about memory memory think of it as uh uh when you go to chat GPT when you are you ask one question and if the next question is related to the previous question how does the model know that the next question is related to the previous question uh we also talk about multi-agent systems we uh discussed langra in detail in depth uh we talk about uh um you know agent uh um applications let me go here maybe I can not here but here um exercises here so if you look at this uh agents and tools uh working with agents let me see what we have here and uh once again we you have the code we go through this you are going to we are going to go and uh you know keep going all the way uh enter the key and you can see that right now I have two Tool uh tools here I'm using a Search tool I'm also using a calculator tool but uh you know then we talk about uh you know even more agents we talk about langra uh and when you talk about langra langra is more of collaboration uh basically what we call the agentic systems we talk about these agentic systems in a lot of detail um and we explain what is happening here and what is happening I'm very confident that by the end of this pretty much you know all attendees they are very very well worsed in how Lang chain Works uh uh then we go and we talk about fine-tuning we uh we uh go through fine tuning in a quite a lot of deta quite a lot of detail all ideas you know why do you need fine tuning well because you need uh to the model to do well on uh out of distribution data right so and well how is it different from transfer learning we talk about uh more uh comp uh numeric optim numerical optimizations and efficiency uh improving measure measures like uh quantization and low rank adaptation we talk about uh Lowa quantization qora uh we talk about challenges uh in um in fine tuning like replaying data and uh um and challenges like uh uh catastrophic forgetting and all of that so we talk about all of those things and then we go and um give you a GPU cluster we give you the sample code and you know some sample data set all of us we fine tune uh every attendee uh you know the instructor actually launches this lab um everyone attendees in their own cluster you you go through this line by line here is this is happening this is happening and you compare the responses of an unfin tuned model uh or the base model and compare the responses with a fine tune model uh we spend uh a lot of time on evaluation we talk about evaluation different kind of evaluation metrics different kind of uh uh um we start with the language translation metrics and take them all the way to um more rag related evaluation metrics like ragas uh you know the context relevance uh um answer Precision uh answer faithfulness and answer relevancy and so on uh um what else and more there's uh and then we have exercises for that as well I think any time I say that we are doing something assume by default that there is an exercise so if you look at this uh this is the ragus exercise that we have so ragus is a package that evaluates your large language model um or rather your rag application so if you look at this uh you know we go through this line by line we talk about these metrics uh you can see that we are creating a prompt template and by the time when people reach here on day four they are pretty well versed and the in I actually enjoy teaching I teach about 30 40 40% of the boot camp and then we have other instructors as well along uh with uh myself um I enjoy it because by this time the kind of questions the quality of questions that start to happen even from the P so we have had people who did not have any background in machine learning at all and they did very well um despite uh you know not having a machine learning background you do not have need to have a machine learning background to to venture into llm um business and you can see that now we are actually calculating the different metrics uh you know what is faithful faithfulness relevancy and recall and precision we have already talked about this in our Theory session the slides and now we are doing all of this in action uh then we have a session on uh setting up monitoring and guard rails uh and then same drill have an exercise and then we also have um um a session on deploying um your fine tune model um as a service um so that's at a very high level uh what we do and on the last day we do a project we have a we put all of this together we everything that we have learned we put that together in a project once again everyone everyone goes ahead and deploys this model uh or deploys their own llm application and then they tweak the application like adding chain making it more agentic and so on um let me see here we have uh uh as I mentioned there are no prerequisites I mean a working knowledge of Python Programming would be helpful but even if you don't know Python Programming um we can we have some tutorials that can actually get you started with Python Programming without much difficulty this is roughly the technology stack that we use you will learn a little bit of all of this that you see on the screen in different uh different uh levels of uh where it belongs I mean llm Foundation models or large language models plus the embedding models Vector databases um and um you know different tools so this is our uh stack I've already gone through the uh this uh I've already gone through this uh this outline so I'm not going to go it bu through it Bullet by Bullet we have talked about Vector database I've talked about semantic search already prompt engineering fine tuning L chain L chain actually is a very lengthy module uh monitoring and guard rails I already mentioned that multi-agent application Advanced rag uh and evaluation I've talked about all of this let me and not talked about the project as well so this is on a this is uh these are the people who are on on our pool of instructors um so they have been a presented at different time I anticipate that for the next boot camp uh the next upcoming boot camp I will be of course there but uh Lis I expect him to be there Adam John Sebastian likely Jack Sophie and uh Sage should be there and then we may get more instructors next time around uh these SE are our testimonials from our partners uh they love it uh the fact that I mean we have done we have put together a very comprehensive boot camp in like one of I I would I would say hands down the most comprehensive uh uh training on this topic um with the intent in mind that you come in and you leave with an ability to actually uh build products and one thing that I always call out here is that um uh that if you come in as a technical as a product manager or a technical product manager you will leave as a technical product manager who understands the the finest details uh of uh the llm uh products when you're building a if you were to lead um or become a technical product manager for uh an llm application uh if you are a great Dev uh if you are a Dev and you leave you come in and you attend you will be be able to actually be a de for an llm application but what we do not actually uh uh expect is that a product manager program manager will come in and suddenly they will become a developer right so that's not I mean you will be so you will you whatever role you are in map the llm application or generative AI domain uh and um you know the people who attended they have switched jobs uh those who were product managers now they are you know llm product managers so that transition is very likely one um and then uh those who were um uh devs and they went back and they started to become uh llm application developers but you know this uh you know this cross uh I would say this this transition from one role to another I don't expect that but um anyone who is a PM anyone who is a product manager uh program manager and um and project manager they should be able to actually follow the content even the labs the way we have structured this is you do not need to be a coder um we have had actually already then the company the logos that you see we already have people from these companies who have attended our training our llm boot camp training our uh data science boot camp portfolio is much much bigger like more than 3,000 companies this is a fairly new product uh that we are starting uh so there are um many companies uh and many uh many companies many uh people who have attended from different companies and uh the list is of course growing uh the next next boot camp is happening from February 3rd to 7th in Seattle uh if you're not based in Seattle and if you cannot travel to Seattle the same boot camp it is going to be you can attend it remotely and we have uh a mechanism you know we have Tools in place that your experience is even attending remotely it is going to be as good as people who are attending uh in person uh but you can attend you can join the same boot camp um even remotely as well and let me actually stop sharing my screen here and bring everyone here and start asking answering questions let me okay there are [Music] are okay um so there is a question let me see the first question here I will go in the order in which they were they arrived uh I have an interest in building lava systems so um I'm not sure what you uh are you refer to here Oris oriso oriso if you can elaborate I can come back and ask uh uh I can answer this question um will you be sharing PBT for yeah I mean we will be sharing it I think someone should have uh already shared a link where you can request uh we don't uh we cannot send uh the deck to everyone of course we don't want to spam you uh but if you want there should be I mean just reach out to us and we can share the deck uh and then there is uh let me see there is other questions uh prerequisites as I mentioned earlier uh there are no prerequisites uh um you know it is as long as you can uh write basic uh code uh you have just in any language I mean uh the it is it may appear to be coding intensive it may appear to be something that is uh very much uh coding heavy but the nice thing is that everything is organized so you can actually go through line by line and still be able to so if even if you don't intend to write code uh you should still be able to um you should still be able to follow um let me see which large language models uh we will be primarily uh covering um we um um please uh you know so the the models that we cover will be covering some open source some close Source we uh we do llama um llama series of models for fine-tuning uh for the most part we uh for embeddings we use open AI embeddings uh occasionally uh uh we use other embeddings but in most cases we are using the um open the ey embeddings so you don't have to worry about you know the the keys Etc we can actually we will give you the keys um uh then uh um so that's the uh you know the technology stack so the in terms of embeddings vector database uh we have ex we mainly use vv8 for Vector databases but in our learning platform we have more exercises for uh redis which they have a vector database and Microsoft Azure search that also is there uh so think of it this way uh the ecosystem is still growing um uh you know do we do Lang chain or llama index well we do Lang chain primarily uh but uh you know once you understand how to do things in Lang chain then you can go and build things in any other uh any other um available open source tool so our focus more is on imparting the skill not specifically um not specifically um a specific vendor or a specific open source project so we are we are focused more on developing the skills uh so there's a question I was a Java developer with 20 plus years of experience now I'm leading to projects I want to learn llm so Shanti not sure um where can I start learning llm uh yes so I I think Shanti this is not only that this will teach you this will also the boot camp we also talk about what are the opportunities uh What uh we talk in a lot of detail about uh you know where uh the technology in general what kind of applications are going to be there uh so many many people leave with these uh these ideas and what they are going to build so if you're uh if you are 20 plus years of experience if you're leading projects I would I would say that this is the place of course you know um depending upon uh um it will depend upon you know your immediate near-term long-term goal because it's a it's a substantial investment of time and money but uh you know we are very confident that that if you want to undertake an llm and J pro pro uh project uh within your company um this training will not only enable you if you have a clear idea how what to build this will also give you some ideas what you can build okay let me see so I see when will this when then when boot camp will organize in nust I am not sure um uh so miam um if you have a question please reach out and then we are happy to talk to you about if you're talking about a boot camp in a specific location or a specific place um let me see okay so I think uh are there any other questions so there is another questions okay uh so I have an interest in building a multimodal llm system for a project that takes both text and image as input uh yeah uh so uh we do not actually yet until the last boot camp we did not do any multimodal but uh for the next boot camp we are actually considering uh I was actually talking to Sebastian who teaches this module uh for from Sebastian comes from bv8 um it is one of the the leading Vector databases in Industry so I was talking to Sebastian and then we may have something around multimodal but I cannot actually promise because getting something in uh into in the content uh it it takes quite a bit of effort but I'm very very hopeful that we will have something but think about this that that's a very small um that's a very small portion of the bigger picture uh maybe it will be even if it is there it is going to be half an hour or one hour of addition to the entire boot camp we are talking about the entire system so whether you build a multimodal system or is it only an llm um without any image support uh your application is not a multimodal still you will need all these moving Parts you still need to understand embeding you still need to understand Vector databases in detail you still need to understand Lang chain you still need to have guard rails you still need to know fine tuning so basically the boot camp is a about enabling and empowering you so even if there is uh this topic that we have not covered um you will still be able to go and build multimodal systems so I'm I'm quite confident about it but we have not covered uh multimodal systems yet in the boot camp but uh I'm I'm hoping that uh in the next offering between now until February um we will be able to I think that also there is this limitation that uh uh um the landscape is changing so quickly that we really have to keep up like every six weeks when we offer a new boot camp uh the next cohort that comes in uh the prev what we teach in previous cohort that always we improve because something has changed industry the good part is whenever we ince a new module we invite our alumni who have attended the boot camp because the landscape is changing so quickly so U we have been introducing something new in uh in the boot camp for the last eight or nine boot camps every single time the curriculum has improved so much and anytime there's a new boot camp that comes in a new module that comes into the boot camp we um invite U the previous attendees uh to attend the session okay I think I do not see any other questions I will give it maybe another uh 30 seconds or so if there are any questions I'm happy to answer if there are no questions then we are going to end the session okay um well thank you so much everyone for attending uh please feel free to reach out to us if you have any questions um we will be happy to get him a call and uh help you out for