Auto-Prompt Builder (with Hosted LangServe)

Key Takeaways

all right let's get started this is the opening eye prompt engineering page before we can start building with it though we need to set things some things up so I've created a f a fresh uh virtual environment what I'm going to do is I'm going to set up a l serve uh uh template a l or application with this the reason that I'm going to be using lay serve is it'll make it easy to deploy this once I finish creating so the first thing that I'm going to do is bootstrap a link serve project I'm going to do that with the Lang chain CLI I'm installing it here after it finishes installing I'm going to create a new app I will call it open AI prompter I am not going to add a package because I'm going to be creating my own I can then go inside it and if I open it up the main thing that I'm interested in is this Lang serve server right here which wraps around fast API and will make it really easy to deploy my Lang chain chain the last thing I'm going to do for setup is set up laying Smith so laying Smith is in uh private beta if you don't have access to it and you're seeing this and you want to follow along shoot me a DM on LinkedIn or Twitter and I will uh get you access to it um what it will make it really easy to do is debug as we go along and this will involve a lot of prompter engineering and so we'll see how that becomes very very helpful um someone in a previous video commented that I should go over how to set this up more so you can go to your projects let's even create a new project um let's call this uh open AI prompter um that's all we can do let's then go to setup and then we can just export some API Keys here um so you can just copy this you can paste it here you then need to add in your API key you can do that let me let me move my face over here and let's move this over here as well you can do that by going to API keys and all right good it's not showing mine so you can create an API key and then put it there um I've already did this ahead of time um so mine's all set up but if you needed to do that this is how you do that all right so let's go back to this prompt engineering guide the first thing we're going to do we're just going to copy it all because we are going to be using this in our prompt to help us write better prompts let's paste it into a file and then I'm going to I'm going to create my chain in a Jupiter notebook and then I'm going to put it into uh and then I'm going to put it into this app and so the reason that I'm doing this there's uh there's there's one real reason and one fake reason the one real reason is that a lot this will it' be a pretty iterative process I'm going to do some prompt engineering some of that I'll do in Lang Smith but some of that I'll I'll do in uh The Notebook as well and we'll see why um and so having a notebook like environment is really really helpful because it's an iterative environment and I can really easy iterate the other reason I want to do this is I want to show how it's really easy to create a chain and then just export it from a notebook most of the time it's it's pretty simple so let's save this and now I've got this jup notebook let's load um what I just put in there all right so let's do that let's import uh we can print out the head I guess just to see it let's import some stuff from L chain that we're going to want so from L chain core. prompts import template from L core do output parsers string output parser from L chain Community do chat models import chat open AI so I'm importing three important things this is going pretty simple chain that I'm writing we'll see maybe we'll get more complex but it should just be pretty simple because I'm just going to use a model that has a long context window and can work with all these instructions so I'm importing a prompt template this will help me uh structure the inputs to the model I'm importing an output parser that'll basically just convert it from a message format which is how the newer models respond the chat messes respond with uh message but I I really want to set it's string so I'm getting that and then I'm importing chat opening app which is the L chain wrapper around the open AI models which are the ones that I'll be using now is the fun part where I'm going to write my template for creating or my template that's going to help me create a chain that can take in an objective and write a good prompt so let's add this there um and okay so so this is actually an interesting point that I'll get to later but basically what I'm doing right here is I'm going to have these instructions as a variable that I'll pass into the prompt and I'll I'll go into that more later but for now let's just assume that's what we're going to do and then let's add just some delimiters here and then let's say based on the above instructions help me write a good prompt I want a prompt that okay so what I actually want to write is not actually a prompt but I want to write a prompt template because I think most people write prompt templates because they want to write prompt templates that then they can use in their code or at least most linkchain users want to write prompt templates help me write a good prompt template this template should be a python s string it can take in any number of variables depending on my objective return your answer in the following format this is my objective this seems like a decent first pass I don't know we'll see um and then we're going to create the chain okay so as I mentioned we have this thing here um which I'm going to pass in the instructions to let's see what would happen if we didn't do that and if we just did like this which you could also easily do so if I did this I would have prompt equals prompt template from template template chain um equals prompt Shadow AI string up the parser chain. invoke and now I'm going to pass in objective what should my objective be um answer B answer a question based on a based on context provided and only on that context so this is a pretty typical rag prompt or retrieval augmented generation prompt you want the the language model to respond to a user question based on the context provided um so let's do this and I get ke because basically what's happening is in these instructions there's other um there's other curly brackets which aren't variables that I want to format they're just part of the text and so this often happens when you're working with code or something like that and we've got a a lot of questions about how to deal with it so my preferred way to deal with it is to do what I'd done earlier treat the the text that has these unwanted curly brackets as input variables and then okay so so one way to do this is you may be thinking okay so that means that I now have to provide text as text here and this is a little bit annoying because now you have two input variables it's not the end of the world but it's a little bit annoying and if if this chain is part of a larger chain then then it starts to get more and more complicated more and more annoying um and ah I had that should be taex this is erroring because the model's not long enough I could do some fancy rag um it's just like barely over the context window and I'm I'm going to want to use this model anyways because this task is kind of hard so I want a good model to do it um so this will just get me around that and then let me also set temperature equals to zero while I'm at it so I could do it this way I could pass in the text here um and this would work let's see what answer it gives um for the next time I'll use streaming um so we can start seeing what's going on earlier um all right it's writing long thing while while it's working okay so let's um this is a little unreadable um because it's not super readable what I'm going to do is all right so this is Lang Smith this is me debugging I want to see the output um this is a all right so this is better output um I get uh oh interesting okay so all right so it's giving me some instructions on how to use it um I actually don't know if this is correct because of these curly brackets but anyways um what we can do is uh uh sorry back to this what we can do there's a lot I want to show off but first things we can do is partial The Prompt so we can now do this and that means we no longer have to pass this in so we're basically just partialing the prompt we're passing in some input variables ahead of time before we even construct the train and then because we know those are always going to be fixed and they're and they shouldn't be variables so we're going to pass them in and then we can do this I'm going to now change this to streaming for Chunk in or more accurately for token in chain. stream print token um all right I need to add something like this all right so there we go it's not exactly um what I really want I really want like just one I really just want this thing um templ should be a python see if this helps so it's not helping a ton I can go back in here and I can I can start doing some prompt engineering here if I want so one thing that I can do it's not really that helpful here because this is a really simple um chain but if I wanted to I could open up the playground and it's also not helpful here because my prompt is really really long and this isn't the grass oh I can okay that's slightly better so now I have this um and um let's let's just do like bunch of new that thing based on the above instructions help me write a good prompt template this prompt template should be a string that can be formatted as if a python fstring I can take in any number of variables depending on my objective um I can now run I can now run this okay okay that's pretty good all right so that's pretty good I'm I'm I'm relatively happy with this so um let's uh take this from the playground let's bring it back in the notebook 24 being a bit slow okay so it's generally doing pretty good I'm relatively happy with this I might do a little bit more prompt engineering by the time this gets out but I'm relatively happy with this for now so what I'm going to do is I'm going to move it into Lang serve now I'm going to deploy it with Lang serve so I am going to create a new file in here I am going to basically copy this chain over that's done I'm going to go into the server um so I'm going to edit this add the chain you want to add um from app. chain import chain chain path prompter cool um I'm going to go back here and let's try it out and see what happens L chain serve doesn't find the file that is right so L chain notebooks open AI prom what did I name it prompting oh right let specify that let's try serving it again and I'm going to poetry add open AI because I'm going to deploy this project later um and this will add it to the Pi Project file so it's going to save that um let's try it again cool so now if I go to Local Host 8,000 prompter playground I get this let's try it out I take this add it here all right so I get back some streaming um I get back some nice stuff okay so there's definitely some prompt engineering that I want to do um it's doing this um Let me let me pause the video a bit do some prompt engineering and then resume it in a second all right so I'm back with a little bit of a better prompt um and I will uh uh I I'll include links to all the code so I won't read out here basically now if I go here and I type in an objective this gets me um this gets me a pretty solid prompt that I'd use for for retrievable augmented generation so so that's basically how we get to something that's deployed with Ling serf um this is still all local and so for the final part of this I'll walk through how to do this um on laying Smith so that you can share it with other people as you see so I have my my personal tenant my personal tenant doesn't have access to uh laying serve deployments yet on Lang Smith so this is uh this is an alpha feature that we're still testing um it's it's it's only available to a few people um if you are interested in being an alpha tester please let us know um or please let me know and uh we are not letting a ton of people off because we want we do want to make some improvements um but as we let more and more people off um you know we'll we'll will uh let me know and and you can be one of them in order to show you briefly what it looks like I am uh going to switch to the Lang chain account I'm going to go to deployments um and you'll see once it loads that I have a bunch of deployments here these deployments are all connected to GitHub um so one thing that I'm going to do after this is I'm going to pause set up a GitHub um uh repo with this code so that I can easily deploy it um so let me pause and do that and then I'll come back and walk through a new deployment and see what happens all right so we're back so I'm going to create a new deployment I'm going to click here I'm going to choose this repo that I created open AI Auto prompter name it uh open AI prompting helper um I'm going to add some environment variables um namely I'm going to add open AI API key um I'm going to make it a secret G to pause while I put this in all right it's in um I automatically get a tracing project for this so that's a big benefit of deploying on Lang Smith is it automatically connects to everything else in Lang Smith um right now that's tracing we'll make that testing um and and prompts and other things in the short term um I can now submit this and it will spin up a deployment um and so here is this uh uh deployment as you can see it's uh it'll take a little bit so I'm going to pause and come back when it's finished deploying all right we're back our deployment succeeded we can now open the playground for this deployment so we can go here task um what what was the task we had way back in the day so let's just use the same task here we can see it streaming awesome so we get this result um one thing we can do is we can give thumbs up or thumbs down I like this one so I'm going to click thumbs up if I go back here I can see it's only one Trace count but I can see the traces for this so here this is the most recent Trace I can click into it this is a tracing from this is the same tracing that's in uh uh langing Smith as you've been using it so that's one of the big benefits as it comes with all this stuff it comes with feedback automatically hooked up so we have a score here um and yeah that's uh basically it for the video now hopefully this was pretty helpful in terms of getting um a pretty simple chain it's you know it's just a a prompt a model and an Alpa parer up and running using lell uh getting around some uh prompt uh annoyances with all the formatting of the curly brackets shwing how to set up langing Smith showing how to set up a laying serve project and then showing uh the new laying serve deployment feature as well hopefully you guys enjoyed this let me know in the comments have a good one