OpenAI announces FINETUNING 👀 for ChatGPT

Key Takeaways

so open AI just announced that users will be able to fine-tune their GPT 3.5 model and looks like the ability to fine-tune gpt4 will be coming in a few months according to open AI early test I've shown that a fine-tuned version of GPT 3.5 turbo can match or even outperform base gpt4 level capabilities on certain narrow tasks so what is fine tuning you remember that movie Rain Man one thing I was really good at math he was really fast at calculating stuff had this like Supernatural ability to remember numbers but he kind of struggled with other tasks I kind of think of fine-tuning as a little bit like that basically fine-tuning means specializing a model to some specific task if you want a customer service chat bot that answers questions about your particular product you can fine tune a model to do that if you want a computer game character a non-player character that's that feels like he's really part of that world you can fine tune a model to do that now fine-tuning can have some downsides basically you should assume that there's going to be some in degradation of some of the abilities outside of what you're fine-tuning for outside of what it's specialized to do for example if you fine-tune a model to answer as if it was an orc chieftain in your Dungeons and Dragons game it may lose its ability to argue the finer points of post-colonial feminism so what are the advantages of fine-tuning a model well there are quite a few one is it can enable a very custom very controlled experience if you need the llm to say specific things to not break character to not say things like as a large language model well then fine-tuning allows you to custom shape what it says and does as openai puts it improved durability reliable output formatting and custom tone this can be extra important places where for example you're doing some sort of code completion or composing API calls you want Chad GPT GPT 3.5 whatever to Output the code in a very specific way you don't want it starting with like oh sure I can help with that and then doing the code you wanted just a code or whatever format you're looking for or if it can't do it you want to throw up some specific air message then you can kind of flag but you don't want it making up something brand new to answer your prompt the other big Advantage is the cost so you can slash the cost of using the llm for your particular tasks now it's important to note that the actual cost per 1000 tokens is going to be bigger than on the base models it's six to eight times higher than the base model but since you no longer have to give it you know let's say multiple examples you don't have to teach it to Output the proper format every single time there's going to be some use cases where using the fine-tune models is going to save a lot of money so where's this going to be used well customer service is going to be huge email chat Bots Etc this can be used very effectively and you have to worry about it making up some offensive crap on the spot and making your customers mad things like language translation you can force it to respond in a specific language so whatever the prompt is it responds in that language and it translates that and translates that prompt into whatever language you want it to so for Education this will be massive for things like tutoring for learning for code searching for therapy stuff like that or gaming like remember our evil orc that's hopelessly stuck in his ways the entire backstory of their own could be given to all the characters and then each would be ready to respond in character individually but they would all have sort of like the same backstory if you want them all to know something like this land is getting invaded they would all know about it so they can respond appropriately to any questions that you might ask fine-tuning can be used in legal research for example remember that Lord I used Chad between chord and it cited a bunch of fake cases he got into a lot of trouble for that fine tuning would be able to help with that one thing where I think personally this is going to be used is where you know if you have certain AI agents where you have multiple instances of Chad gbt that are kind of working together to achieve a mission for example you have one that's making decisions one that's writing little scripts maybe you have one as the base model and one as a fine-tuned model that's specialized in doing that specific thing that he needs to now now that I think about it almost certain that we're going to see a model that is fine-tuned to produce prompts for another model for another base model basically something that you teach to Just Produce like the perfect prompts every single time based on whatever input you have that you needed to take into consideration so two couple other places where fine tuning can help according to open AI high quality results then regular prompting ability to train on more examples that can fit into a prompt so this is interesting because so basically oftentimes when we're using these llm models there's something that is referred to as for example few shot learning or zero shot learning shot meets examples basically so few shot means you give it some examples as what you want the output to look like and sometimes a few will do but sometimes you get better responses when you provide a hundred two hundred a thousand different examples for it to go off of so that's one way to kind of get around it basically you can fine tune a model and use you know a thousand examples to really fine tune exactly what you want the output to be so token saving is due to Shorter prompts again you're training it it's almost like custom instructions but for like the whole model and so so there's you're using less input tokens you're using less output token and lower latency requests so basically you're getting the answer back faster so here they're saying how fine-tuning can be good for specific applications but that's not always worth the time and effort they're saying first get good with prompt engineering prompt chaining breaking complex tasks into multiple prompt multiple prompts excuse me and function calling by the way function calling is not going to be available for training these or fine-tuning these models so their main tasks for which our models May initially appear to not perform well but with better prompting can achieve better results iterating over prompts and other tactics has a much faster feedback loop than iterating with fine tuning and so the use cases where they think it's going to be very good so specifically when you when you have to set the style tone format Etc like kind of we talked about improving reliability so for customer service where you really don't want to say the wrong thing correcting failures to follow complex prompts handling many edge cases in very specific ways and performing a new scalar task that's hard to articulate in a prompt this might be something like trying to copy a certain Style of writing like if somebody has a very specific style you know the more sort of examples you give it the better the outputs is likely going to be but that might be difficult with sort of the base model so something like this you can give it I assume you can probably give it a whole book's worth of writing styles and then train on that one high level way to think about these cases is when it's easier to show not tell and another scenario is where you need to reduce costs in our latency without sacrificing quality and then they have a little bit of a fine-tuning steps kind of like a brief tutorial so basically where we prepare the data we're going to be feeding it upload the files using your API key we're going to create the training file and and then we're going to be using that sort of new model that we created with our with our ID and so I think a lot of this is going to depend on the cost structure of this so you're paying let's say seven times more per thousand tokens but you're saving money on how much tokens are going back and forth potentially so it sounds like there's I mean obviously there's gonna be some use cases where this is going to be great and for a lot of the other stuff might not be so I feel like I don't see a current knee need for me to use anything like this right now but maybe maybe in the future there will be some specific case where this would work obviously if this was much cheaper than running the base model this would be a big deal like if you can train the model to do the thing that you wanted to do and have it cost a lot less than you know let's say gbt4 that would be excellent but that's not in the cards right now unfortunately but I'm sure as we see more and more people testing this thing out and you know publishing their use cases we're going to see specific use cases where we're gonna be like of course this is great for that why didn't we think of that curiously know what you think if you have something that you're going to use it for where you think it's going to be like the perfect use for fine tuning instead of the base model please let me know and again that could be 4gbt 5 3.5 turbo or later for gpt4 and they also have two sort of of the much cheaper models DaVinci Dash 002 babbage002 whereas you can see here the cost per thousand tokens is much much less than sort of like all the other ones so maybe if you have some specific use cases for that I'd love to hear it let me know in the comments and my name is Wes Roth and thank you for watching