I built a *streaming* AI chat app

Key Takeaways

a few days ago Purcell jumped on the AI hype train and released their brand new AI SDK an open source tool that makes it really easy to build apps with large language models like gpt4 and others and it handles text streaming on the server allowing you to recreate the typing effect that you see in apps like chat GPT in today's video we'll build an AI chat bot from scratch and spell kit to get an idea of what this tool is all about because it is incredibly useful if you plan on integrating llms into your app this video is not sponsored but I do want to take a second to show this demo that versal made that allows you to compare multiple llms side by side just by itself it's a very useful tool for comparing responses between different models under the hood it's using xjs along with the aisdk that we're about to look at to get started let's go ahead and create a new spell kit project and by the way if you're new to spell kit I've been working on a full course for the last couple months that will be out by the end of June I'll leave a discount below if you want to get access with fireship Pro now inside the project let's first install the AI SDK then we'll need to decide which AI platform we want to integrate like anthropic cohere hugging vase or open AI I'm going to go with openai here and install the openai edge Library this is an alternative to the main openai package which is based on axios whereas The Edge package is based on the browser native fetch API that's mostly important if you plan on deploying this to Edge functions like those on Purcell but that's completely optional and it's important to point out that you don't need to use resell to take advantage of this Library the next step is to go to the openai dashboard and get an API key let's go ahead and copy this API key into a DOT EnV file in the root of the spell kit project now in order to interact with this API we need a server endpoint install Kit we can easily create an API endpoint by right clicking on the routes directory then use the extension to create a new server.ts file that'll give us an HTTP endpoint in the chat directory that's initially configured to handle get requests in our case though we want to change that to a post request which is accomplished by renaming the function the code in this function will only run on the server which means we can safely import openai and the environment variable with the API key a cool thing about spell kit is that instead of using process.env we import environment variables from the dollar sign EnV namespace which provides intellisense to prevent you from using a variable that doesn't exist now we can pass that API key to the configuration object to initialize the openai SDK now when sending a post request we'll include a body of messages to give the AI model the context of the chat from there we can use the create chat completion method to specify an openai model like GPT 3.5 turbo and then we'll make sure to set the stream option to True when streaming the API will send back small chunks of the response instead of waiting for the entire response to process and this is the point where versel's AI SDK becomes useful it has a helper function to First convert the response to a stream and then a class called streaming text response to then stream it from your own server endpoint which might also be running on the edge that's pretty cool and it also makes it possible to run callbacks as the stream is being processed like for example on start you might want to save the proof to the database or on completion you might want to save the rest response from the AI to a database and that's all it takes to build a streaming API on the back end now that that's done let's head over to the page.svelt file to code up the front end on the front end we'll go ahead and import the use chat helper from AI spell use chat as a function that can configure settings and returns a bunch of helper functions and stores to help you manage that state of the chat by default it's looking for a post endpoint with a URL of API chat on the same origin however my endpoint is on the chat URL so I'll go ahead and update that in the settings one thing it returns is a messages store that we can subscribe to by putting a dollar sign in front of it its value is an array so we'll Loop over it with each which will display and update every message in the chat in real time next we'll create an HTML form so the user can submit a new message to the chat the library provides a function to automatically handle the submit event and also an input store to bind the proper form input to the chat prompt congratulations you just built a streaming artificial intelligence app that runs on the edge now run npm run Dev to run it locally when you submit the form it will automatically make a post request to your API endpoint to communicate securely with the AI on the back end then stream the response and update the UI token by token on the front end thanks for watching and I will see you in the next one