Build a Full-Stack GenAI Project in 4 Hours (FastAPI, React, Supabase)

Key Takeaways

Okay, so this is going to be a big one and something I've never done on this channel before. We're going to build a full stack Genai application. This is the final result where users can ask questions and the answers will be grounded in hundreds of documents that we will all set up within a database. But the beauty of this is that we're going to cover the entire architecture, something you rarely see on YouTube. We'll cover the front end, the back end, the database connection, and the ingestion and rack pipeline, and how everything ties together. So, if you want to get into AI engineering or simply want to get better at this, this is really the best video for you. I'm going to share my entire process of what we do inside our AI development company. So, I'm going to show you how we set up the repository, the agent.mmd files, which packages and tools we use, and how we deploy everything and tie everything together. And I'm probably going to condense about 40 hours of learning or so into this 4hour video. But now that does come with a disclaimer. This is not going to be a beginnerfriendly step-by-step click here then click here. Now I'm going to move at my natural pace and I'm going to use AI agent. So if you want to follow along, we have to get the repository. The cool thing is I have this over here on the main branch. This is the starting point. So this is the almost empty repository that we're going to start from. But I also have the final result which is the development branch. Now if you want to move along, I recommend to clone the main branch and then start coding with me and follow along with me and kind of like augment and emulate the prompts that I use which I will all show to you so you can follow along because if you go straight to the vinyl result, you can also do that. You have a cool project but you not really will learn anything. Now next to that if at any point you feel like this is going way too fast, I have no idea what's going on. Just know that on my YouTube channel for about every topic that we dive into, whether that's hybrid search for rack, whether that's setting up agentic search, whether that's setting up like CI/CD pipelines and tying things together, I have videos for all of this. So if at any point we are in a certain section and it's completely new, completely foreign to you, you can't follow it, please go to my channel, find a video on that, and that's how you can kind of like go back and really accelerate your learning. or of course just ask AI right all the code is here if you go to the development branch everything that we work towards is here as I go through the process some of the things will be messy right this again this is not I would say a course this is engineering in the real world so sometimes I make mistakes I get stuck I have to debug things I'll use AI to fix certain things but you have the final results there the results that will run and you can always like use AI to kind of like fact check get back on track again ask what the hell am I to hopefully give you a great learning experience and to give you a great portfolio project. So, welcome and let's kick off this project. I got a fresh cup of coffee over here and I'm going to take you through the exact process that I follow whenever we take on client projects. So, I've prepared the repository for you. Link will be in the description of course for you to follow along. We are currently on the main branch which is also what I have open here inside cursor. This is what we uh will be working from. Now what I'm going to do as I'm going to progress with the project, I'm going to work on a development branch. So this main branch stays exactly the same. So you can take that as a starting point. But before we actually get our hands dirty, I want to take you through high level what we are going to do because when you are using AI and AI agents, the better the initial prep work, the faster you can go really with these AI agents. So a lot of the the setup and the things that I normally do when I set up a project have have already been implemented here. So beyond just following this tutorial and learning from this video, this repo alone is probably going to be a game changer for you if you've never done something like this. So it includes the agents.md files both in the top level as well as in the front end folder and in the backend folder over here because what you'll find is we're going to set this project up as a mono repo. Meaning that we have both our back end and our front end in a single GitHub repository. This works super well when you are using coding agents because they can reason over the entire context. So before diving into all of the nitty-gritty things, let's start with the actual client brief. What are we actually going to build? Why is this worth your time? So let's quickly cover this. So the whole idea of this video is to essentially create a hypothetical case study that is as close to what we see in the real world for you to kind of like simulate the ultimate learning experience. So we came up with this or we AI of course came up with this name Driftwood Capital which is an independent investment research firm. So I want to take you through this brief. It's not that long to understand what we're going to build because any project really should of course start with a brief whether that's something you came up with, someone within your company came up with or an external client came up with. So the whole idea here is that this is a company for the analysts and what they do they sell deep equity research to uh institutional clients. Think of hedge funds, mutual funds, pension funds. So they sell this information And they do this do this based on subscriptions, right? So they can earn good money with this. So they generate these reports where each analyst has to cover certain like public companies and all these companies have to make their uh annual reports and even more reports they have to make that available. So this is all public information that we can download. So we can use this to our advantage. So you can actually work with a relevant data set within this project. So a little bit about how they how they make money, how an investment firms make money or a research firm I should say and how they add value. So their client their clients which are portfolio manager at the actual funds they don't have the bandwidth to read every 10K uh 10Q earning transcript and industry filings for the companies they invest in. So we are going to use the following data source. So this is from the US security and exchange uh commission and here we can via an API download all of these these filings. Now I've already prepared the repository for you with a download file. So let's let's start there first. So I want to be I'm going to be it's going to be interactive, right? So instead of just reading this entire uh document first, let's already actually have a look at the data so we can start to get the feel for what this is what is this going to look like and what we're going to put in our system to actually ask questions about and what to what to reason about. So I addit in the downloads uh in the data folder I added this download.py file. And if you follow the instructions here in the uh read me in order to follow along you're going to need a couple of dependencies. So we're going to use Python and I recommend to run Python using UV and later also we need notes and we're going to need pmppm. So the first thing in order to get the to download the files pretty much we can use this file over here. So if I open up a new terminal let's see we're already we're already here in the download folder. So if you were to open this up in like a clean uh in a clean project, you would probably open up your terminal in the root of the project and then you would just cd into the data folder. And now what I can do is I can do uv run and then I would do a download.py. So this is going to download all of the documents. And all of the documents in this case mean we are interested in the following tickers. So we do Apple, Microsoft, Nvidia, Amazon and Google. So here you can see it's now going through it and it's set up to nicely split this up into the years. So we download the last five years of data for all of these companies and there's also a manifest.json which pretty much is an index that reference look for this particular ticker here are the dates here are the the filing date and this is the local path URL. So now we have all of this data in here. So let's actually have a look at what this is because these are as you can see HTM files. So this is not HTML. This was also actually a new thing for me HTM files. I'm not but like it's it's a kind of HTML but not really. So let's have a look at what this looks like. So I can say open this in the browser. I can do this here inside cursor and we can see this. So which one do we got? We got the one from Apple from 2021. Right. So, as you can see, if I scroll down over here, this is a huge document. It's very long. It's got a lot of information. We got numbers, we got tables, we got earnings. So, this is going to be a very interesting problem, of course, to start using AI to extract information from this. And the key thing here is going to be how do we ensure that our AI agents are not going to start and hallucinate, right? Because this is a lot of information. We got a lot of years. So if you are not careful, the agent may come up and hallucinate certain facts, right? So we are going to uh we are going to counter that and that's one of the most important things of this project. So if we come if we then come back to the problem right from the client brief. So every analyst spends roughly h half of every week. half of their time doing source document intake, meaning they go through all of these filings, they scan for the certain sections they care about. So they go through these documents that I just showed you, they copy and paste. So they they create their manual reports, they take what they need, and they need to compare this year over year and only after that intake work, they can produce an original analysis. So the intake, so the start is very boring. It's very repetitive. So every time there's a new company or new documents say is okay what what are we dealing with right now going through this again reading checking everything. So that's the process and it's very repetitive across the analyst right because they all have to do this and they may all have their own kind of like way of doing things in their own like local files. So it's the the single biggest drag on an output from an analyst because if we unlock this, if we can speed this up, they have more time to actually add value to clients and that is by actually synthesizing information from those reports. So what they want, so this is ultimately the preferable situation after we implement a solution for them. They want some type of internal chatbot. You can call it a document co-pilot. That's what I'm going to refer to uh to this project name within the example over here where the analysts can ask questions in plain English about uh filling in driftwood's created corpus. So f about any filing I should say sorry. So that's what they want and then it's very important that they get a sourced answer that sites the specific file in a specific page. Here we have to understand that if if the AI gives an answer, if it makes something up, which it's very easy to to for an AI to do that, we can get a faulty input, faulty analysis, and of course, that is can be a very costly mistake for an institution like Driftwood Capital. So, we always want to make sure that we reference sources so we can backtrack and fact check to okay, where does this actually come from? So that we can also instruct the client when we implement this first proof of concept for them to say look here's how the tool works but your analysts are still responsible of course for the output and for factchecking. So we should use this tool as a way to speed up and accelerate that work to do a lot of the the heavy lifting and the plumbing but not solely rely on it. Maybe we can grow towards something like that. But in the beginning, we really we need to be sure that we can fact check this. So we really like what they want they the answer they it needs to be trustworthy enough to base downstream analysis on it. So we need to engineer the whole application around that and it should be uh they should be able to use it from the browser when they are logged in with their driftwood email address. So this is something else, right? So, we're going to make this available as an as a application that runs in the browser. And I'm going to show you how to deploy this and h how to really set this up with an actual login. That is going to be really the extra mile on top of what you normally see in other tutorials. That's why this will be like an end to end project. And they also want to see their own past conversations. So, really think of it as their own internal chatbt, right? And you may be wondering like okay but why wouldn't they just use something like chatbt or co-pilot or any other of the tools that are available out there and what you'll find is when you are working with clients especially once they go bigger in size there are two major things. So first of all a a big big issue is privacy compliance and security. So for most companies of any reasonable size, it is simply just a nogo to let everyone pretty much use CHBT and just pluck the data in there. That is just off the charts. You cannot do that. So you need to be way more strict about how you handle your data, where it can actually be stored, what it can sh what it can be shared with. And then usually you would set something up because you still need to use a language model, right? So that's the that's the only thing that needs to be whitelisted. But you would either do that through AWS, through Batrock or through Microsoft Azure. And you actually get an enterprise ready, if you will, endpoint with very specific data policies. That's usually the only way that you can get an application like this approved in enterprise environments. So that's one thing. Now number two is the customization layer. Almost any off-the-shelf tool that is currently available on the market whether that's copilot cla chatgpt they're generic tools anyone can can use it and now yes you can give some prompts you can use some skills but if you want to use an application like this really as part of your organization as part of your strategy where you say look we got 40 analysts and they all use this tool you need to standardize it and you need to add custom features on top of the generic tools because otherwise you'll always get generic output. If I take all of these documents that you see over here and I chuck them into Chat GBT and I'm going to start asking questions about it, like within a matter of minutes, if you let five analysts use that, they will all get different answers. it will start to hallucinate and it will just it will just get messy because everyone uses the tool a little bit different. And also these tools are not really these off-the-shelf tools are also not great at handling really large volumes of of data because you pretty much put everything in a context window and that is quickly going to burn out. And I know Microsoft Copilot can link to SharePoint and all of that but that's it's it's bad. it's really really bad. So those are the two arguments as to why this is still a necessary step and a necessary application for a lot of companies and why I wanted to showcase this because if you understand how to set this up, you will learn how to work with a a backend, you will learn how to work with the front end, how to connect them and engineer everything together following rigid AI engineering principles as well as rack and information retrieval techniques on on top of that. So that's why this is worthwhile to go through. Um, and like I've said, if you follow this, it will be a great portfolio project and I'm I'm I'm sure you'll understand like the endto- end fullstack engineering around everything better than than than most people out there. So that's what they want and also why they want it and why it's relevant. Now then some example analyst questions. I won't go through all of these, but think of it like this, right? So let's consider Apple. So across 2021 to 2025, how did the revenue mix between the iPhone, the services, the Mac and the iPhone, iPad and the wearables change? And which category appears to have contributed most to any mix shift? Now that is a pretty deep question. Like if you have to do this without AI across all of these documents, you're going to have to do some digging, right? So that gives you a sense of what we're going to work towards. And I have no that's the funny thing about this project. I have no idea how this is going to turn out because I haven't built this. I've set up the project but I haven't built this before. Okay. So then what the trust means right? So pretty much we always want to have citations uh no hallucinations of course show the underlying sequence where or passage where things are coming from and then out of scope for now. So this is not going to be trading recommendations. We're going to not going to use any external data beyond what we put in the system. no multi-tenant or multiclient setup, no billing uh plans or pay walls, no mobile app. So the definition of done of this project which we're going to work towards is where a pilot a group an a pilot group of analysts can actually use this application in the browser and start working with it and it should at least save them three hours per week. That's that's the client brief. That's the starting point. So, with that introduction out of the way, let's come back to the project one more time and make you a little bit more familiar with what we got going on over here. So, first of all, if you haven't already and you want to follow along, make sure that you have all of the prerequisites set up, right? So, by now I've just shown you how you can download the data. We got that in there. Make sure you have Python. You should already have that now. Um, note and pmp. Other than that, we are going to use a couple of different services also to to actually deploy this. So, we're going to use Superbase for our database. We're going to use a hosted project there. This this can all be done for free. And then later, we're also going to use Railway to deploy our solution. And now, this is going to be an interesting one. I have never used railway and I'm going to show you how to use it while I am also going through this. Why? I did some research and for this example it's it's going to be the easiest to show you how to do this still creating a production app but still keeping it easy. So when we do our client projects, we always use some of the major cloud providers like uh AWS or Azure or we work directly on Hner but that is a little bit like that is a little bit too deep for a tutorial like this and trust me this will already blow your mind and go really deep but if we have to add that as well that will be that will simply be too much. So railway that is also something that we are going to use. And now let's come back over here. Let's talk a little bit about the project structure, right? So, we got the back end and the front end folders that are going to be separated. I also have an agent.m MD file in here. So, this is going to instruct the AI agent that we're going to use what how we like to work and what we want to use. So, this is going to be injected into every system prompt. And I'm going to use cursor for this. It doesn't really matter if you use cloud code or codeex. Like if you want to follow along using cloud code, I'm not sure if cloud code already imports agents.mmd. You might want to swap that out to claw.md file. So what you would do is you would literally simply set it up like this and then just uh turn this into a clot.md. So that is something that you can do. But for cursor, if you come in here and we go actually to my settings, you can see that if you go to rules, skills, and sub aents, you can see that right now it has the agents.mmd. It loads that in automatically. Now, I have a bunch of skills installed, but these are mostly for MCP servers like Sentry and I'm not going and for Figma. Here you can see I'm not going to use these. So, I'm I'm going to keep it really raw over here. And I'm also going to try and use little little prompts to walk you through this. Let's actually start one up over here because the thing is when you are working with these AI coding agents, you can there are a couple of ways that you can work with that. You can do like the huge crazy big plans and then just let your AI work in the background for two hours. But I actually find that when you are working on a project and a problem that you are actually comfortable with and deeply understand it's usually way faster to work in small inter incremental bite-size prompts and we can even like experiment with different models here. Probably most of the work I will use composer 2.5 fast from cursor just because of how fast it is. This is not the best model. I would say the best model right now is GPT 5.5. as of today's recording. When you watch this, it's probably already different. But I'm going to show you and teach you really how you can work with AI in a way where it's really going to work together with you. Because what I see most people do nowadays is it's very easy to get lazy and to just offload a a bunch of stuff and more and more to AI and simply don't even look at the code anymore. And yeah, while you can move very fast, you will also quickly like just create a mess within your codebase. And I don't care which skills you're using, like what prompts you're using and which models. If you don't have intermediate review points and decide what good looks like, it's going to get messy. Now the setup over here the agent.mmd this is already really going to help because it's it I already made a whole bunch of the decisions for us based on literally the past 3 years of working on projects like this. So for our back end we're going to use Python and we're going to use fast API to create the API layer. For the front end we're going to use a simple plain React single page application. and we're going to build it out in Typescript and we're going to use fight to um to to run the whole thing. Now for database super superbase then later we'll get into migrations and how we can actually create the database tables retrieval we're going to use PG factor for out we're also going to use superbase we're going to use openAI for the large language models and the embeddings and then hosting on railway other than that there are like short rules over here on how I like to set up my projects especially around minimizing dependencies as you know with all of the recent supply chain attacks, especially on mpm, this can really get messy and we just like to set up a project nowadays where we rely on dependencies like as little as possible. Now to to go the extra mile with that, I've also in the project setup already here in the back end, I've set up uh an npmrc file that pmpp is going to use where we're even going to specify a minimum release age so that we cannot import literally we cannot import packages that are younger than 7 days. So this is one guard rail really to protect ourselves and then in the pi project atml I also have it. So I'm I'm going to say we use exact bounds for our dependencies and also exclude anything newer than 7 days. So that is most of the of the project setup that we have and we also have files like this in the back end agents.mmd. So here is another short file that pretty much guides the AI agents when we're working on the back end. And similar we have that for the front end the agents.mmd. And as you go through this you might already be thinking wow this is pretty cool. And like yes it is like the fact that I'm giving this away to you for free already is actually pretty crazy because we've been optimizing this for for three years working on projects like this. And if you have this set up and just start to rip with AI your results will already be so much better than someone who kind of like vibe codes things. So that's going to be exciting. Okay. Now, I notice I I am aware that like this is a long kind of like introduction, but in my opinion, we we can't jump around here and and start skipping the fundamentals because otherwise you have no idea what's going on. So, that's why we need to spend some time here. There's one more document that I haven't covered, which is a this is actually a big one. This is the whole architecture diagram that I already went ahead and kind of like put in here based on how all of these layers work together, right? Because it's one thing to say, look, we have a Python back end and we have a TypeScript front end where we use React and we use Superbase, but it's also okay, how does everything work together? And the architecture doc is something that we will keep coming back to. But for now before we actually get get moving and start to produce some code here is the last thing that I want to go through. So this is the highle architecture diagram right so you can start to get a sense for first of all like the complexity of this whole system but also how everything works together. So everything starts from the user right I always like to reason from from the user from the user story. So an analyst goes to the browser they open the app and that is hosted via railway. So you can see this there's the front end that serves the app and then when we do a sign in so when they put in their account that whole signin process is going to be managed via superbase. That's really why I like using superbase. makes the whole authentication part really easy beyond just also being a great Postgress SQL database where we can store all of our data. So there's that and then we also have our backend surface and that is going to pretty much be the communication layer that handles everything right really the nexus that ties everything together and then separately you see the underlying um the underlying pipeline over here that's going to be our document injection pipeline. So this is this is something we do locally right now. We're not going to do an online upload or syncing mechanism. We're simply going to take all of the the the files that we already have and we're going to create an injection pipeline. create the embeddings using open AAI chuck it into the factor database which is in this case going to be a simple postgress SQL database as part of superbase and we're going to use the PG factor extension and then through that we enable all the logic in order to serve our application. So that's how this is going to work. So, what I'm going to do right now is I'm going to come over here and let's actually create a let's create a to-dos.md because right now we should already conceptually have the map of what it is that we need to do. Right? So now let's get clear on a list of action items that we need to work towards and there's not really any single right or wrong way to to start this. So we can start with the front end. We can start with the back end. We start with the database. We have to do the document digestion pipeline. And I am pretty much open to any kind of structure in here. I don't really have anything in mind right now. Uh but what I'm going to do as I'm going to talk through this, I'm going to use a speech to text tool called glido. So you might be seeing this popup. So when I talk over here, you can see this little thing, this dictation bar. This is Glido in action. And you can see when I let go, this is what we can use for talking. Now I know here in cursor for example, you also have the native one, but I don't find it as accurate. And the thing about glido is that you can use it across any app, any tools, and it understands all of your settings, your user dictionary. So glido is a software that I am a co-owner in. And if you want to check it out, make sure to check out the link in the description, get.glido.com/dave. You can get started absolutely for free. No credit card required. I can guarantee it will completely change your workflow. And if you're already using something like Whisper Flow, I would still highly recommend to give Glideo a try and see if you like it better. We built our application in Rust, so it's very lightweight. It's not this Chromium monster that eats up all of your RAM. And we're also bootstrapped, so we don't care about stealing all your data. And we are very privacy oriented. So let's plan this out, right? So how are we going to go about this? So let me reason through this. So, help me set up a checklist here in to-dos.md and what is the what is the most lo logical way to set up this project? Right? Do we start with the the back end, the front end? What what do you want to do? Give me a checklist that I can work towards in order to implement the entire architecture and satisfy the client brief. So, there's that. Okay. Little bit of um easy way out, right? Normally, you would just get to work and start working on this, but for the sake of this tutorial, usually I just keep this in my head. For us, it's I think it's good to all be on the same page. So, we can keep coming back to this. Let's see. Here you go. There it goes. So, we can keep coming back to this and keep referencing all that needs to be done. Right. So, also what I'm going to do right now, this is not something you have to do. Let's see. Yeah, I am going to create a development branch. So, let's switch over to that. So, we're now on development. Yes. So, we keep the main clean and let's actually see what what we got going on over here. Are we still good? Yes. So, the document copilot implementation checklist. Where do we start? Let's see. Um, yeah. Okay. We actually start with a database and sample the data. Okay. All good. all the dependencies. Okay, let's go. Let's go through this together. So, for me personally, I got Python installed. Now, we need a Superbase project. So, in here, if you've never worked with Superbase, there is inside, let's see, where is it? Guides, Superbase setup. There is a setup over here that you can go through. I already pretty much have it up over here. I created a new organization. So, you can just go to superbase.com, create a free account. You can sign up with GitHub, Google, whatever. Free account, free organization. And I'm going to create a a new project. So, what I'm going to call this, let's uh actually just call this driftwood capital. All right. Now, we need a database password. So, you want to set something strong ideally in here. So, what I'm going to do right now, I'm just going to do like password gen. Just keeping it simple. Give me give me a strong password. Something like this. Okay, that should be fine. So now what I'm going to do is I'm going to plug this in here. So that so that's good. And let's see coming back to our project. I'm also immediately going to set up av file and I'm going to say database password. And just for the sake of simplicity, I'm going to just show this to you. And like by the time I put this online, I'm just going to nuke the database or change it. But this is the easy easiest way. So you can just keep following along. Okay. So we got that. Um you can decide where you want to put this. Like you ideally just want to put this to wherever your uh closest to where you live, right? So I'm in Europe. Makes sense. Also important thing later when we are going to railway you also then want to have this in Europe because what when is this going to matter right so our application is going to query our database so if your back end that's going to connect with the database is in Europe let's say and our uh superbase is in the US it has to do a big network hop across the ocean right so that could add some latency so always make sure that all of your services are within the same region. If you really want to make this production ready, if you will, then you probably just like want to deploy everything in a in a private uh virtual network. But again, that is beyond the scope of this this video. Okay, so we keep everything on and uh we also enable automatic rowle security. Let's put that on. create the project and now I am here inside the dashboard and you can see that right now if we go to our tables there's there's nothing here so empty project this is the start now um what I also did is I added the env.example sample files both for the back end and the front end. So because this can always be a little bit tricky to configure this. So what we're going to do right now is we're actually both on the back end and front end we're going to create copies and then we're going to name them. So in the previous step you already saw me essentially creating this with data the database password. So let's actually get this out there and actually create a fresh copy. [clears throat] And when you go through this, you should already see those in the repository. So there's that. And now look, here's what we are going to need to fill in. So first, if we go over here for the front end, it's pretty simple. We need first of all our uh base URL for the API. We can leave that as is. So I should use this one. And now we need two things from Superbase. So we need our superbase URL which you can see this is going to be your project.supbase.co. So if we come over here you can see that here you have this code over here that is your project. You can also see like if you go over here let's see framework where do you see it look? Well it it it is shown in multiple it is shown in multiple ways however you want to connect to your database but usually the easiest thing to do is just get this from the browser. So let's get that in there and then just like replace this part with your [clears throat] project. And now we need our superbase anon key. So this is your public key. How this works within superbase? This is a key to connect with your database. But the important thing is that this is like publicly available and that's why we really need to be secure with rowle security. So only users that are actually logged in can access their own user data. But this is just a way of how Superbase works. If you set it up correctly, it's totally safe. If you don't know what you're doing, which a lot of people are that are just foding this, you have a backd dooror in your project. So, let's see where we're getting this from. If we go to our project settings, we can go to API here, you can see your publishable key. So, you can also see this key is safe to use in the browser if you have enabled row lever security for your tables and configured uh policies. So remember that's why we checked enable automatic rowle security during the setup. So let's copy that key and plug it in here. Right? Okay. Now this project is already set up in such a way that these environment variable files are get ignored. So if I now look over here into my uh version controller really in my work tree you can see that only the only changes here are this to-do file that we added. So anything we add in here is not there. So just something you want to be aware of. Okay. So that's the front end configuration configured. Now let's get to the back end. Right. So what we got over there? Yeah, I already copied it. Okay. So again, we need the superbase URL. So we already know this, right? Our superbase also needs the publishable key. Just make sure I'm using auto complete over here but just making sure that uh that is all correct. Okay. Now we need our superbase service ro key. So this is going to be our actual actual secret key. Now again I'm just going to show it here so you can see these API keys allow privilege access to your project's API. So this is this is going to be used in our back end. So we really want to handle this securely because this is also going to allow you to literally like make changes to your database and pretty much read everything. Okay. So coming over here the surface ro key going to plug that in. Again I'm showing you this here but before I will upload this before I will upload this video I'm going to nuke this database. So that's how I'm I'm going to uh use this. So now we need our database URL. So in here we need to get this URL. Now there's one thing uh that we need to be aware of if there are special characters in our database password because remember right I set up a weird password which was the one this was the one you can see there's a special character in here. So I think I may have just actually messed that up and ideally want to set it to this. So I may even just like change that just for the sake of it being a little bit simpler to just encode here in a string. So let me also show you uh how that works. Let me see actually if I if I remember where you one one does that in superbase. So got project settings I think it's database is it database and then in here settings reset database password. Okay. Um this is like reset type in strong password. Okay. So now I'm actually going to use the following without that little weird back tick. So I'm going to put in reset password. Okay, since I'm already logged in and authenticated, I can just do this. So now you also know how that works within Superbase. So then what I'm going to do is I'm going to uh I need to get my database URL. So how that works is I can click on connect over here top level and then I'm going to do direct. I'm going to use session pooler. So, the default is direct connection, but I'm going to click session pooler over here. And here's the connection string. Now, I'm going to copy that and I'm going to swap this out over here. And I'm going to take my password. And I'm going to plug that over here. Okay. So, now you can still see the issue over here because this is not parsed correctly. So, probably if I just put quotes around that, that should resolve it. Okay. These are always these kind of like little things that can really trip you up. And regardless of how many times you did this, because this is always such a one-time set and forget kind of thing when when something goes wrong, you're always like, "Okay, what what is going on?" But you can see if I remove those quotes over here, I think it's because of probably that hashtag and these like percentages over here that will break essentially within the parsing of uh environment variables. So that's why use the quotes around it. So that's uh that makes it safe. So for me, removing the back tick didn't really solve that issue. Okay. Now we're going to set up your OpenAI API key. So um let's see. Also for the sake of demonstration and without me having to hide it, I'm also simply just going to use a temporary one. Uh, we don't need this right now, but I'm already rather set it up. So, we have the whole project set up. So, what you can do if you go to our guides and to where is it? We may even already have it in the readme. If you go to platform.openai.com, here you can see your API keys. Let me actually just like remove this one. And then I'm going to create a new API key over here. So this is also something uh what you can do and then uh let me actually call it uh what was our company called driftwood capital. This is just for me so I later remove this so you cannot use my key. Okay. So now we also have our open AI API key and everything else for now we can leave as is. And then real quick, if you've been following along and getting value from this video so far, would you please do me a favor and hit that like button? Also consider subscribing to the channel if you want to see more content like this. Thank you. Much appreciated. So that is already quite some plumbing, right? I know if you're going through this, you might be wondering like, hey, this is taking a long time. But yeah, welcome. This is engineering in the real world. You don't just jump jump straight into like your AI agent and say, build me this, because then it's it's going to turn into a mess. Once we have all of this in place, we can now actually start from a good foundation. Okay, so there's that. Let's come back to our docs and to our to-do list. So, we got the OpenAI key set up. Let's see. We have the We have the download. We do that. And we have Do we have the download manifest? Yes, we have that because we data downloads. We have it. Cool. Okay. So that's phase zero. That is the setup. And now if I preview this over here, you can see, look, we got these fancy check marks over here. So now the AI recommended, okay, let's just first start with the back end. Usually I would also do this. Makes more sense. Like I know it's it's fun to play with the front end, right? And immediately get something going, but without a back end, a front end is nothing. Empty shell. So we're going to set up our project. So let's take highle overview of what we're going to do in this phase. We're going to set up the project dependencies, then our config file, set up the fast API input, and then we're going to start with our database models. Okay. And then perform the migration. Yeah, this makes total sense to start working on this right now. So, first before I jump into any AI tool, what I'm going to do is I'm going to go into our backend folder. Here we are backend. And we're going to look at the the project that we have over here, the pi project toml. So, when we're working with UV, what I can do is first of all, I can do a UV sync. This is going to create the virtual environment that we're going to uh that we're going to use. And now we need to add some dependencies to this. Now I let's see in the docs and in the guides and in the backend setup I showed you what we kind of like need and what we're going to work with. So I'm simply going to let's see I'm going to copy this. These are the original dependencies that we're going to work with. So let's set this up. Let's run this. And now here's the beauty of UV. It's going to add all of these dependencies. And remember in order to avoid getting yanked we specify exclude newer seven days. So now we have our project setup which is uh which is great and then we can add a defaf dependency. Let's see where were we. So yeah so we can also do when you do uv uv add if you do d-deaf and just put any other kind of like dependency after that it will add it to. So we do pi test and we'll do rough. It will add it to you can see dependency group it will only import those on uh when you're working locally right not when you uh deploy this to a uh server. So you can exclude that okay because right now also what I need to do because because I just added them they will be here but I now I can do UV sync and I can do d-deaf and then it will also import those right. So let's see. Yeah. Okay. Let's clear this up. So now this is our backend setup. Let's come back over here. So we got that. And now this is the point where I am going to start leveraging AI because now we have actually have to like we have to move and we need to create some files. So what I'm going to do is I'm going to come in here and create a new agent. And I'm just going to use composer too fast until I feel like it's not really surfing us. But what I'm pretty much going to do is I'm going to start very simple step by step and we're going to keep keep checking it. Hey, you're going to help me work on the to-dos. Uh go to phase one. We need to set up our appconfig.py. Can you set up that scaffold? Okay, let's run that. Okay, just checking whether you can still see everything. Okay. So now what I expect is I expect my AI agent to go into our backend folder to create an app folder in here. And in there we're going to create a config.py that is going to use paidic settings in order to load our environment variables. That is what I expect right now. So let's see if this sets everything up correctly. So first it needs to get a little bit because it's a fresh new session. It needs to get a little bit of an understanding of the codebase. There we go. All right. Okay. Expected an app config.py. And this is also where it's going to check. Okay. This is looking good. I can already see that my editor is giving some uh issues over here. And that is likely because I have not specified the let's see the um yeah okay I need to specify the environment and this is always one of those things because normally what you do is your virtual environment lives in the root of the project right and then your in this case cursor just knows okay we're using this environment variable but right now since it's not in Here what we need to do is we can um either set this up in our configuration or we can enter a path manually. So what I can do right now if I copy this path and just plug that in here now let's see it should have the correct configuration and it should also resolve those import dependencies. Right? So this is one of those things that you just need to get right and otherwise you get these errors. So let's see what it came up with. Okay, that all looks good. So let's let's quickly go over what we're doing over here. So we're using paidic settings and with that we can create a settings uh class from base settings. And this is one of my favorite ways to import environment variables because in here what you'll see is we can first of all load uh or link to where the uh to where the environment file lives. I have to check though because we we're probably going to because it now says that it has to look in the root but this is not in the root. So it depends on where we import this from in the next step. So this is just something we need to keep in mind. And then let's see what else. It's going to uh create an origin field here. That all looks good. Yeah. Okay. Cool. So now it's going to load everything. So now I'm going to continue. Can you now create a main.py Pi file and see if we can actually import this because this is always the the test, right? To see okay, can we now if we set up our main.py, it may Oh, it may already had that. I was looking I was looking in the wrong folder. So now you can see if we start to start to run this and actually you know what I'm also going to uv at d-dev ipi kernel. And now before we can actually run that, there's one more thing that we need to add to our pi project tol. So I will put it in here. And like working with Python, one of the most annoying things and I've been working with this language for over 10 years is managing paths. So the way you set up an entry point for your project to start importing files and to start running everything needs to be very specific. And if you want to go in and you want to run isolated files to run some tests instead of for example running it from the root and then uh just calling the uh main.py in order to start up your uh your app for example you'll get all kinds of of errors because what we're doing over here as you can see right now we're going to import the app uh or the settings really that we just created. So if you don't put this to your pi project autotoml I can let's see can I I can even show it to you if I just get rid of this and I now want to test this and for example just see like if we can import those settings it will probably say unless it's still got it into memory or something like that of course now it works you know because probably I didn't do the sync if I do the UV let's sync this because now it removed that and then probably what it will say it will say I don't. Yeah. Okay. There we go. There we go. Every time I think that I understand it, it's like Yeah, but do you really understand it? But I I understand this. Okay. So, this is now this should now uh this now errors because we don't have what I just removed. So, let's put that back the hatchling and then specifying app as an additional package that we can import. So, save that UV sync. And this is the the thing is this is the hard this is the hard part usually of projects because when you are following tutorials everything is very easy very cookie cutter but this is where you spend time in the real world. So now let's go through this and let's see if we can import because look now we have our settings object import [clears throat] settings over here settings. There we go. So now you can see our app can actually import everything. That that was the main goal of this first check. Right. So why I like using paidic settings is because let's say we have our superbase URL right let's say we come over here and we say superb base URL let's say we don't have that let's now come back let's try and import this again look what will happen we got a validation error and what there is a particular required field that is missing and that is the database URL so that is just very helpful especially if we move into deployment where it can be a lot trickier to manage all of your environment variables. So, let's put all of that back. I also noticed that I made an error and actually was putting my uh credentials uh for the database into this env. So, just make sure if you follow along, always cross check that, right? So, you don't you keep your example clean. That's also in version control. Um I see like I like you see I messed everything up here. So, this is always great when you're doing these tutorials and you're talking through things as well. Cool. But anyway, I'm going to nuke the database. Uh, anyway, so just double check everything. This is still good. This is all placeholder. This should be actually your project ref. Okay. And we're good to go again. Okay. So, we got our setup correctly. Now, this is a placeholder, of course, or an entry point really for our fast API app. So, we got a health check in there, and we can already start running this. So, what I'm going to do is I'm going to uh set up, can you set up in the backend folder, can you create a very short readme file with instructions on how to manage our fast API app? So, this is also an exercise that you can go through. We want to document everything in here just so we can not only start working with this kind of like file by file but also test the whole thing overall. Right? So what I can do uh once this is set up let's see let's get rid of this create a new terminal session over here you can see we can go into let's go into the back end and now we can run UV and we run it and we via Uicorn and we do app main app reload. So this is going to be the command to start running everything. And we got a health endpoint. So if I now come over here, this will there's nothing there. And we can just run this that will two slashes status. Okay. So we got a connection with our app. So that all is great. Of course, there's not much else just yet, but at least we know we have an entry point and we have our config and we can import our settings. So, that is great. So, let's see what else we now need to do. Just clean this up. Okay, simply going step by step. We got our entry point. Cool. Okay, so now we're going to actually start working on our database. So, we're going to let's see where are we? We're actually going to create some tables. And one of the easiest ways to do this, which was a big unlock for me, like instead of manually tinkering with SQL code in here, uh, which you really don't want to do, we're going to use SQL Alchemy. Um, that's a Python library that we can use. And with that, we can define our data models within Python and then use that to sync them with the database. And then later if we want to make adjustments to that we can even use to make database migrations which pretty much means tracking changes and applying changes to your database schema over time. So now this is another excellent point where we can start running things. So let's do things step by step and let's see let's open this up over here. Okay the next step is we're going to create our database models. Can you help me set up the following? Now again what do we expect? I now want our model to come in here to app create a database folder and I want to have the following tables created. So let's take a look at that. We have some profiles. We have source documents, document chunks, our chats, our messages, and our citations. And we'll have to kind of like see if this is really the ultimate final final thing that I want to dive into. I'm not so sure about the message citations whether we need that and also see if whether we because the thing is we of course asked to create this to-do file uh in the beginning right and I'm not so sure what is meant with profile so we may need a little bit do of a push back there uh to see what we're going to do okay there we go so it now created the database table and we have a basepy in here so this is going to be the base interface is from SQL Alchemy that we're going to use. Now, we're going to create the models in here. Yeah. And this is where we're going to map out everything. Okay. So, that's starting to look good. Are we already done? Yes. Yes. Yes. Okay. So, let's see. So, profiles. Yeah. So, I don't really like pro like first of all, can you change profiles to users? because this is going to be the email optional display name. For me, it just makes sense to use users. Then we have our original source documents. That's going to be okay. Then we're going to have our document chunks. That's okay. Then our chat threads on by profile. Then our chat messages. That makes sense. And then okay, message citations. What is the idea of this? Link assistant messages to chunks with denormalized filing metadata for the UI. I don't understand what this means. So this is also look the process right of not just blindly accepting everything but pushing back and then saying why do we really need this? Okay, it changed this. We're currently going to keep everything and then once we sync it to the database we can check it for sure. So let's check it. Right. So what's the idea behind the message citation table? Why would we that was called message citation? Why do we need that? So let's put that in there and just because maybe it was part of the uh architecture dock that was created at the beginning. So let's do a little bit of a of a push back. Okay. So short answer is how the product keeps it trust contract for Tiffrey. Okay. So what's the idea? It is a what does it do? What problem it solves? There are three different things in play. We got chat chat messages document chunks message which which passages this answer actually relied on. Yeah. Okay. So, it's going to be related to the chest chat messages. So, I think it makes some sense to have a separate table for that. We could also just put it in here, but we can enforce the grounding a just a little bit better. Okay, let's just Okay, let's let's keep it for now. So, we got u we got that in check. And now we just want to see okay, how everything comes together. And now we're going to create the original albeic initialization and do the first migration because right now if I look in here we still have nothing right we just what we did if is you if you come in here you can see right we created the uh we created the models and that is now all in one file that is okay for me right now normally you know what it is actually cleaner can you take the database models and split them up into different files. So each model has one particular file. That is actually cleaner because then we can actually click on um a chat message and see directly what it uh what it what it is. That's usually how I like to set things up. So let the model rip through that and then also check about what the next action item is. That is really creating the migrations and that is where we're going to see things show up in the database. So, while we wait for this to rip through it, we can already put in line the next uh the next message in here. Is always keeping my context really uh in check. So, we're still right now since we're still working on this task, this is still good. But after this, I really want to like make sure that we reset everything. So, okay. So, look now in database, we have a models folder. So, now you can see every let's see keep keep the file. Now every database model which is going to be table within our database has a single file and that is specified over here. So we got what's the table name what what data um what data do what columns do we have in here and what data type are they. So this is a nice and clean setup. Okay perfect. Now let's continue setting up the migration. Prepare everything and then tell me when you're ready before making actual changes to the database itself. skipping the control loop over there. So now what we're going to do and for those of you that don't know what alamic is, this is a way to like I've said to manage a database migration. So we can define our data models here over time, make changes and everything will be tracked and then once we perform the migration it will show up in superbase. So that will come together in a bit. So let's wait for this to finish. So now you can see it decided to add the Olympic folder over here. It did it outside of the app folder which well it's okay. It doesn't really matter. Depends on how you set it up. Usually I like to have it in the app folder itself. Just keep everything contained but let's see how it goes. Don't think it matter that much. Okay, there we go. So what was set up? So here in Alamic you can see it created the first version of the database migration that we can apply. So if we keep everything, this will pretty much be a this will be a single Python file that says everything that needs to do. So you can see right here are all the SQL queries that we can apply. And then once we make changes later, we simply have more and more files in here. And now what it will do, you can see before you apply, confirm your we have your database URL, everything is set up correctly. So now we can actually run this. So let's come in here and now here you can see we have two options here. We can first do a preview or we can directly apply it. So since this is just a fresh new project clean database for now I'm totally fine. Let's just run this and first of all see if everything works and you can see we got some errors. So where is this coming from? Yeah. Okay. This is always a really annoying library with within Python G2 the one that you see over here. So, it could be that we need a binary for that. Uh, where is it? It already have the binary. Okay. Well, instead of just spending time figuring this out, we're going to do we're going to hit the easy button and we're just we're going to hit it with a please fix. We're going to hit it with a please fix. We're just going to do that. Um, so what what's going on over here? So, there there is some import error related to this library. So let's see what it does. Yeah. So it that's the thing. If you have Psychob G without the two after it, it uses version three. And there's a little bit of a difference between how that works. So what do we do? Normalizing database URL to use version three. Yeah. But a plane. Okay. Yeah. So what's the fix? This is why we actually should run commits. This is that is not something I'm doing. So this is uh this is actually the part where we should start doing that because now you can see yeah it normalizes it normalizes that okay um yeah okay so what's going on over here it uses we use the uh where is it the database URL right so here you can see it uses that database URL and then when we try to connect to it using the URL we need to rewrite it to the following so we need to add that cycle Jacob G in there. So, we can either add that in the database URL itself, but what the AI decided to do is add a little helper function in here because this is specifically for using SQL Alchemy, which I think is a clean thing to do. I'll use it for now. I'll accept it. And let's see if we now clear this up and we run it again. Look, now the migration seems to work. Okay. So, first of all, we are going to make sure we have a commit in here. And let's see. Do we want let's put everything in here because now as we start to like progress now we have an actual project, right? Before it was set up, now we have a project. Now, we want to track keep track of the changes. So, we can clearly see our divs when AI starts to mess with some things. But the cool thing is if I now come over here and run this, you should see we have tables. So we have all of our empty tables, our chat messages, document chunk messages, source users, and also our almic version, which is the only table that currently has something in here. Cool. Okay, so this is great. We're moving along. So now if you come over here, let's see. Okay, can you now update the to-do file with and check off everything that we did and run the final check if any? Okay, so now we want to update everything. Let's see. Let's clean this a little bit up. don't need this. So, we're getting ready for phase two to set up the authentication. So, still a lot of the plumbing, right? It's not as sexy as some of the tutorials that I usually share on my channel where we go straight into like, hey, here's how you build stuff, but this is how you set up a project. I really can't make it any easier than this for you. Well, the thing is if you go through this and you have all the code, you can re drastically speed this up with AI because I am on purpose going a little bit step by step there. But that is also how you really ensure that everything is according to your best practices and standards, right? Okay. So, we have everything set up. One of the important things to understand is that we also have already used the PG factor extension for Postgress to create a vector column. So let's actually validate that over here. So if you use document chunks uh let's see where is it we have the embedding. Look at this. You can see that embedding that is a vector. So this is something you can do in Postgress using PG factor which is very convenient and we're going to use that later down the line. So there's a lot of even though we're moving step by step there's still a lot of stuff that goes on in the background because of that architecture file that we have created. Hey, then quick note. If you're serious about AI engineering and you are watching this video not just to build portfolio projects, but really to make the next move in your career and you want to continue learning beyond what we do here, make sure to check out the links in the description and in the pinned comment below this video. Beyond everything that I share on YouTube, I have two flagship programs. One is really all in on the technical side of AI engineering. So, not just four hours like this, but 60 hours covering everything step by step. And also another program is our freelance program where we teach developers, data professionals, people with a technical background that currently already have a job how to start taking on freelance projects with a focus on AI next to their full-time positions and start to make money from that. So if that sounds interesting to you, make sure to check out the links and our website to learn more. Okay, so we're ready for phase two. So what I'm going to do is I'm also going to create we're going to create a new uh chat. So we reset the context window and the goal is to set up the authentication full stack. So we're going in the back end we need to set up our authentication scripts and in the front end we first need to set up the scaffold. Okay. Yeah. So this is going to be fun. So first what I'm going to show you is on the front end right now let's clean this up. We have like an empty project over here. is pretty pretty empty, right? There's not much there. If we go into the front end, where are we? Front end. Yeah, there we go. And we go into our guides, front end setup. We are going to um use pmppm for this. And we can set up a project template. So, if I come in here and let's for the sake of simplicity, I'm going to run this. We're going to use pmppm. So, make sure right now you have note and pmppm installed. We're going to create a fight project like in the front end folder with a template react typescript. So we're going to run it and this is going to set set up a whole bunch of files. So uh it says is not empty is not empty. Yeah, that's correct. So we're just going to say ignore current files and continue and install with pmppm and start now. Yes. So we also run the installation. It's going to run everything. start a dev server and boom, we have we have a setup. We have something running running on local host. This is going to be our template. Um, but why this is so useful? Like look at what what we created like like uh let's see where are we? Let's switch sides. There's a whole bunch of stuff. This is also like I'm I come from like the Python world, right? I'm a back end engineer, worked as a data scientist, AI engineer. Front end has always been a little bit foreign for me. Now with AI, I can do it, but I am by no means a TypeScript expert. So on this part of the codebase, you'll see me like vibe code a little bit more, if you will. But like look, there's a there's a lot of setup that's required on a project like this. And instead of just like letting AI create everything, let's just use a template so we know we have the proper setup. So uh it already started everything up. I think with that command, it already ran the install, but I'm going to run it either way. Okay, so this is all good. Now, we're also going to add our dependencies. And like I said in the beginning, I want to really minimize dependencies, but there are a couple of things that we are going to use. So, we're going to use Superbase. We are going to use Tailwind and we're going to use SHT CN. So, Superbase is just you know like the database. So, we can interact with it. That's that's great. Tailwind CSS is going to help us with styling and shet CNN is a component library that is going to give us all kinds of cool components out of the box that we can use and this is going to make the whole process of building cool UIs extremely simple. So again this is this is a tip over here like this is this could change the way you build projects forever. So uh yeah this is going to be really exciting. So we're going to run those just one by one. We're going to add superbase. And what this will do if you go into where is it? Package.json it will pretty much like install it and add it over here. So you can now see that uh superbase should be somewhere in here. Yes, there. And whenever you sync your project and you run pmppm install or I it will import all of those node modules similar to what your pi project autotoml will do in your virtual environment. Also node modules excluded from version control. So you can see in here will probably be something. Yeah, note you don't want that in version control. Okay. Uh where were we coming back to our guides front end setup. Okay. So, let's get tailwind in there as well. Let's get tailwind. And then finally, let's get shn in there. Let's get the components in there. Okay. And then we can just hit enter in here. And let's see the component library. We use radics. And then it asks us which preset we would like to use. So, there are different presets to start from the if you want to preview them, you can go to uh let's see, let me move me over to the other side. So, you can do go to uh UI setten.com/create and you come into this fancy little editor over here. So, look here. You can see all of the different styles and they're very subtle. Some have like changes in the fonts or the icons that they use. Uh but again, it doesn't really matter as much and we can always change it later. And then here you can see, okay, look what what do you want for now? I'm just going to keep it simple and I'm just going to go with noa. So we'll use Lucit and Jist as the font. Okay. And then we do we do hit a little bit of an error here which is expected because if you follow the default setup right that we have in here. So the front end doc what we [clears throat] were following after you install Tailwind CSS it still needs to be wired up. This is just a little bit of manual plumbing plumbing. There's probably an easier way that you can set this up. Uh but in this case I just like ask AI to set up the right imports. Right. So here you can see what was fixed. So we needed to add the tailwind alias and also like it's in all of these config files, right? So over here config.json app.json and the source index that is uh AI is just like of course really really good at that. So just set all of those up. So now you can see let's see now we're good. And you can also see that in the component we have imported the simple button. So that's all working correctly. Now, if this is the first time you're doing all of this, this is super abstract. I know it. So, let's actually continue moving so we can actually start putting some of the pieces together. Now, one of the things that I noticed was that we actually skipped one step in the to-dos. So, if I open this in preview, you can actually see it. So, we still need to wire up our superbase.py, PI which is essentially going to be our client that we're going to use to connect uh and interact with our database. So right now now that phase two uh that was started that we started with the uh with the setup first I'm going to quickly tackle this. So can you let's see we're going to let's just copy the path. Let's link it to that and then I'm just going to put this in here. Can you implement our superb basease file right now? Just going [clears throat] to put that in there. Okay. So, while we leave that running and tinkering, let's also again zoom out a little bit and talk a little bit about how I'm using AI here right now. See, because there are so many different ways that you can use these AI models. You can use them in the right way, you can use them in the wrong way. But where I see a lot of engineers make the mistake is they get they just get so so lazy and everything should be this big gigantic plan. It should be huge and then you probably don't even read it anymore and then you say okay go build and you let your agent work for I don't know 30 minutes an hour on it. And I think there is there is a place for that. I also definitely sometimes do that, but especially if you're setting up the beginning of a of a project and you know what good looks like. Staying in the loop like this is often in my opinion a way better approach because the thing is if you use these like big big chunks, right? You're not going to understand your codebase. And I don't care how good AI gets, you still with every project, you need to have this mental map of, okay, where do things live? How is the project structure? Where do I go? Because otherwise, any addition on top of that, you're going, oh, what is this file? Oh, it it changed three files. And then, oh, okay, sure. And that's a very slippery slope towards pretty much just vibe coding a codebase. and they can work but you don't really fully understand it and this way especially if you use like a faster model we can also like in a bit just like even switch to GPT because GPT 5.5 is really good and then me maybe even set that on like the fast mode and set the reasoning very low that will probably even still be good because we've put on put in so much work in the beginning right so now you can see let's actually check did it update It's the let's see is it done? So now if we go here in the database we should see we have our superbase.py. So this is going to be our client that we're going to use to connect with our with our database. So we run everything through that right now. That is using our superbase anon key. That all looks good. You can check off phase one. Okay. Perfect. [clears throat] That is now implemented. Let's put that over here. Now, let's come back to phase two. Hey, can you check our current progress into to-dos on phase two? What do we need to set up? I already set up a scaffold for the front end. Let's get clear on our next steps. [clears throat] Let's just keep it at composer and continue to run at this because looking at this from the data from the back end point of view, we just need to set up the authentication and then in here we have our scaffold. We need to manage environment variables. So that's that's going to be the next big thing as well. Okay. So that's a lot of information. Okay. Yeah. So it recommends to start with the back end. Okay. Let's go import or let's go create the dependencies.py right now. So first going to in the app going to create uh an out folder in here and set this up. Okay, there we go. So it created the dependency. So while we're going to take a look at that, let's already instruct the following prompt. So now on the front end, we already did the scaffold, right? So now let's see if we can do this in one chunk. So we have we have the environment variables superbase http and then let's see let's first set up this and let's say continue with the [clears throat] following. And now let's check what we got going on over here because now we have the O dependencies file in here. Okay, so this is what we're going to use as dependencies for our fast API endpoint. So we can interact with the database and pretty much can check whether someone is authenticated. So let's see coming back over here. We're working through things. So now we're going to work towards the signup process. So this is always a little bit tricky because here we also need to uh interact with superbase again and for now we're just going to set a use a simple like email only uh setup. You can also with superbase you can also do like the login with Google single sign on all of that. But now let's see. Yeah. Okay. Let's continue with the signup process and also tell me what I need to do in Superbase in order to continue with this. Okay. Let's check this out because right now this is always where uh the difference between working on a project locally versus what happens um on a deployment can be different right because from superbase we're going to have different callback URLs. If we do a signup we may need to send an email and I probably in the beginning just want to avoid email confirmation just to make it really simple to work through things. There we go. Okay. So now it's just like moving through this. Now we need to verify this. So here we're going to spend some time again. So we have the front end signup login that's created which should be protected. Okay. Okay. Okay. What to do in superbase? Okay. We need to so before we can actually test this let's actually check the following and indeed disable that user authentication. So, first we're going to come over here. Let's say we go to authentication. Where are we? Authentication. And then we go to emails. And let's see. Confirm sign up. Uh that's the email itself. Let's actually see where do we do this. It's dashboard authentication signin providers. This is what I need. Yeah. Okay. And then we turn off this one. And then for now we're also going to we're going to disable this. So you can see here here are all the various ways that people can uh set up or log in via an account. Right? So very a lot of providers out of the box. We'll not go in that. We'll only do um email addresses. But since we don't do any email confirmation, this is going to be an internal tool. I'm going to disable allow new users to sign up because otherwise if we deploy this, if we don't put it behind some type of uh firewall, VPN, whatever, anyone can access it and anyone can create an account on the Driftwood Boot Capital website. So, what I'm going to pretty much say right now is uh let's say we set up we disable that and then we need to go into authentication and we need to do the callback URLs. So uh let's see that is in URL configuration and then we need to set up the following. So we set up the following redirect URLs and we store that in here. So that is under authentication URL config then here. So now let's actually see um we already did this and we tested that. So now what we can do is see if everything already works. But there's going to be one thing that I'm going to say. Okay, one thing I disabled new user signups because I don't want to expose this via the front- end application. So how can we create or what's the easiest way for us to create a user? The thing is we can do this actually via uh if we go to authentication and then to users probably the easiest way is for me to right now let's just do Dave at um let's just call it driftwood.com doesn't really matter. The thing is we're not going to use any email validation. So let's do I don't know test 1 2 3 4 5 whatever. Create the account and then we do auto confirm. So that should now be set up. So this is also a process that you can follow and it should now already automatically be uh be authent or uh be confirmed right. So this is a whole big world of okay how do you do this how do you send out how do you let users sign up but in the beginning for an internal tool I always recommend to just disable this and just manually add people. So now we have that in place that's probably also what uh what was set over here. So now we can let's see keep all changes. We're going to open up a new terminal and first of all I am going to say let's first run the back end. We're going in the back end and remember right we had our command to run the back end. Let's see where is it just the following. We can make that simpler in a bit. So let's run this. So now we have our back end running and we're going to create or open up a new terminal and now we're going to CD into the front end and then I'm going to run the development server. See if this is already running. See I'm not sure what it ah beautiful look. So let's clean this up a little bit. We can already see some type of scaffold that we have over here and I created the account right. So I can do dave at driftwood.com and then do my super strong password. Now this should work theoretically. Let's see. Do this calls get me with superbase. Yes, this verified. Let's actually check. Okay, that looks good. Looks like we got a good configuration over here. So that is always nice. I'm just just checking like this is where like we go pretty fast in here. So it set up a lot of stuff to um to perform the validation. So now I'm actually just going to say hey this is what I got after logging in. Are we good? Very naive I know but like the whole thing with like explaining reading I don't have time to like read for like 15 minutes like go and understand this. Ideally, this process you should understand a little bit better to see what we got going on over here. But we got the account, we created this, the back end verified it. H and we can even like start to experiment with it because if I just do something like I don't know like this, enter email at testest.com. We'll probably say infot login credentials. So, it seems like we are doing pretty okay here. Yes, phase two is done. And that's what I thought. Okay. Amazing. So, you can see we're already bringing a whole bunch of things together. So, let's see. Let's close this out and come over here. So, all of phase two is completed. So, that brings us to phase number three. So, let's open that in a preview and close this out. Have a little bit of a check to see what we're going to do. So now we're going to create the chat cell the vertical slice and this is going to be first of all we have our back end setup and then our front end setup. So let's just first create the uh the endpoint pretty much in our back end that the front end is going to work towards. And here instead of directly going to like go implement what I'm going to do over here is we're going to f here I first just want to see what it's going to do because this is going to be we're going to take bigger jumps. Can you look at a to-do list phase three? We're going to implement the backend part that I listed here below. Can you first create a plan to show me what this looks like? So, I'm going to start that and then I'm also going to do exactly the same for the front end and I'm going to plan that. Can you look at our to-do list and check the front end tasks that we need to work on for phase three? And can you create a plan for this to show me how we're going to implement this? Okay, so we're going to run that side by side and wait for those plans to complete. Okay, there we go. So, we got the backend plan. So, let's have a look at this. Okay, phase three back end. We're going to do the chat uh cell stop means we're just going to create the boiler plate, the starting point really without implementing the actual logic in there. And uh let's see what's already done. This is going to be the architecture. So, yeah. Yeah. Yeah. Get current user blah blah blah. Okay. So the API surface we're going to so the whole thing that we're going to do right so we're going to users can have chats. So in the database a user can create a new chat uh which has a thread ID and then underneath that we're going to store messages. So if we if we come back to our original database design that's how it's set up right. So we now have a uh user is probably not in there. That's something I created a I created um an user within superbase for our authentication but that is not the same as creating a user here in the table. So that's something that we need to uh that we need to set up. And then chats and messages are connected. So that's what we're going to set up right now. So let's see what this is all going to look like. Key decisions. Yeah. And the the the reason why it's getting all of this like pretty good out of the box is because of course we have this whole architecture MD that is constantly referring to, right? So that's always something to to keep in mind. Now this is definitely going to be a big a big chunk where we need to do some reviewing over here. But for now again I'm just going to uh switch over to building mode and let it work on this. And before that I'm actually also just going to do just going to commit a checkpoint over here. And then going to the front end we can now look at that plan to work on that in parallel. So that is just pretty much setting up the scaffold here of the implementation and creating the layout. And with the front end, I'm always a little bit more forgiving when it comes to like VIP coding because pretty much from the front end, once the foundation and the authentication is set up correctly, it's just like look, if it looks good visually and it works, good to go. That's also probably because I'm a mostly a backend engineer and that the whole world is a little bit more foreign to me. But for for this, I'm just going to hit build over here and go through this. So now the back end is starting to work on things. And now this is this is the moment really where we're going to accelerate because now you can also see like this is a lot. If you see all of this, if you have to like manually refuel all of this, it's going to be it's going to be a lot, right? It's going to be a lot. So coming back to our architecture, just zoom out. Uh where's our where's our dock? Yes. So we set up our superbase and now we're working really on the communication between okay how can our front end application communicate with our back end with our API while handling the authentication via um via superbase and of course retrieving the the messages. So that is really where we're going to configure because like look this is now it's just like this is too much to like manually go through right now. So with this project is like I'm I'm this is also the first time that I'm doing an actual live build really like this. So for me it's also finding the balance between making this still enjoyable to watch, right? And not me just like reviewing code for an hour. So let's just see how things how things go in here because the back end already it already went through it and now the front end is also done. So you can see it like cranked out a lot. So here we're moving fast. So, one other additional thing that I'm going to do is I'm going to like first of all like and I'm also like I'm not giving these commits any meaningful messages right now just because we're working towards the first version. What I'm going to do is I'm actually going to open up dedicated terminals right now. So, I'm going to open up this folder and I want pretty much one in the back end. So, you can see over here. So this is going to be a terminal in the back end and then I'm going to uh create one in let's actually do that over here. Open up terminal here. Going to do that in the front end because from now on we going to be looking at these terminals a little bit more often. So in the back end [clears throat] so let's see we're already in the back end and let's just run the API over here. And let's see, ppmdev, we don't uh wait, we don't need. So, okay, here we go. So, now we can um have a little bit more space really within cursor over here and not look at the tiny terminal. So, the tasks completed, plans implemented. Now let's come over to our front end application and see what we So this is the sign up on the sign in. So again we do Dave at driftwood. We do our sign in. Okay, look this is what it created. So we're already starting to get a little bit of a feel for what this could look like. It's still a little bit ugly. Not much much is happening here. We can sign out. We can start a conversation. But this is we can at least log in right now. And we have the scaffold. So that's great. So now let's clean things up over here and come back to our to-do phase two. Okay. Okay. Can you now update our to-do MD file? Also check the back end and the front end. Where are we at right now? Okay. Phase number four coming up. That is actually is going to be an interesting one because that's mostly going to be the separate part underneath over here that we're going to work on. Once we have this scaffold, we're already in a pretty pretty good spot. And as you can see, phase three is now done. And that's also because I let the AI of course create the entire plan for both of those faces. So on the back end, we have set up, let's have a look at what we did, right? We have our app, we have our API, we have our endpoints. So we have chat, we have O and on the front end we have created everything from like the login to the chat list also the API endpoints to work with the AI SDK from PCEL and we have set up the basic uh the basic chat. So things are coming together but now we're going to take a step kind of like in a little bit of a different direction and start working on the ingestion pipeline. So here remember right we have to work with the HTM files that we downloaded. So these files and if you open these you can see there is a whole bunch of HTML uh crap pretty much that we don't want in here. So all of these XML text. So we first we need to parse those. So, what I pretty much have in mind is first I want to take these original downloads and I want to turn them into markdown and I'm going to create a separate folder for that. That's I think that's the right way to do it. So, we have the original, we have the downloads. Let's actually put it in here one level higher. So, not in downloads and we're going to model the exact format over here. But now we want markdown and I want to use a particular library for that and that is a dock lane library. So what I'm going to do is I'm going to uv this is going to be a dev dependency because we're going to run it locally and I'm going to run uv at defaf dockling. Uh and that is because I'm not in the back end. It should be here. So now we're adding dockling as a defaf dependency. So what doling is? Dockling converts messy documents into structured data and simplifies downstream document and AI processing by detecting tables, formulas, all of that. So that pretty much sounds like something we need, right? So I know this also because I did an entire project on dockling. So, if you go over to my let's see AI cookbook and I go into knowledge, there is an entire there is an entire setup on how to work with darkling. So, I may even reference that. But let's just see because the library may have changed a little bit. So, where are the docs? Docs are over here. So now first what I'm going to do is I'm going to say um I'm going to start working on this. So let's keep the let's keep the to-dos open. So where are the docs? Let's keep the to-dos open. So we need to do an ingest normalized chunk. Yeah, that's what we're going to start with. Okay, I am going to come in here and I'm going to first link the docs. And let's see. Here's what we're going to do. Can you go through our downloads folder and take all of the HTTM files and convert them to markdown using dockling? I already installed it as a dev dependency. You can reference the official documentation below. And I want you to export them to the markdown folder which is in the data folder. And I want you to use the exact same folder structure with the years and the manifest but now in markdown. So we make them accessible for later processing. Okay, that's that's the prompt. So now I the only thing that I did not mention is where to add that script in order to do it. So actually I'm going I'm just going to stop it one more time and adjust it. Can you create the script in order to run this and add that to the data folder itself? So don't make this part of the backend code because this is separate. put it in the data folder as a separate file. I think this is how I want to do it. It doesn't really matter as much, but I think it's just clean because again, this is going to be isolated in and only something we run locally right now. So, we're not going to do an online generation process. And with that, it's always just best practice. Anything that's not required for your deployment, like keep it out of your like application folder because it's code essentially that your app doesn't rely on. So that's why we can do it. The only thing is like the pi project autotoml is here. So let's see how it's going to how it's going to handle that. So it's going to create the convert. So in the data folder convert to markdown. Let's just keep an eye on it. So it's going to convert to markdown. All of that. Yeah, looking good. And dockling has a very simple interface. So if you you can see so you can from dockling we implement the document converter which is the class that holds all the functionality and then here you can see it it sets up the paths correctly to make sure we know look here is where the files are this is where we need to put it to and then there's one it creates one simple function and it's going to run it over here. So let's briefly so this is probably running in the background right now. This can take a little while. By the way, this is all open source. This is all free. It just uses your compute, but it does use some uh it can use some or it does use some models which are component can be a little bit expensive. So, if you if you run this and your laptop's fans start to uh start to run, you know where it's coming from. Nothing's wrong. It's just a process. So, here you can see, okay, how does it work? So, first we start with a manifest because that essentially links to all of the files. So, we load that. We create the folders if they don't exist already. This is where we set up the converter. And then we have the manifest. We load that in. And then we're going to run for everything. So, you can see for the filing in filings just going to loop over all of the documents in here. We're going to let's see get the path and let's see where the actual magic happens. This is where the magic happens. result is converter.con convert HTML path and then we're going to write that back to markdown like this is it this is the logic everything else around it is just managing paths and imports and this is where uh like look there we go we this is already always cool to see so that converter you can see that is set to the document converter so that's why we are able to uh to run this let's seeun Okay. So, okay. I think we're already through it. So, let's now actually open up a preview and see what we're working with. So, now you can see we have a markdown a markdown file. See all clear with tables in here. So, let's actually see whether it did a great job at that because um yeah. Okay. So, that's starting to look good. So this is really great about duckling is it can recognize these tables and now uh there are lots of ways that you can process data and put it into a structured format right so there are various paid resources like all the big cloud providers they also have tools for this but I found duckling to be really good and I think especially for documents like I'm just going to scroll through it to see like look like and this is going to be the most important part right all of these tables do we get the tables correctly. You can see so there are big tables in here. So that all looks it looks good. But then this is something that we may need to keep a check on like here like is this empty? I'm not sure. Are we missing information over there? So the funny thing is like also on this right now it's a simple short step within the whole uh pipeline but in the real world it could very well be that this whole ingestion process of taking the raw data and turning it turning it into markdown could very well be the step where you spend the most time because as you imagine right if if if you get something wrong here then downstream information is all is going to contain those those errors. So, I just really want you to be aware of that constantly kind of like striking the balance between me like moving forward, showing you the best practices. But if this were a paid project, I would spend some time probably if like a day a day or two to really dial this in and see and experiment with different methods to see if we really get everything correctly. So, we may come back to this later if we we notice an error. Like usually like the the plain text that all works but the tables like having and you you can imagine like having this column for example just like one to the side can mess up all of the information. Cool. So now we have all of that. So we can now put this into um into our database if we want to or immediately go to the chunking. Let's actually see if we Yeah, the source documents. Let's see. Did it already put something in here? No, it's still empty, right? Yeah, that's good. Okay. Um, yeah. So, now Okay, so we exported all of the documents. Can you now um put this into the database? So, it's in our source documents. And you know what? Actually one thing that I now think of what we can do is we can use a parallel kind of like agent in the background and we can say I just converted all of our htm files that are in downloads to markdowns markdown using dockling. you are a document reviewer and can you audit whether all of the information is correctly? Now, you can never check everything because the these are long reports, but I want you to specifically pay attention to some of the tables and do a quick sanity check to see if Dockling was able to correctly convert the tables. I just like I want you to do some quick random checks and if you run into any error, let me know. Now, that could be a little side quest that we use another agent for. And as you can see, I use GPT high. So, this is okay. We use a B for your model. This could easily run for a couple of minutes. So, let's see if we get anything anything back. Hopefully not because I don't really want to spend a lot of time fixing that, but at least we know, right? At least we know. Okay. Um, let's see. Let's see. Let's see. We're now going to uh ingest it. So, let's see where are we? We have load source documents. What did we create? We created an app in in our back end. We created an ingest folder. And this is going to load the source documents. So, it's going to take our markdown files. We have our company names. And it should put everything into the database. Now since this is a one-time manual batch upload, we don't need this in our Olympic migrations and I actually prefer a simple simple script like this. So this is again something we can just run from here. So let's see. We have our database with source documents. Nothing in there. And we do UV run and we run it as a Python module. I do we Okay, let's do it. I don't I'm not sure if we need that. Okay, skipping existing. What? Why is it existing? Let's see. All right, there we go. So now we should have 25. Yeah, we can check. You see 25 records, five times high, five years, five companies. And it's beautifully put everything in here. So we got the ticker, we got the uh like the number in here, company name, we got the which form is filing date and let's see, we got the source URL. And now we have one beautiful mark uh text field in here which just contains the whole markdown file. Okay. So this is good. All right. Let's now come back to our progress because now next what we're going to do is we pretty much did the beginning. Now what we need to do is we need to take our documents where are they? And we're going to create document chunks around this. So, I'm not going to go super deep into how you set up rack pipelines and how all of that works um conceptually and theoretically because I have a whole bunch of other videos that do this. So, um I'm just going I'm just going to do the process, right? So, again, if with any of these techniques, if you're like, "What the going on?" Like, you can reference some of my other videos where I go like one entire video just on that topic. But the main idea is that we're going to split them up into chunks so that we can later use them for retrieval so that not every time a user asks a question about the specific document, it has to go through all of the entire documents. That's the first goal really and we're going to use embeddings for this. So there there's going to be a lot that's going to happen behind the scenes here. Uh Dockling is also going to use us in here because we are going to use the hierarchical chunker. So the hybrid a hybrid chunker hierarchal chunker we're going to use that to respect the documents. Now also our analyst came back and it pretty much said look the information is factually correct but there are some duplicate columns and some tables are a little bit weird formatted for me. That's okay for now. I don't want to spend more time on that, but uh it pretty much said, look, if if it's if that information is going to be analyzed by an LLM, will it still be faith faithful? Um, and mostly for like a rag pipeline, it will still be okay. But just know that we may uh in the real world need to spend some more time there. So now I'm going to create Yeah, we're going to create a plan over here. And pretty much we're going to take phase 4 and we're going to continue. And I'm going to say, okay, can we now continue with phase 4 and set up the ingestion pipeline to create the factor embeddings and input the document chunks. So what we need to do here is we need to use our local markdown files and from dockling we're going to use the hybrid uh chunker and the hierarchical chunker in order to set things up. So I want you to look into the to the documentation from dockling and figure out the right um the right way to set this up and then given our embedding model that we're going to use make sure you respect that length as well and set up the whole script to chunk it. Then create the vector embeddings and then put it into uh into superbase into the right table as well. Don't run anything just yet but first just create a plan on how we need to set everything up and what are the different steps before we can actually run everything. Also when we do the embeddings this is a expensive process. So we first want to test it with one chunk upload it to the database so we make sure everything works before running everything. Okay. So that's then the a prompt that I would give to start planning out this this section over here. Okay. And now it's going cursor is coming back with some questions for me. So this is good to implement because when we're working with dockling first once it kind of like reads it in we exported it to markdown. But under the hood it works better if we uh have it in a dockling document which is the temporary file before it exports to markdown. So what we can do uh what we can say right now is let's just indeed take the HTML HTML files again create dockling files from that. So we preserve the page numbers etc and then turn them into uh and then in turn them into the chunks. So that is okay. We'll leave that. Okay. So now it came back with the plan. So let's go through this. So we have the documents. Yes. Database is ready. Key decisions. We're going to use the hybrid uh chunker because it already includes that. Yes. Okay. Okay. Okay. Model input. That all looks good. Where are we going to put this? We're going to put it in the ingest folder outside of the app. That also makes total sense. Um let's see. Ingest. Embeddings.py. Probably better to like Okay, let's see where it puts that. We may need to move things around a little bit, but let's just first start building this. Now, this is going to be quite the pipeline that's going to be uh going to be put together. All right, so the agent is making good progress. So, here you can see all of the to-dos that it has checked off. So, we first have the we have a chunking strategy over here. We have a simple file to create our embeddings and we have the file that pretty much brings it together. So if we look at the plan, it first set everything up and then it quickly did a a test with a single file, a single chunk in order to see if everything works. And we've now continued with doing pretty much uh everything. So that's going to take a little bit. So, also keep in mind that when you're using your OpenAI API key, this is going to um incure some costs. It's it's not going to be much. I can maybe check in a bit to see what kind of like the usage really is on this. Let me actually see if I can pull up the usage data for you. I'm not sure how long it takes for this to kind of like update. Seems to be a little bit of a lag because it says 11 cents spent right now. I don't see much yet, but we do have all of our documents in here. So, if you now check, we got our chunks. So, this is what we we want to see, right? We have a document ID, we have an index, page number, section, then we have the text in there. Then, we have the actual vector embedding. So, this is the this is the factor that you can see over here, token count, and let's see the search factor that we're later going to use and the text. Now, I can already see over here that I I don't really like this. So, there is table data in here, but it has like very little context. So, I can already tell like for like pure semantic search, this is this is not going to be that good. So, like optimizing that whole process of how you get your chunks in the database, again, it's one of those things where you can spend an entire week, an entire month sometimes per project. So we just have to kind of like bear with that for now and set up the whole pipeline and later also potentially just explore some agentic search over the markdown files. But at least the pipeline works, right? So we got the factor embeddings in here. What's nice about the hierarchical um chunker that we can use from let's actually see if this um how this is explained because the thing is it considers headers within a document. So instead of doing a very naive split on saying I'm just going to split every I don't know 500 tokens it does the following. It uses a hybrid approach. So it applies tokenizationaw aware refinements on top of the document based hierarchical chunking. So that's why this is a good solution out of the box. It's way better than just naive chunking. But with tables like this, it can still get a little bit a little bit messy. Okay, let's come back to our process and uh see what we have done. Uh, okay. So, I think we can actually check off phase four and start actually building a retrieval pipeline. So, let's quickly do a quick check in our to-dos and see if we have checked off or can check off all of the boxes in phase four. And if so, please do because I think we have everything going which brings us to step number five. So this is where we're actually going to start building the retrieval pipeline and um create create everything. So let's see what do we have in here. We do a reciprocal rank fusion. Yeah. Okay. And we're going to set all of this up. Yeah. Okay. So now what I'm going to do is I am going to actually link to a previous project that I did from hybrid retrieval. Now, this is not the exact same implementation because we're going to use PG factor in here and um Postgress full text search instead of BM25. But I want to kind of like model it off this. This is uh it's a public repository. So now what I'm going to do is I am first uh let's see are we completed? Yes, completed. Okay. So, now reset context. Uh, go back to Composer 2.5 fast. And I'm going to link this in. And let's tag the to-dos and do the following. Can you check out our to-dos and then read phase five and understand what we need to do here? We need to set up the retrieval pipeline. Below I will link a repository where I've implemented hybrid search before. Now this is a little bit of a different situation because we use Postgress we use PG factor and we're going to use Postgress full textures search and not BM25 but I want you to model the patterns that you see in that hybrid search repository and specifically also how we use Pentic AI in order to bring everything together with that in mind can you create the plan to implement phase 5 retrieval right so that's then oh wait wait wait wait that's the uh let's stop stop we need a plan. We need a plan first. So that's how I would set that up and start working with that. And while we're on this and kind of like going through this, I have used and still use like a lot of the tools like cloud code, codeex, cursor right now. If you ask me about kind of like my favorite favorite setup to do like really complex heavy work, I would say right now codeex with GPT 5.5 for like familiar pro uh projects uh like this cursor and the planning uh feature in cursor and also like using some of the fast models. This is the way I am most productive. Meaning I like move the quickest. These like bigger models with like high reasoning and these agent harnesses usually it just like takes a lot of it takes a long time for them to like create the plans are very thorough but you end up waiting a lot and with that usually what you do is you start to kind of like work on a lot of stuff simultaneously but then you cannot like fully keep track of everything anymore. So like this is also for me like the sweet spot between moving quickly. Uh these are already like really big chunks. But since I'm referencing familiar stuff and using an architecture that I've kind of like designed beforehand and this is currently all dummy data, I'm okay with with moving fast because there's not much to mess up, right? If we if we delete the entire database like I don't care. I just created it. We just we just reenable it, right? Okay. So here we get a question. So the hybrid reval cookbook adds co here as a cross encoder after re-ranker. True. Do we want that? Let's keep it simple for now and only do do this. Continue. We can later potentially add a uh another reranker in there, but for now I want to keep things simple because otherwise you also would need to get an additional API key. So this is uh this is easier. Okay. So now we got the plan and Okay. Okay. Okay. So now cookbook. Yeah. Yeah. Yeah. Yeah. Module layout. This is important. We're going to create. Yeah. The retriever. Okay. Look. All looks good. Small modules. Retrieval models. Yeah. So, okay. This is going to use pidentic models. Okay. Looks good. Query embedding. Yeah, we need that. SQL queries. Okay. This is for the full text search probably. Yeah. Okay. Yeah, that's easy. Okay, looks good. And like don't get me wrong, this is like this is a lot what we're doing over here. But since I'm referencing the um this hybrid retrieval, which is a tutorial I did like 3 weeks ago, I kind of like know what good look good looks like and I'm okay to pretty much the only thing that was really interesting that's going to be different is in this tutorial I used uh BM25 which is when you implement hybrid retrieval and you essentially you you implement retrieval via vector embeddings and then also via uh text search keyword based search. BM25 is just a like a nice algorithm, a really good algorithm to do that. Postgress out of the box doesn't really have a native implementation for that, but it does have a full text search. It's a little bit different, but it can still search uh and match keywords. So, that's what it explained over here. So, we're going to do semantic search, and then here's what the SQL query for that looks like. and um building on top of this and this is going to replace the BM25. So this is pretty much how we how we query how we query that. So now this is going to let's see it's going to work on that. This is going to take some time and now probably if we go into retrieval yeah this is where we're going to see our retrieval module come up. So let's give this oh it's already done. So like look this is super cool. So now how okay let's see how do we actually test this we keep all because this is the part where we really need to make sure that everything works correctly so is uh let's see okay perfect this is already checked off so why while we are going at this context is still reasonably okay let's actually I'm going to summarize this just for now because these really go hand in hand so I'm okay with continuing in um in the next threat and then let's see this is the grounding part. So ground answers. Yeah, this is where we actually go. This is all still backand stuff. Is that correct? Just checking. Then next we're going to while So this is summarized. Okay. Can you continue from what we just implemented in phase five while I'm going to test and experiment with that? Can you create a plan to implement phase six? And then we'll put that into plan mode. And okay, so what have we created? um like a smoke. It created a smoke retrieval in scripts. Okay, let's actually see what this looks like. Let's see this. Well, this keeps on keeps on running. So, what do we have? We have a couple of queries. We keep all the changes and we have the retriever. Okay, so this is where let's get rid of this where the interactive session is going to be useful again. So let's see if we have all the imports correctly. Okay. So we're going to check let's say a user would ask the following. So this is the first question that we had in our like example questions right about Apple's uh 10ks about the revenue mix and we have the search filter. So we put a particular filter on the uh Apple documents and then the particular form and then we have our document retriever and then let's see this is just going to check what it already imple okay so this is I think there's nothing this is just retrieval so there's no llm hit just just yet to uh where are we to actually synthesize the answer this is just checking okay what do we get back What do we get back from the model? So let's actually take this. So we have our retriever which is implemented in retriever.py. Here you can see all of the filters topk. So this is I believe it out of the box created the hybrid retriever because there's like there's a lot of stuff going on behind the scenes here already. Does it import does it use Yeah, it okay. Yeah, it uses fusion already. So this is this is the this is the full thing. Okay. So now for query comma filters in here we're going to retrieve everything. So let's just first see if that like if that works at all. There we go. Okay. So we're retrieving the documents. Okay. So this is always the point where let's see if we scroll up. We really need to check and validate whether this actually makes sense. And those tables, they they don't really help a lot here because it's really hard to to look at this. So, one thing that I one technique that I already have in mind that would really greatly benefit this whole pipeline is creating a little bit of a summary to every chunk so it has a little bit of context. So, that's probably something we can do later, but at least the search works. So now what this does and again if you've never built a rack pipeline I this is way too abstract and you need to go back to one of my earlier tutorials in order to better understand this but what what right now what I'm interested in is just does the pipeline work. Can we actually get documents back? Uh I'm not as much interested in the quality just yet because I know that's that's an optimization game that we that we need to play later. So for now just getting all of these uh results back. We filter for top top KS5 and then down in here we since we have a hybrid retrieval pattern we first get way more results and [clears throat] then we actually boil it down. So what I what I now also want to do is in here let's see uh what is this is seems like it did not create the actual oh here it is here is the actual plan. Yeah, what I also want to do is in the retrieval folder, can you create a quick readme file where you explain all of the default settings and how the pipeline works also with a mermaid diagram in there just to give you a little bit of a visual representation uh about what this does here. I saw it in the plan what kind of like setups it did for both the textbased search and the vector embeddings and how it brings together and then how it selects top K. So I want to bring that together for you. Now, if we go to step number five, this is where we're going to add the LLM agent. So, now we're going to actually create something where we take the user input, we take the retrieve documents, and we're going to give an answer. It gives a summary of what was created. Here's the patterns. And yeah, yeah, yeah. Okay. This is a lot of this is a lot of work that's going to be that's going to be done here. This is a lot of work. But again, since I know we're going to use pideantic and I'm referencing the other repo where I already set something up like this, this should all be uh should all be good when we tie it into like the orchestrator. That's already where where we prepare it to be an entry point for the front end. So let's go and start working on this while we wait for this to come together. So let's see this is done. Now we also have that readme file. So while let's see just trying to figure out yeah this is part six where it now starts to work on on this plan. Now let's open the preview. So while the agent is working on phase number six we can dive into our retrieval pipeline. Right. So hybrid search over the SEC filings chunks stored in superbase. Every query runs semantic search like using PG factor and keyword full text search in parallel. It fuses the rank list with reciprocal rank fusion and then hydrates uh the top hits with document metadata and optional neighboring chunks for context. So here's a visual representation of how we currently have it. So on the full text search, we probably need to make this a little bit more uh a little bit more clever because usually out of the box what this will do is it will take the whole user query and that is that just doesn't really work well. We probably first need to just extract actual keywords from that before we run that because this is way too naive. Uh and then this is like just a normal rack pipeline. We embed the query open AI we get all of the top candidates and then reciprocal rank fusion top five. Good. Okay. Yeah. And then this is what I wanted to check. So we have the default settings. These are all all retrieval tuning lifts in config. Okay. So that's good. So it decided to add everything here. So here you can see. So here you can also see that it's it's now working on this. So this is the step that we do over here. But this is not what I want. I actually want GPT 5.5 in there. Text embedding three small is is correct. So here you can see it's really uh making our config bigger by explicitly stating okay what do we want in here and then this is interesting to optimize the retrieval pipeline. So let's just see what we have. Retrieval top K RF 60. Okay looking good. And again there's no perfect set of numbers out of the box where I say yeah you always have to use this this greatly I I get these questions a lot like this so much depends on your data set on uh your chunking strategy on pretty much all all of the components that uh that involve your projects. But this is a good setup to begin with optimization we can we can take care of that later. Okay. So, it already ripped through things and now we can already test some stuff. So, let's see what did it create. It created the smoke. [laughter] I always find it funny when it calls it like smoke tests. That's like the most AI thing to do right now. I would just call it I don't know run assistant or something like that. But anyway, now you can see in the smoke retrieval, we just run the user queries and we would get the documents back. Here we actually bring everything together. We have the following queries and we run the the whole pipeline end to end. So we're also going to synthesize an answer using the language model. So even now checking um we can run the we can run the test over here. Run the live. Yeah. Okay. Okay. So we'll get back to the front end later. First just I want to see what this what this looks like and whether this is whether this actually runs query. Just want to see how it's can you simplify the smoke assistance.py file and instead of it being a CLI command just give parameters in the like the main function that I run it. can manly change in the file and then run the Python file without parameters. This is very much an AI thing to do, right? It adds um arguments so that I can then come in and run it here manually. But AI is really good at this because it just plugs in the parameters for you. But I can't be bothered with trying to understand it. I just want to see it in the Python file. You can see. And so now it has all of these queries. Can you give me some controls over which query? I want to run one query at a time. So extend the main function. So I can do that. Just want to simplify it a little bit because this is going to loop over all of those queries and then it's very hard to test it. Right? So let's see. Keep file. Where is it? Let's see where. Okay. So now, okay, this is not I don't like this. Can you simplify it even more? Let's just maybe create the user queries. Let's just create a dictionary and then below I just want one function where I can plug in a dictionary key and then that is the question. Just make it simple as short as possible. This is just for me to run everything. This is again is way too much and I you have all of these numbers in here. I just want to simplify things. Look get it just get rid of get gets rid of a whole bunch of code. Quick key. What? Uh okay. Okay. Still not really what I had in mind, but I'll take it. So now you see it created this dictionary and it like for like now Apple mix is the default query. So we start with that and then if we run this you can see this is now going to process everything. Let's see what we got. We got a whole bunch of errors missing credentials. We do have our open AI API key, right? Just checking. Yeah, when you run the file directly it doesn't have the API key. Can you make sure we import the settings and make sure everything is correctly set up? Because this is because we don't have the API key in here because it doesn't import our config file. So, this is something it just needs to needs to load. So, it needs to adjust that. So, pretty much what we need to do and let's see, let's get rid of this. I don't like this. Just import the settings. By importing these settings, you can see it will load all of the environment variables. So I believe if we set this up, let's just get rid of this. Does this all work? Yeah. So now we get our main function. And then in here, let's run our Apple mix. Okay. This is expected because this is because we're running it in um in a Jupiter process. So I can do two things. I can from uh nest I think. Let's see. Nest iso import. Um, no, it's import nestio. And then let's see. It's nestio.apply. And then if we do it like this now, depending on which Python version we're in, let's see if this works. This should work. Yeah, there we go. This is just something you need when you are using this interactive session. So, we can have a look at what's happening over here. uh and when you run any asynchronous functions uh which bidic AI relies on under the hood. So let's actually see model we get the query. Let's actually see this is taking long. So this is means it's either stuck in a loop. Let's see because it does may need to do a little bit of extra printing here. So it does model query and then the answer. Okay. So now we need to check is it actually stuck or is it actually taking that long? Shouldn't take that long. I think this is already something going wrong. Okay. And the good thing was that it was not stuck. Uh but it just took some time especially on the last part. So here in the file I want to do a couple of things. So first of all, I want to add a little bit of logging so we actually see this and then also I want to better understand how we set up the pipeline right now and also check if we can run some things async. So u that's pretty much what I should have should have said to the AI. Okay. Can you make sure we add some logging to the smoke assistance uh file so also when we use Jupiter we can actually see what's going on and are there optimizations where we can run some more checks in parallel so doing sequential LLM API calls uh sorry parallel instead of sequential and then also can you create one document a readme file put it into one of the modules where it makes most sense to describe how the retrieval pipeline now works and especially how the grounding works. Because the thing is it already it already went a little bit too far in my opinion like and that's also because we're moving so fast I cannot read everything. It went a little bit too far on the on the validator on the grounding already because it was a little bit kind of like overfitted on this concept of the client brief that every answer must really be validated, right? So instead of just doing the normal retrieval pipeline, we also after we get everything, we also take the answer and then we fact check it again ba on uh I believe on the chunks that are in the database. So that's what's going on over here. So I just want to get a quick document to see um if that makes sense. I mean it's a good it's a good uh setup, but it could very well be that the whole implementation is just a little bit funky. Also I want to do one more check. When we look at the full text search from the user query against the post SQL database right now the keywords that we put in there do we now just put in the whole user query or is there some processing on top of that? Because the thing is if you do a full user query there will almost never be a match. Right? Can you tell me how that works and then also give me some examples that you actually test against the database to see if the current implementation actually makes sense or whether we first need to do some keyword extraction. Okay, so then that's the prompt that would be going through my mind when I think about that. Okay, so what's running this? Okay, so now we have the progress over here. Let's see what we have. We now have timestamped logs in there, which is good. We have the agent start and then like the tools. So which tools did it we have? Let's see. We can see all of the tools that it is using. So we do search search filings reach chunks. Okay. I think it already it already implemented way more than I wanted at this point. So it's already using uh a lot of these tools. So it actually created a search agent like let let's actually see. Okay. And this is why you need to be so careful with AI, right? Cuz that that that phase number six that was definitely way too big. So let's see in the instructions. Uh let's see your document copilot product contract. Yes. Corpus tool usage. So here is actually what it's doing. So use the analysis questions. Yeah. Read the chunks. Okay. Use read the surrounding chunks only. Okay. So this is this is actually good. So it can it can expand a little bit and minimize. Okay. So this is actually already quite sophisticated but again I'm trying I'm now just like trying to figure out what it already did. Okay. So this is like this is also fun. So here look this is this is the whole pipeline that was implemented already. So the end to end flow we got the question then we create the agent and it does the search listing document retriever embatic search for full text. Okay then we get the chunks plus neighbors. Yeah. Okay. So and it crowns the answer pack. I mean this is already this is quite sophisticated. In order to fully understand this, I probably like have to like spend like at least half an hour or so to to to to check this out. But the thing is like right now there is no real need for optimizing any of this before we can actually like play with this in the front end. So also to keep the momentum going, I want to keep keep moving a little bit little bit fast over here. And then let's see over here that was a text query. to see how that is implemented. So yes, yeah, that is what we need. Yeah, that's what I thought. So can you implement a step to based on the user query extract like three to five terms that actually make sense for a keyword based search? Can you implement this into our pipeline? I'm going to plan for that. Like the good thing is like with all of this, we're we're moving with a lot of a lot of different things right now. the full entire codebase is going to be available for you, right? So you can always backtrack everything and kind of like begin with the end in mind and then reverse engineer. So let's work on this and um come back to our to-dos because I am I want to start moving things together in the front end. What about you? So okay, trust UIS. Okay, cool, cool, cool. Uh, let's see. Like now that we're moving so fast, like it's probably also it probably also wired up a whole bunch of like this is all back end. Like I'm wondering what it already did on the front end though. Okay. Uh, that's pilot readiness. Yes. Okay. Now, we're going to open up a new one. We're going over here. Can you check our to-do list and continue with phase 7 and create a plan for that? We I want to wire everything up into the front end application. So that we can actually start experimenting with this. And one important thing since we have quite a sophisticated pipeline already whenever the user asks a question I also kind of like want to see status updates for when the user is waiting so it understand what's going on. Okay, let's plan for that. And then how should we pick the three keywords from N? Uh let's see. This should be an LLM call. Yes, that we should do that. Okay, let's clean things up a little bit. Do we already see something over here? Yeah, now we can see some spent. So, look, now you can see that we're actually spending some money over here. So, this is good. So, you keep an eye on on this as you start developing with this. So, let's come over to the front end application and see this is still okay. This is still not wired up. That is really what we're going to do right now. So, one thing that um when you're working on the front end, the models from Entropic, the cloud models are generally way better at that. So, I don't expect this to be really great out of the box when when we start working on this. So, I may do an additional kind of like layering with uh with a cloth model in order to make it just a just a little bit little bit fancier. Okay, let's see. the full text search plan that is pretty much user query parallel prep. Yeah, that's what I want in there. That's exactly what I was missing. So when I looked at the retrieval pipeline, right, this is already what what I set where we go straight from user query to full text search. And I I know this because I've done this in previous projects as well. Um if you do if you take the the full user query so the like full original question and you search it with a text search against the database the like that question is not in that identical order going to be in there right so what you need to do is the following it's just like literally updating as I talk which is nice so you first extract meaningful keywords with an LLM which is just a simple prompt like I can see so it's going to do filler words not not really be happy with how it's doing all of this. It should use a library for that. And then here you can see system prompt you are uh you extract search word for post SQL. Here's what it should do. And then uh let's see is like terms list. Okay, that's great. So we keep running this. That's okay. The only thing I like I don't want this. We need to do something else with that. That's just ugly. So I am already uh let's see. Okay, that's looking great so far. However, I don't like the filler words. Is there a library that we can use in order to remove those filler words? I think we can use something from NLTK, right? Can we just create import a lean set from that or is there another library that handles this for us? So, what I can do, I can already send this out. So, now it's cute. So, this will keep on running. We can come back over here. Okay. So, for the source, showing the side, which layout do you prefer? Slide over sheet simpler works. Yeah, that's totally fine. So that means when we click on the source, we have a slide out panel from the side where we can actually see where the information was coming from. Okay, good, good, good. Uh let's see low value words. How is this doing? This is still running. Let's just keep working on that. Um let's actually come back to our assistant and try to better understand our retrieval pipeline. So I pretty much understand everything up until let's see the grounding validator. That is still a little bit of a of a black box for me. And I think we can probably better understand that if we read our instructions, right? Because this is pretty much what it should do. So here is um how the Let's see. I just like I'm I'm I'm multitasking too much because I can see already NLTK. Okay. Wait, wait, wait, wait. Um short answer. What is this? Okay. Yeah, NLTK. Yeah, it is like it is a beef it's a beefy library. We can get a stopboards out of it. Well, again, anyway, we can optimize that later if we want. For now, I'm not really happy with all of those manual words in there, but let's just let's keep it for now. Optimization later. And then we can already let the model start working on the plan to build out the UI while we go through our system prompt of our agent. Right? So what we do, we have tools and it's going to site everything. Here's the corpus and then the tool usage. So start with search filings and yeah. Okay. Prefer re chunks where you need full text for multiple chunks I Yeah. Okay. So the thing is how this is set up if you and this is this I like this. This is what you would work towards. But let's see what we're what we're doing over here. um we go to we go to piantic because this is pretty much where the magic happens. So normally if you set up a any type of rag pipeline right you um you you would get the user query would go to like your front application and via the back end you retrieve you retrieve the data and normal rack pipelines are very like pretty much any tutorial that you see on YouTube is very naive simple examples just like hey it's like here's the user query embed it get it and here is what the model should do. Um, now we extended that already with hybrid search. So, it also does some keyword-based matching off of that, but still it's very naive. It just like kind of like does it does the calculation, but then it it gathers the final list and it chucks that into the system prompt and it just and it just says, "Look, here's everything. Go answer the user query." And there's no way to kind of like have a fall back or to course correct because in case it did not find enough evidence the default loop is uh is not implemented and what it did over here which is I think it's already quite good is instead of making it naive we create or we set up a pidentic AI agent but where the looping mechanism comes in is when we give it access to those tools because with paidic out of the box when you have an agent and you tools, the agent can use those tools and it can then check what kind of like answer it gets back from that and then use tools again. So that's why it can search filings, it can read uh chunks, it can read a chunk and it can also read surrounding chunks. So this is already quite a sophisticated way to go through our uh to go really through our our data, capture everything using multiple mechanisms, bring everything together, but then also when it when it decides it needs more information, it can go back in the database and it can read surrounding chunks. That's really uh one of the clever clever things over here. So that's all implemented here in the tools. So if you see what this essentially does is if we have any type of a retrieve chunk we can have a radius around that. So you can see the retrieval uh radius is set up over here. So let's see retrieval neighbor is set to one. So it will take it will take that chunk and it will also look around it. And here we're really bringing together like the the thing is like this could be an entire like 2hour video just on this retrieval pipeline to really help you understand what's going on over here. But we're combining essentially hybrid search with agentic search and we're bringing them together. So I think if we start to experiment with it, it will take the queries will probably take 60 seconds or so. So it will take some time to get an answer but it will it will already become quite quite good because it can go through these it can go through these uh documents. So if we then look at the whole grounding process at the grounded answer you can see these are also then the data models that we're we're getting back that we're getting to. So this is uh this is cool to see. Okay this is also already completed. So now let's continue. I already get quite a little bit of a better understanding as to how our pipeline works. So now we should be able to run some queries in the browser. Let's see. Going to refresh this. Okay, that's already great start. We cannot like we may need to let's see where are we? Let's run everything one more I need to let's come over here and run our back end again. That's good. And also run our front end again. Where are we? There we go. Okay. So now we you can see we have an error over here because pretty much if I create a new chat, we get an unprocessable entity. So that's not good. So let's see. We can still do this in here because it's the pretty much the same task. We can say, "Oh, where am I? When I try to create a new message, new conversation in the front end, we get an unprocessable processible content. So, can you check the back end all the way to the front end to see if this is implemented correctly and debug?" Now, then this is typically I feel like this is the point where we might run into some of the issues of composer 2.5. I feel like if we would have used GBT 5.5, this wouldn't have happened. So, let's see how we can uh how we can solve that and how quickly we can get onto that. And then what I'm going to do what I'm going to do over here, I'm going to use Opus 4.8 over here. And it's also time to do another checkpoint. Not a checkpoint. Okay. One skill that I am going to use is front end design. So this is a skill if you don't have it already. Front end design skill. It's from Entropic. It's this skill. This combined with the models from Entropic always works really well if if we want to do anything design related. And what I want to do is I want to pretty much let it rip on just the visual components of things. And here's how I would prompt that. Uh let's first see if this works. Let's see because that would be a good checkpoint. Doesn't work. Okay, we may need to switch to a beefier model at this point cuz like I've said, normally this would just work. Now we got an error. Yeah. Yeah. Yeah. Okay. So, what I'm going to do fresh thread. I'm going to do GPT 5.5 hide. And I'm going to say some previous agents worked on connecting the back end with the front end, but I'm not sure if everything is wired up correctly. Can you do an audit and really check correctly whether the API communicates correctly with the front end and how we can create a new conversation and whether everything is set up correctly also driving into the database. That's then what I would do. And also I would pretty much whenever something like this happens like this chat is off. Boom. it's broken. Don't continue in that with kind of like try to fix it, try to fix it, try to fix it. You usually need to zoom out, clear context, and in this case also a beefier model. And then let's come back to the UI because we do have this over here. So, okay, this is okay. I I'm I'm going to yap a little bit, but this I think you're going to learn a lot from this. So, anything front end related, here's how I would do this. Um, come back over here. I'm going to use glido persistent mode. You are going to help me to create a really cool design for our front application. First, your goal should be to understand what we're creating. So, we're building an internal tool. We have the user login. Then, we have the sidebar with the conversations. User also needs to be able to log out. And we're going to use the chat CN component. So what I want you to do is I want you to look online and I want you to figure out what are the best practices around creating these chat applications right there are a lot of uh there are a lot of examples what are the elements that we need from a user experience and user interface point of view in order to make this just a pretty application that just works. Now we don't need to go fancy here with all kinds of coloring. We just simply use black and white and use the chaten components. But I want it to look proper. Like think about the side menu bar that we have over there. Like we can maybe like hide it. We can open it up. We have the user settings at the bottom. We can log out the user. Then we have the list with all of the conversation histories. And also then what does the chat interaction look like with all of the citations and the loop and while users are waiting for feedback. I want all of that in there. So first start doing your research. Look at what we have currently. Then perform kind of like a gap analysis and create a plan to implement everything for the front end to really view it through the lens of like a proper system. And also whenever possible reuse components. So try to set up some kind of like design system where we can reuse the components over and over again instead of like trying to um create all of these isolated components again and again again. That was a yap. long yap with the help of glido to easily do this for me and correct everything that I needed. There's also a little bit of formatting on top of that. Now, was there a particular like structure to all of that? Switch to play mode. Yes, switch to play mode. Not really. But there were a couple of important things that if you pay close attention that going to make all the difference. Do research. Use a good entropic model. High reasoning. mention UXUI. Use the front end design skill. Mention design system to use a design system. And I think that's it. I think those were the most important things. So now this is as you can see this is going to send off a back end uh a background job. This will continue to run for a while. So okay, what do we got over here? This is the beefier model coming into play. Yeah. So this is the now we got we we we got the um we got the big model jumping in to look at some of the changes we need to make. So let's just run this and run a couple of tests before we are done. Okay. And so while the other range is working we get some questions from the front end application. So it asks okay which chat components do you want to use? Right you already use the AI SDK plus so prompt kit is a natural fit. This is actually a nice library. So let's let's give this a spin. So you see props it gives us this chat GPTG like vibe you know so we can import all of these all of these components. So let's say okay let's do that. How strict should a black and white team be light only like yeah okay let's just shouldn't be only black and white gray is okay gray is okay. Okay let's see how is this going. So GPT 5.5 already found some issues here. So let's see while that works on everything and look you can also see like the like this the the the simpler like the smaller model um worked fine in the beginning but we are quickly kind of like ramping up in complexity and right now we're not we we don't have a simple project anymore. This is already like okay we have a project we got a lot of moving components we need to make sure everything is set up properly. So, this is really where bigger models really help unless you continue to go bite-sized, right? Also, like the thing is I also noticed like in the beginning I was like, "Oh, let's shoot this first and then check this, right?" But I also noticed that if I continue like that, this will be like a two-day video in order to do that. So I also want to find the balance between showing okay how fast can you go while using some simple methods and and what what do we need to offload to to the heavier model. So okay this is running now we are going to create the plan this is running okay okay okay let's see so the monogchrome it's called the monochrome chat design perfect call strict light monochrome neutral grays yes layout industry it's going to do cleanup appshell yeah this is all front I'm just checking to make sure it's not going to mess with anything on the back end because that's what we're working on right now okay let's kick this And this is always really fun, right? I mean, with with the back end stuff, we just tell it what to do and you kind of like expect it to do do just that. With front end here, I have no idea what it's going to come up with. So, remember, right now, our front end looks like this. So, let's see what Claude Opus 4.8 in this case is going to make out of it. while GPT 5.5 is still trying to clean up the mess I guess. Okay. And that audit is now gone and it adjusted. So it had something to do with the chat files both in the database schema and let's see this is just a test. So we needed to update something there in order to make it work. So let's see we can now create a new chat. So let's see let's create another one. Got a new chat. Now again, you can see from the UI, it's still it's a little bit clunky. I don't really like it. But let's see what actually happens if we submit this. So you can see there's already some stuff going on. And if I let's see if I open up, you can see the logs right now. So it's actually doing something. So you can see analyzing, searching, reading, verifying. So this is already like this is cool to see the process working. I would love a little bit more feedback or some kind of like loading. Yeah. Okay. It's good. The thing is because it has this tool loop already, that's why it actually takes quite some time to to go through all of this. So now it's verifying and boom, we got an error. So let's see what we got. So this is why we want to have the terminal open to see. Okay, what that's a big error message, but what went wrong over here? and it tried to get something. It's like there was something with the message citation that violates a no constraint. So it's something with the data model. Let's see. So couple of things. I mean the thing is like this already this this okay. Yeah I see this is actually the answer but there's a couple of things wrong with it. First of all it cannot format the markdown table. So that's something we that the front end should implement. But look, you you can already see we we got quite we got some cool we got some cool stuff over here. This is getting quite but it should it's centered. That's weird. And this uh and we just should mark have markdown formatting in. So let's plug this. Let's see. I thought I copied it, but I guess not. Uh so we're going to copy the error. come back in here, use Glido and say, "Okay, I just run a query and can you check what's going on over here?" Kind of like what's going on with the citation message and fix please just always add that makes it work. Okay, now our clawed opus agent is also Let's see. Looks like this is literally waiting on a user input. Just checking if it isn't somewhere. Now let's see can you continue tell me what I need to run if you get stuck I saw so you can see it tried to what's let's see so we were stuck on an import right because we were getting the prompt kit in here so you can see we do shet set shen and we get the prompt kit and it asked look avatar.tx TX already exists. Would you like to override it? And then it got stuck in the process over there. So now it's going to run that with the yes flag. So let's see if it now works because otherwise just we just need to do it manually. This is also one of those things where usually I just prefer AI to like hand over the command to me to run it because this is still ass. Um give me the command I need to run. Where are we? We just need to go let's see run this command. We need to be in the front end and then we just we just add all of those blocks. Okay, let's see. Let's create a new terminal cd into the front end and let's run that. Okay, done. Now continue. That should do it. And now also let's see what we got going on. Fix the citation error because was not null. Okay, so now we should be able to come back. I mean, yeah, this is this is ugly. I keep doing those checkpoints. Checkpoint. Okay, so now if we do new chat and we say let's do this. Let's run one more query to see if we don't get the uh we don't get the error now. And uh like the the formatting will still be a little bit uh a little bit messy because that's what our other model is uh working on right now because now the dependencies are all in. Okay. And there we go. Like there's there's still a lot that we need to optimize but there is an answer in here. It gots the the relevant chunks. I need markdown formatting in here. But this this is a little weird here. But you can see it has already set up the conversation with a name. We can sign out here. So this is starting to look like something we can use. And now it's just we need to like this is going to be a big one that runs through all of this. Okay. And while the uh front end agent continues to optimize things, let's quickly let's see. This is opening a terminal. I don't need it here. Let's quickly go one more time over the full pipeline because this is the part of the process where we went really quick and I just asked AI to write to just write it in plain English to understand what's going on behind the scenes because this also what kind of like blew my mind that it kind of like oneshoted that whole tool loop. So what you should understand we created our agent.py. So we did that using pientic. So that's what we said over here and paid AI abstracts a lot really of the heavy lifting around working with language models. So which model the chat conversations the system prompt the tool use the tool calling that's all abstracted within this agent.py Pi and when we do like the run document agent so when we run thing um it takes the instructions so this is instructions is the system prompt essentially this says okay here's how you should behave and then what happens is wait where are we open preview what happens is the agent has a has a set of tools so first of all it's it's going to run the search filings so this is the retrieval pipeline that we set up, right? The hybrid retrieval. This is it's going to do the embeddings. It's going to do the full text search. Uh it ranks them. It takes top K fuses them. That's that's how all of that is set up. Now, you can see uh it includes nearby neighbors chunks by default. So, this was already a design choice that was added in there. We set nearby neighbors to one. So it already every chunk that is retrieved we add a neighbor around it as well to give it a little bit more context. The agent then decides whether the returned uh excerpt excerpts are enough to answer. So that is also something that within the instructions is decide and this is really um nice about having the tool loop so that it it's not this naive path that I explained right where it's like here are the documents go answer and if you don't know it simply say that you don't know it like that's naive and it will for the user it it can result in in weird things now there's a decision-making step where the model actually decides okay these are all the retrieved this is all retrieved information can I synthesize an answer from this and if it's the case we force it to create a grounded answer using again this is structured output the p identic model if not it can use the read chunks and the read chunk tools in order to go back into the database manually and do a gentic search around the queries for example or the chunks that we already retrieved. So then once we have once we have that final answer, we're going to run the validator. So this is an additional like guardrail that we put on top of on top of here that is implemented here where it's going to again check one more time. Look, do do we have enough information? And it's going to give a hard yes or no pass. And only if here it decides that it doesn't have enough information, it's going to explain or tell that to the user which is explained over here. So uh after the agent returns validator, compares the structured answer against the strategy, it tracks the answer citation contract and verifies every kind of like citation that we get um and rejects chunks ideas that were never retrieved and requires each except to be copied from the register chunk. Okay, so that whole part this whole validator then ultimately says okay this is good to go. We can give our answer or we say no and it will pretty much tell the user look we could not find an answer that we could grant any information. Now again any of these little building blocks that you see over here you could probably spend a day or two optimizing all of those. That is what I want to keep stressing over here. Even though this is like a speedrun project, right? This is not how it goes in the real world. You need to set it up first and then make sure to go let's see to go back and keep iterating on it. So let's see where we are with the front end will resume when background shell exit. What's going on over here? Check environment variables where we are. Okay, this is already looking pretty cool. I'm not sure if it's done already. I do see a couple of weird quirks, but let's see if we create a new chat. Okay, I like this. I like this. I don't like this. Okay. Okay. Okay. Already going to create a screenshot. I'm going to say I don't like that. I don't like that. And I don't like this. And I also don't like these loading dots. So, this is a tool. By the way, this is clean. I use this on Mac. This is really helpful because what I can do now. So I can come in here and I just want to check if it did. Yeah. Okay. I think this is done, but it's just somehow stuck in this loop. So what I'm going to do is I'm just going to come back over here, start a new session, and I'm going to um so one agent just worked on the front end application. And there are a couple of things that I would like to adjust. I attached a screenshot with numbers. First of all, if you look at uh number one, there is a little glitch in there. There's a divider. Get rid of that. Then number two, we got this like default AI sparkle thingy. Skip that. I don't want that. We don't need anything there. Just remove it. Okay. Then number three is the whole cycle of going between or through analyzing, searching, reading. I don't like how that looks. Can you just like create on the kind of like show a loading state in text that kind of like has a gradient going on over it. So a gradient wipe of color of gray and then simply as you go first show analyzing and then show searching. So we don't show the whole thing but we simply replace it. And then also number four it shows these three dots. I don't know what that is. So can you improve that as well? It's pretty much the whole interaction around like what happens when a user is waiting for a question. That is something that I want you to optimize right now. Boom. And don't you just love how fast Glido is? I mean, you should really try it out. Okay, let's just work on this now and then come back and check if we actually So, this is okay. We got we got we got the table renderer in here. That is good. So, now we get we need to get rid of this. This This is good. Yeah. So now I want another So now pretty much where we are arriving, we we we are making great progress. And before I really go into optimizing this thing, I want to go back to our to-do lists pretty much and see okay where are we at? So phase eight before we actually are going to deploy this the pilot readiness. What do we need? We have running locally documents 10 examples confirm like the scale structured logging. This is more so like a test. I'm actually I think I'm going to like let an agent run on this and then in parallel I'm going to work a little bit on UI refinements because we're like we're pretty much getting there. And I want to get this out to the deployment quickly because that's the most interesting learning part. And then I want to wrap up this video to show you the iteration loop of how to improve this. So I'm going to do the following. And since this is kind of like review stuff, I am going to do the following. We have entered phase 8 of this project. Um phase 8 is pretty much doing a bunch of checks and like not everything is directly relevant right now. Can you just do a review analysis an analysis of the codebase and pretty much see if we are ready for phase a to cross everything off and just make sure that everything works everything runs run a couple of tests and see there if there's anything we missed based on the client brief so far like all like user feedback and whether it actually saves some time I will take care of that like don't worry about it and I'm not going to take care of it [laughter] because I'm not going to test it within the scope of this video but let's run this and then slowly start to prep for the uh so this is working on something. I'm going to create one more agent ramping things up because what I uh yeah what I essentially had right we had this yeah when you there are a couple of things that happen when you click on a citation and I want you to improve this. So first of all um I've noticed that when you click on the inline citations it opens a new browser tab. So, can you make sure that an clicking an inline citation also opens the right citation on the right bar? And then second, can we format the can we format the uh original sources or the chunks that have been retrieved? Can we format that in a markdown table? And also, can we also show the neighboring chunks around it to make sure we give a little bit more context because now it's it feels a little bit out of nowhere. If you just show the table, what would that look like? It could be that there is an issue with the markdown tables because it's not strictly markdown. If that is the case, can you tell me how we could potentially solve that? Okay, so what I referred to was first of all the following. If I click on this, it kind of like opens up a new tab. something I noticed and I just wanted if you click on two it should open up open up this and then also like this is of course very messy and I think this issue has to do all what we noticed all the way at the beginning right where we the do uh dog link had a hard time extracting the PDF sorry the the markdown tables right from the HTML file so that's why that may be a little bit miss messy so I'm not sure how that works and I want a little bit of context around this. I want what was before and after so I can actually see this? I want to get rid of this. But other than that, I mean, don't get me wrong, we're actually we're literally building chatpt over here. So, this is pretty cool, right? So, we may see the occasional glitches over here. So, I can also sign out. So, look, this is also it's cool, right? We got we got we got a cool app, Drift. Oh, okay. Yeah, this is this is what happens when you start messing with things as an AI agent is working on it. So, we may just need to be a little bit careful here. I'm actually going to let this run and then come back. And while we're at it, just like like let's put the cherry on top. Go to flat flat eye. Can I search for wood? I think this I think this would work really well. So, let's see. Let's get the We do the free download over here. Let's get that over here. And then let's get the log and put that into our project. So let's see where is my project. Let's put it into public over here. And now let's say uh [sighs] okay we can go we can go back to composer fast. Uh let's take the log. Can you take the icon that lives on the top of the menu bar and then use a black background but then use this log as the icon and also make sure you update the faficon with this as well. So we actually create uh an avatar around this and then this becomes the main logo for our application. Okay, that was the wrong choice. Or maybe just need some visual feedback. I don't like the rounded corners here and uh let's see. I'm actually going to take this and I'm going to say one, two, take this. Come back. Okay. Please look at the image attached. You can see we need some padding. So there are two places of course. First of all, um, make sure that we have some padding. So it sits nicely in the center and there's enough negative space. Bottom, center, front, right. And also on the second image that you see where that I labeled with two, you can see the corners are too rounded there. So please unify that with what we see in the menu bar. Going back to GPT 5.5 in order to try and solve that and see if we can make that a little bit little bit cooler. Do we also have our our favicon? Usually like it like browser cache can um can prevent that from loading. So, I guess we'll have to check that later. But this is already looking pretty cool. Uh, that is looking much better. That looks like the logo of a very professional investment analysis firm, [laughter] right? I mean, no, like it doesn't match, but I like it for now. Meanwhile, I also implemented the function that we can actually delete conversations. Looks it looks a little bit clunky, a little bit AI generated, but for now, I think it's good. If we come in here and we actually look at the source documents, this is already starting to look way better as well. We may like still need to do a little bit of cleanup here because there's definitely something with those tables that is not 100% correct, but we are getting there. Okay. Also, does it work if we click on this? Yeah. So, you can now see. Okay. Perfect. Okay. Now, this is starting to look really good, and I actually want to move to the deployment side of things. So, we can also start to kind of like move towards the end of this video. There's one thing that I don't like. Uh, and that is still the issue with the table. So, again, this is probably a bigger problem to solve, but what I'm going to do is I'm going to let one research agent go at it to see if we can fix it using the easy button. So, I'm coming back. I'm going to use GPT 5.5 high. You know what? Let's let's make it extra high. Let's spend some time here. But then if we do that, we also need a good prompt. So, um let's do that. Okay. So, I need your help with one thing. So, we got quite a nice pipeline going on over here. We can take all of the HTTM files. Then, we use Dockling to extract that and then we chunk it using Dockling and we put it into the database. Now there's one issue that the documents that we are working with contain a whole bunch of tables and I feel like the current extraction method is a little bit fragile for what we are trying to accomplish because if I look at the citations over here you can see it's just a little bit messy and since this is so critical to this project I feel like we need a better way to extract the tables and the thing is since we are the original files are HTML files. We can probably extract them from the HTM files with a dedicated surface for that. Now, what I want you to do is I want you to look at the current problem. Look at the input HTM files and then look at the output markdown files and see where the issues are coming from when it comes to formatting these tables. And then we should you should check and search online and validate whether we can fix this using dockling or whether we need a better approach to do this because I could also see a version because it's ATM htm where we write a custom extractor specifically for these types of documents. So we parse it and we essentially convert it to markdown and we have a special converter for the tables. So I want you to understand the whole problem and give me your plan for what's going to result in the ultimate best and most accurate parsing of this data and also later down the line chunking and putting into a vector database. So you should consider the whole pipeline. Bam. Okay, that was a big one. E now I again should put this on planning. There we go. Okay, so this is going to uh this is going to run. This is doing something. And now let's go to the deployment side of things. So, as I've said, I have never used Railway. I literally just created an account. So, you can see I have 30 days, $5 left. So, the whole goal is that we can set this up for free. Now, what we want to do is we have a little bit of a um special setup because we have both the front end and the back end in one GitHub repository. So um what I did is I asked AI to give me a guide for railway on how we should set up this uh this project. So let's actually go through it. We have this and it did a couple of things. So first of all, it created docker ignore files and it also created the docker file for the front end because there's I think there's no way around it because we really um need to set up the project properly in order to be able to deploy this. So um let's see how this goes. Usually for all of our projects we have if you look at for example our geni launchpad and uh uh like the the project starter pack that we use for all of our project it already comes wired with docker files because that's how we always like to set up our project. So that's why this is for me a little bit like figuring out what the right way to do is and I guess it's also just going to be trial and error with this because it first lets me um ask like what is it going to so we check how good it actually is because let me actually see if I can walk through it. So I'm going to let's first kind of like update this uh okay and we [clears throat] actually publish that branch. So the development branch and now I am going to say let's I don't know let's just let's just see how good it is railway let's put it to the test. I have a mono repository with both a front end and a back end and I want to deploy both. The back end communicates with the front end. The back end is a Python application using fast API and the frontend application is a plain react app built in Typescript that we run via fight. Let's make sure we spell that correctly and let's see what what realway will come up with. I probably should have mentioned that we're also using superbase externally already. So we just need to connect with that. Okay. Okay, so when I try to set up the connection with GitHub, it asks me to authorize via GitHub. And here is a tip that I can give you. Whenever you are testing out these new tools, I'm usually not a big fan of these when, like I've said, when we do production deployments, we use Azure, we use Hatsner, whatever, and we do everything manually because what I don't like about these tools is that you essentially give them access. And out of the box, usually it's all repositories. And that means if someone compromises this whole system on railway and it can somehow access and be and act on your behalf, it has access to all of your GitHub repositories. So whenever you do use stuff like this, make sure that you scope it off. So instead of all repositories, I say look only select uh selected repository. So I'm only going to give the document copilot access. Okay. So once I go through that loop and you may need to refresh your project, it took a little bit for it to recognize. I can now select my project over here. So now this is all new to me. So let's see what we want. So I'm going to deploy from development which usually is not something you want to do. And now we just need to see how we can set up the Yeah, this is actually I may because I I always hate kind of like the setups like this. I may even see if we can use the railway CLI because this can get really tricky in order to get it right. So, what I probably want to do, let's see. Can we Yeah, I'm going I'm going to install this via Brew because the thing is what otherwise we just have to click through all of these and set everything up and uh do the startup command and then we need to do that for the the front end as well. Whereas, if we can use our agent to pretty much like look into the CLI, it's it like it's going to be way faster and you're also going to be able to check uh way more settings. So, what I'm going to do is I'm going to open up a terminal and let's do a new one. This is our back end running. I'm going to do brew install railway. So, if you're on Windows, you can use the following command over here. And let's see. Then we need to just authenticate that which we can do after the installation. So let's clear this up. Let's log in. Open the browser. Now let's see. Let's like for now this is the only workspace I have. So for now that's totally fine to manage that. So now the authentication is successful. So now I should be able to have access in here. And then what I probably want to do is check railway MCP is do they have an MCP server? I want to get that for cursor. And I'm simply going to check let's see first of all it's going to get that out of here. And I'm going to do cursor settings. I'm going to see if we can let's see MCP if I can search for this. Let's see manage view strap. Uh okay. So we can actually do it like this. We get MCP servers and we just need railway like this and then probably I don't we it could be that we don't I it likely we don't even need the uh CLI if we use MCP. So, can you access the railway MCP? Cuz we may need to authenticate that one cuz if we now go back in here. Yeah. You see, we connect. Okay. So, I don't Yeah, different setup. This is always a little bit confusing, but we don't if we use the MCP from cursor, we don't really need the uh the CLI. So now it should like let's see let's ask that one more time. Can you access the railway MCP? Beautifully typed. That's what you get when you use glide all the time. Like you you stop learn you you stop typing properly. Okay. So now because now what we can do is first of all we can check okay that worked on that one. We got a plan for let's see for the this is the table extraction. So this is a big one on the side. Not sure if I already want to run this. The current pipeline has the right highle shape but the wrong table representation. So recommendation use a hybrid to keep dockling for narrative sections headings and basic but add a SCP specific HTML table extractor. Okay, like this is going to make it better. Um, this is probably going to require us to run the document pipeline again. So, that is a little bit of a of a risk there, but I know it's going to improve the outcome. And I want to show you how to do this. So, I'm just going to hit build on this one while we Yes. Okay. So, now here's what we can now do. Uh, which is going to be the cool thing. Now that we have our railway MCP server and our deployment guide, we can ask our AI to help us with this and we don't have to tinker through menus. Okay. Can you now take the steering wheel and set up our project in Railway using the MCP server? First confirm if you have access to it. Just want to check because this was an older file uh chat message. Sometimes it can uh sometimes old chats need to be kind of like closed and opened again before it can access all of the MCPS. But let's just see if this one has access to it. Yes. Okay. Checking the read only project schemas. Yeah. Okay. So now and the thing is it created Yeah. It created this kind of like dummy project. But I probably just want the whole MCPS to create uh the entire project uh without this kind of like default name in here. Okay. Yes, I have everything authenticated, registered uh one existing project. Before I make I need to go ahead. Okay, it. Let's use cheerful generosity. That's the project we want to use. Um does it make sense? It doesn't make sense. Let's call it document. Let's do the actual document pilot because that's way better. Let's create a new project uh call it document pilot and um let's just use that and can you deeply search into the railway documentation as well as the structure of our codebase and make sure everything is in place and then create one plan in order to deploy everything. Also let me know if there's anything else you need from me whether that's like providing credentials. We have everything in the repository. So feel free to also use the CLI in order to um input the variables on railway. Okay. Now I do want to give a note here on the deployment side of things. I'm going pretty yolo mode here. MCP server giving access to credentials. This is not something that I would do on a client project. Again we have our own Docker files. We have HNER or Azure where we deploy. We have our secret vaults where we put the API keys in here. So they don't even live in the project. So there is still a gap between what I'm showing here and really what you would call production ready and how AI engineering works works in the real world. But this is just a fine balance between seeing things end to end while still being able to byte-size it into one video. Right? So that's the constant kind of like thing that you have to keep keep in mind over here. So I'm going to let this run and kind of like come up with the plan and then also run our So this is also our big um our big extractor that's going to work on extracting the HTML tables that's going to improve our application. I'll get back to you once all of this is done. Okay. So, it came back with the plan for deployment and I am simply going to hit build on that. Right now, we're going to use the railway MCP server to go through this. So, one more time over here to the dashboard and it should create a new project. So, let's see what this is going to do. And while we're at this, I am going to show you kind of like one thing that is at day luminina we have one project or course really I should say which is called the genai accelerator. So this is our flagship technical program where in six weeks I teach you AI engineering really from first principles and everything that we do on our client projects. So, like I've said, what you're seeing is in this video is a speedrun. Like, this video will be four 4 and a half hours, I think, something like that. Maybe I trim it down a little bit or we still run into something and it takes 5 hours. But you can see on the Genai Accelerator, it is split. It's not split up over like four hours. It's split up over literally six weeks of content. And there is, I believe, over 60 hours or so um for everything that there is to it. So instead of just fasttracking everything and hitting the easy button on AI and let's just AI figure everything out in this course I teach you everything really step by step. We'll break down we'll we'll look into the exact steps into the prompts into the loops into the tool calling into the design uh patterns and principles that we're using into like how to use Docker properly all the way to also deploying your AI applications but doing it with a really a a production rigger right because again this is going to be fun I'm really excited to see what this is going to end up with and it's easy for you to follow along but This is just not something that you do in the real world, right? Your all CI/CD pipeline, your deployments, that is just not something you give to an MCP and and let it rip. Um, I am only okay with doing this because I know first of all that I'm going to nuke the database and all my API keys after this video. So, that's one thing. And also, all of the data that we're using is uh public data. So there's nothing in this project that I uh want to hype like literally nothing. So given that I am okay with this. So you should approach this in a similar fashion even if you're using your own API credentials like just chugging that in there and letting the AI set everything up. if you really want to be careful like just like remove your credentials or refresh them after you've worked with AI on this. So that is just something just to to keep in mind that I can't stress enough because there is still a big difference between what AI engineering looks looks like in the real world and really what we're doing over here. So if you are enjoying this type of work really and these types of projects and you really want to do AI engineering seriously not just for I don't know a hobby project but this is in your career to advance your career or you want to do this as a freelancer you can check out the accelerator. I will leave a link in the description for you to check out. Now that we're on that if you feel like you really are already kind of like capable enough to start freelancing I have another project for you. So, next to everything that I do with data luminina and our agency, we run two courses pretty much in coaching programs. It's the accelerator and this is data freelancer where we pretty much teach you how to start as a freelancer and start landing freelance jobs like on top of your tech job or if you're unemployed right now like it could be it could become your full-time thing. But the it is essentially designed for people who currently have a full-time job. So, you can do this next to your job get start. So, that is also something you can check out if you're interested in that and you've been liking the content so far. A little bit of a plug. Now, let's go back to let's see our document copilot. I love how this is just showing up right now. So, we can see our back end coming together and we can see our front end coming together. So, there's there's something happening. There is something happening. Cool. Yeah. So as you know so when we are deploying to railway it's of course going to use internal uh internal but publicly facing domains right so you can see you have um I don't know where this up is coming from up could be that is that is railway internally so it uses up railway.app but the document copilot backend production that is now what we are um and it could even be that because this is the production yeah yeah yeah okay so it's production.up.railway.app app. So this now becomes an actual deployment because similar to what we what we were running locally, right? Um we were running on uh what was it port 8, I believe it's port 8,000 that we're running on. So this is now you can also see if you do docs. Oh, because fast API exposes this by default which is something a lot of people don't know. So now you can see look this is now our API documentation but this is local host right. So what you can see right now if we deploy our back end and we don't kind of like protect this via some kind of um some kind of um firewall or like e pay restriction this becomes public and once this is deployed people can also literally just view your uh docs and they can see your endpoints. Now, for an internal tool, this is something you don't really want, right? So, the just all there's there's a lot of extra additional stuff that you need to keep in mind when you're going to throw uh actual client data at it. So, let's see if this this is actually already doing something. Could even be that like because like the thing is it could even be that because I'm doing it via the MCP that it's doing that it's setting this up properly. But let's see if um it does that. So it creates the back end. Let's see. The back end is public domain. Uh yeah. Okay. Still in progress. Well, we need a little bit more time here. So you can see the build now. Interesting. And pretty much for if this is the first time you're working with the deploy, pretty much all we're doing right now is we're putting this code, this backend folder on a server. And once the code is there, we run our startup command which is this. So the code lives on a server which is pretty much just a computer that's always on instead of doing it locally. We run the startup command. So it actually exposes and then um what railway so railway then has this way of exposing it via a public uh domain, right? So that is usually something that you have to set up yourself. But this this is where just railway makes it easy. So we get that uh domain that we uh that we just saw. Okay. Now little update progress on this because this can always be really annoying when you are using anything other really than uh a virtual machine where you can directly deploy and just run the exact same commands as you do on locally. You get into errors and these platforms always can have a tendency to just hide a whole bunch of the information that you need to see. So we can now here we have a sneak peek into the logs and see what it was doing but it essentially failed. So this process is also going to do look different for you. So this is the thing is I the final repository will be final right for you. So everything is there and probably I can also create some type of deployment script in there uh depending on how railway and what railway supports if I can put some type of YAML file in there. Um, but this could also be the part where you have to do some extra tinkering in order to get this up correctly. Um, and that's also where you learn, right? So, I'm just going to check like the MCP is still going at it. So, it's it's going to check the logs and it probably will continue in a uh loop to continue with this. But yeah, this is not like the prettiest example or demo really to walk you through because this is kind of like new. But let's just fast forward to when we fix this and then I'll tell you how to properly set it up and what you can do to avoid this. Okay, and we're back. It's also the next day because I had to go out for dinner. But look, we got our back end and we got our front end deployed. So, there were a couple of changes that we had to make during the deployment process. So, luckily for you, everything is going to be in the repository. So as you kind of like go through this and have the the the final result next to it, you have a little cheat code, right? You can look into it to avoid messing with that. So pretty much what it came down to, you can see on our work tree, we have a whole bunch of diffs because there was also one agent working on the tables. Remember to optimize that. But pretty much uh what was required for railway was a docker setup both on the uh on the back end. So this is the docker file in order to set that up and also on the front end also the docker file and even a caddy file to expose the uh endpoints and essentially make this available via HTTP. So again if you if you've never worked with this this is quite abstract here for now. I can't really go into what all of this means. The best thing always to do is of course like just use AI. Literally use AI and ask look what is this Getty file? what is it used for? That's ultimately the best way to learn. So, right now, um I'm going to show you how we're how we can wrap this up and how you can essentially work with this yourself and also continue to improve this. So, a couple of things. So, first the uh the agent that I asked to improve the table structure did its job. Look at this. Now, we got actual proper tables. So this is this is really cool. This is actually it's coming together quite nicely. So in our database we now also have a document tables where you can see we have the document ID the table index and then you can see the actual markdown and [clears throat] it extracted to actual JSON data because um remember right when we were working with the uh the documents let's let's pick one of those let's see open in the browser one of those files right because they're eight files they are structured all the tables they have a structured format. So it's better to just rely on HTML parsing in order to get this out. So that's what I asked uh GPD5 in this case to set up. And you'll find under the ingest that there is a load source documents. So and then also the section tables. So these are helper functions to extract the tables correctly. So in the process I had to redo essentially all of the chunks for the um to extract the tables properly. Right? So if we now have the source documents and we have the document chunks that is now correctly uh referencing the documents table I believe. And we still have to like if I look into this this is still a little bit ugly. I do have to check if I may still have to clean up some uh some records in here. So that could that could be the case that we need to do some cleaning. But if we now come over to our um to our application again, right? So we can ask a question, we get the answer back, we get a beautifully rendered markdown table and we can even look at the sources and it will load exactly where it's coming from also the neighboring uh citation or the neighboring chunks. So this is for me already like a perfect kind of like end to- end example of what a solution like this could look like. So here I also want to wrap up. Of course there's tons of more stuff that we can do to improve this to make it faster to make it cheaper to make it more reliable. Uh but that is a whole process on its own. So what I want to you to show right now is uh give you a little bit more insights into the deployment process on railway. Right? So I'm going to first create a checkpoint over here. Let's commit that. Uh let's also just sync that in there. So in the guides there is a railway deployment guide in here and I updated this based on what went wrong during our deployment. And also by the way I remember one issue that we ran into as well was that um the agent at some point added the dockling in the main dependencies and dockling can is a quite an extensive library. So if you try and install that on uh let's say a compute that that has like low RAM or has doesn't have an a bit large enough or enough CPUs it can error on that. So it's really important to keep that in the um defaf dependencies in the optional dependencies here. So there is that and okay now let's come back to what is what is going on over here. So I use the MCP server in order to help me walk through this. But what I want to check right now is that first of all I want to check if we can actually go over here. Right. So if we now uh so this is our back end. So that is expected. I need to go to our front end application because now you can see uh instead of just doing it at local host you can see over here at the top right we can actually go to document- copilot- front end production up railway. So if this is now a public URL like after this I will take it down but this is now um you like at this moment of recording you could go to this website and then if you um would use my credentials you could log in to this. So that's also something that I want you to be aware of as you start to use and start to deploy this. So right now we have a very simple just username password login right. So probably especially if you want to do want to deploy this for a client make sure that's a little bit better protected maybe use like a Google sign in at least like a two-factor authentication make sure that as you work with your documents you keep running some like security checks to make sure all the endpoints are properly um are properly protected but now we have we have a live version. So this is literally some something that we could share with someone. We could now go into superbase and we could create another if I go to authentication. I go to users. You can see over here I could now I could create a user and I could add another analyst in here. I could give them their password and then auto confirm like this is the simple approach and I could send them those credentials and then um they can log in and they can start using the app. They have their own history, their own kind of like little setup over here and now they can start to ask questions. If I open up a new chat here, let's see if there's also this local host. Uh let's come over here to the real one. Let's create a new chat. It's taking a little bit long. I might may have to check what's going on over there. But now let's uh let's see. Let's run one for Nvidia. So how did Nvidia describe demand drivers customer concentration? Okay. Now again, this will go in search. Now, please be aware that currently how it's set up, we focus on uh getting like really making sure we're right and grounding the answer in all of the chunks, which is what you want to do with a solution like this. But as you'll see, it's like working in a loop. It's using GBT 5.5, one of the more kind of like frontier models right now. So, this will incure some costs. So um if you are experimenting with this what you may want to do is also just pick um a cheaper model like a cheaper faster model knowing that the results will not maybe be as good but this can easily become like one one to two $1 probably something like that per query. So just something to keep in mind. also when you calculate the business case. There are still tons of improvements that we can do to this in order to speed this up and to make it cheaper. But for now, this is a this is a good one. So, let's uh let let let's run this and then come back to what we got going on over here. And then what I still need to do is we had some changes. So, now I was on the development branch, right? I had some changes. And now I want to uh I want to update this again. And I haven't set it up right now where it has an automated CI/CD pipeline. Meaning that when I push to a branch that it will automatically redeploy. So I'm going to show you right now how we can manually push to the back end and the front end so that as you continue to work on this app, you can um keep updating it. So let's come over to the uh sorry to the guides to the railway deployment and let's see where was it. Okay, so we can see that here in the deployment document. So this is uh it shows you what you need to run if you do this manually via the CLI if you do this for the first time. Um but you can also do this exactly the same if you want to redeploy, right? So we use the railway CLI. So it's railway up. Then for the back end, we link to the backend folder. Then we use these params. We link our project and then we detach it. So these are the two commands that we can run. So if I now come in here and let's clean this up and let's see make sure we're in the in the root so we can actually run this. I'm simply copy pasting this just seeing how this goes right now because we made some changes. Let's see index build logs. Okay, let's see. Yeah, so you can see this is now building again. So just checking where. Okay, so yeah, I think this is currently what's active. You can see this is the active deployment and then this is now uh what's building. So this is just the command that I ran and it's just overwriting everything again with the new changes. So we can we can also do exactly the same for the front end. So let's come back clear this up and deploy the front end and let's see there we go. That is now also look you can see that's also building this is completed. So now server started again. Great. So this whole world of deployments, it's uh once you get the hang of it, it doesn't really matter which platform you really work with because you understand the principles, right? Because but the first time, maybe it's the first time you're doing something like this, it's like black magic. Like you have no idea what's going on, right? But like if I can give you one mental model around this, it's you work on something locally. You have to startup commands, right? to start the both the back end and the front end application like we did in this project. And deploying is nothing more than pushing all that code to some kind of like cloud server, cloud compute, which is just a computer that you're renting somewhere else that's always on. You put your code there and you run the startup command. That's that's pretty much what it is. Now, there's usually one other thing that always needs to be done and that is variables, right? So, you can see this is our back end. We have all our secrets, all of our environment variables. Now I can add those manually. So I can create a new variable and I can say let let's say if we want an entropic let's say API key uh because we we want to use entropic as well. We can add that here. Now uh the MCP server added this for me. Again not really recommended to let an MCP and let an AI agent handle all of this but again I'm going to nuke all of this. So that's why we're doing this. It's usually better to do like go here. Here you can see you can see all my credentials. Then uh you can also so you can use the raw editor in here and you can simply copy paste everything that you have locally. So boom you copy it over and then this is also where you can make a distinction between um parameters or secrets that you run in production versus what you run locally. Again if you have different database different API keys that you want to use uh that's what you can do over here. And then same look here's a front end this also updated again. So this also succeeded and here you can see also the variables. This is all similar to literally what we have locally, right? So we had that uh over here you can see these and then the only thing to keep in mind because we are working locally here you can see the fight API base URL is on local host and I assume that this is exactly this is the URL from the back end because now it's not connecting with local host but it's connecting with our um with our back end which is hosted under that URL. So probably same over here. Let's see database URL. That's all the same. Okay. So that's how the whole deployment side of things works uh with uh with render. Now there's a lot more that we can do. If I dive into the settings over here, you can for example see like I um this is ideally let's see let's just see if this works because it was now in the US. For me that doesn't really make a lot of sense. Right now I'm probably on a like a free plan. Like you can see I have 30-day laps left. I'm just using some credits in here. If you need the beefier like CPU or more memory, you can increase that. I think for this project, it's fine. Although one gig um one gig of memory is a little bit on the low end. So, especially like if this were an application with 40 users, like lots of chats and all of that, you probably want to bump that up to four. That's usually our default that we uh so two two vCPUs and then four gigs of RAM. Usually good enough. Um, so the whole deployment now works pretty much via those Docker files. So it auto detects those Docker files. So that's something to uh to keep in mind because in those Docker files that we have both in the front end and in the back end, which I did I didn't really quite cover the AI created those. But here you can see this is what essentially sets up the environment, right? So you can see it gets so this is for the front end. So, it it gets um it gets the dependencies. It installs everything and then uh let's see. It does the build and that's pretty much all that's needed on the front end. And then on the back end, you can see again gets Python. It uh let's see, it gets UV. So, it installs UV. We make sure we set up the virtual environment. And then again, we expose via port 8,000. And then here's the command right we we run u uicorn app main app and this is how then the application starts running. So docker is just with docker this is just a simple way to set this up so we can kind of like avoid that problem of hey it runs on my m machine and not on the deployment right but there's always a little bit of tinkering in order to set it up correctly in the beginning. Cool. So now let's come back to our uh let's see let's first refresh this because remember we did a new new deployment. This all still works. So here you can see the Nvidia uh question, right? So you can see it has all the sources cited. We can click on it. We can see it. We got the nice tables over here. I mean this is fancy. Like this is actually really cool. This is really this is starting to look like something like for a first like proof of of concept that you could ship towards a client, right? We've we've done smaller builds than this for uh for clients. So, I'm quite happy with how we're progressing so far. All right. So, I think now is the perfect time to zoom out a little bit again and bring a little bit more structure again to everything that we've we've done because I can fully imagine when you followed along like things have been a little bit messy, right? And it was on purpose. This is what development looks like in the real world. It is messy. It's not as like here's exactly what I want to do and then go there in a straight line. So if we now come back to our original architecture diagram, we have implemented everything which is quite insane that we did that in one single YouTube video. I mean, if if you follow this and if you copy this and show it as a portfolio project on your resume or something like that, like it will greatly increase the chances of you being hired anywhere because not a lot of people know how to work on applications across the full stack because usually in when you have a job, you are focused on like more of a vertical, right? So, you either are like a front-end developer, you're a backend engineer, you're I don't know a data engineer or and you're a DevOps engineer. Like these are separate worlds. So, not a lot of people know how all of this ties together. So, we now have a public URL where users can log in. They can access their own chats and we can start to ask questions where all of our data lives in the remote database that's hosted on Superbase. We created the tables originally in Python. So in our back end, in the app, in the database, we had all of our uh here all of our models, right? So we mapped them out in Python using SQL alchemy and then through AmIC and the migrations, we've been able to sync those with the database. Also big unlock. A lot of people have no idea that you can do this and they manually kind of like tinker with with the database and then if something changes, they have to do this as well. complete game changer using a lambic in order to sync all of this. Okay. Then we set up both the back end and the front end. We set it up in a single mono repository with very strict agent files so we could leverage AI in a way that can quickly move back and forth between the two surfaces and always reason over the full context. Then we started to work on the whole document ingestion pipeline where we had to deal with HTML htm files and we used dockling in order to parse this but then run into some issues with the tables and created a custom table extractor so that when we look at the sources we [clears throat] actually see nicely parsed data. Great improvement. Then we continued to put everything really into our database. So we have the we have the the document tables, we have the source documents and eventually we had the chunks which were set up with a vector embedding and also with a full text search. So we can combine the hybrid search. Right? So that's the whole pipeline over here. And we use OpenAI to create those embeddings. So those were all of the individual pieces of the puzzle and then we brought everything together and created a very sophisticated search agent really. Let me let me come back because that is I want to show it to you over here one more time. If we where are we? We need to go to development and we go to the back end app assistant. Where is it? Is it retrieval? I think it's this one. Yes. We created a very sophisticated retrieval algorithm using multiple techniques in order to bring everything together and we used pyic AI to create an [clears throat] agent or is it with a set of tools. So the search process it is not as naive and linear where it just grabs something and then gets it back. Now the agent can also reason in a loop with search filing reach read chunks rechunk and read surrounding chunks. So we give it a little bit more time. That's also why you see that when you ask a question coming over here. Let's create this local host again. Do it here. That's also why you see that when we ask a question you can see it takes some time. So you can see it's going to search and then [clears throat] later on it's going to do another search. That's that process in there. We're trading speed and cost also really for better, more grounded answers. This is a trade-off that you have to make on every project. How fast do we want it to be? How accurate do we need to be? And this can be a whole optimization process on its own. And then we brought everything together using Railway where I've shown you how you can deploy a back end, a front end, how to let the two talk to each other and even [clears throat] connect to the database. And that's really how you build a full stack genai project. And regardless really really of what you work on, what stack you use, wha