🔐 Deploy a PRIVATE Chroma vector database on Render.com | tutorial

Key Takeaways

hey everyone my name is Timothy kbat founder of mlex labs and today is my going to be my third video on chroma that is hopefully not going to get outdated where we deploy chroma with persistent directory so our data is always ours and we can grab it with an API key and we do this all on render render is by far the most expensive deployment option setting you back at about 25 bucks per month and some change for storage AWS will set you back around 13 bucks or so but it's way more involved if you're not comfortable with command line render is by far and way the most convenient option it is also the fastest so this video will be quite short and also just so everybody can timeline this it's November 21st 2023 we are on chroma version 0.4.1 18 this was deployed about three hours ago so very recent and obviously to run this you will need a render account with billing attached again as I said this will set you back around 25 bucks a month at minimum because of the demand requirements for CPU that chroma requires to run fast you really don't want to go smaller your performance will be horrible so on your render dashboard all you need to do is go to new web service now there are two options here building from GitHub or building from a repo which you can do but takes way longer or we can just use the advanced option which is frankly much more simple and actually just pull in the chroma DB image so we go chroma DB SL chroma just click away and you'll see that it finds the docker image for Chroma and then we can give our service a name so let's call this chroma with credentials as you can see I've already done this before we can add it to a project this is a render specific thing it's just for organization um we can deploy in a region that's closest to us I recommend doing that reduces latency and then of course this is where I was talking about where the money comes in you will need at least two gigabytes of RAM and this isn't because chroma is eating up an insane amount of storage there's actually a lot built in here and that Ram will come in handy and you will need it as I said about 25 bucks a month minimum Advanced options however are going to be very important here and there are a couple nuances just for render they're not complicated we're going to walk through them right now so we need to add some environment variables luckily a very kind contributor Yours Truly updated the chroma doc recently to show you how we can set environment variables to use a x chroma token header while we're using chroma so that we can have API authentication and I do want to give a shout out to I believe it is Taz and the core team who actually got authentication embedded into chroma extremely quickly uh Round of Applause for them honestly I did not help I just updated the doc so literally if you're saying thank you say thank you to them now we want to use the X chroma token type of authentication schema so we're going to need a couple environment variables this first one right here the server off credentials and what we're going to want to put is this is going to be our token so let's just call this like SK secret that's going to be the actual token we use to connect then we need the credentials provider now be very careful to not include the double quote on any of these values you do not need quotes around any of these they will be automatically quoted for you by render do not accidentally did that do that I actually record this video previously and forgot to press record and I did that so you're going to get the express version of this video because you will learn from my mistakes of which there are many and numerous now we have set all of our environment variables uh this x chroma token is going to translate into the header X hyphen chroma hyphen token so just so you know that that's what that's doing the next thing is we need a disc so this is going to be a persistent storage basic basically hard drive and we're going to call it chroma storage doesn't matter what you call it what does matter is where you mount it so I'm going to mount it to slash storage has to be an absolute path and you can't mount it to root just do SL storage if you have no idea what I just said the next thing is you're actually going going to go back up to the environment variables and you're going to want to paste in a new environment variable that you haven't seen in any of these other videos and that is persist uncore directory and what is that that is actually going to be the location of which where we write all of our chroma data too which is slash storage and then you can just make a subfolder here you don't want to write it directly into storage in case for whatever reason you want to store more things just it's just bad practice don't do that we're going to write it to a correctly spelled chroma hyphen storage folder it chroma will automatically create the folder in the persistent directory so that it makes sense and that it all works now after we have set that last one we're going to want to modify the size of our volume with render you can only go up in size you can't go down and you pay 25 cents per gigabyte per month so you're going to want to make sure you use the smallest storage capacity that you can stomach before upgrading because it will affect your monthly cost which is again at minimum 25 bucks a month depending on your storage or depending on your server type lastly we do want to actually do a health check and the way to do that is/ API slv1 heartbeat this will return a 200 okay response if for whatever reason your server crashes render will actually send you an email saying hey uh your server appears to be unhealthy which would then prompt you to go restart the server or do whatever needs to be done after that that's it just click create web service this process of pulling in the container and restarting it actually will only take but a couple minutes honestly be aware on render when starting your container for the first time like we're doing right now the process can take a little bit longer but subsequent restarts will be much faster and also so they will pull in the latest image and all of that so what you're going to see in the logs here is you're going to see it boot then you're going to see render restart it to map the URL here directly to Port 8000 this is also nice you come with https included which is really nice okay so it looks like our uh service was completely deployed you see we got the latest uh event log and we're good to go um it even hit our heartbeat URL in Postman let's hit that same URL the heartbeat endpoint actually will always return 200 even if you have authentication it isn't a private endpoint what is a private endpoint is collections so let's hit that you see we get unauthorized this is expected so I'm back in anything llm because I just use it as an easy way to chat with docs and also embed documents in Vector databases like chroma so I can chat with them so I'm going to select chroma as my option here I actually have my URL chroma with credentials but let's just copy it to be sure we know we're using the X chroma token so let's go back into here and so if we go back to our environment variables you can now see the misspelled SK hyphen secret with two e for some reason because I typed it wrong we go back into here and we go to turn on this thing go to SK secret with misspelling and then we now have our collection we go back to anything llm post that token in there save changes and I'm just doing this again because this is easy for me to embed documents I already have two documents embedded so let's just move them over save and embed and if I go to postman and request our collections we should have one now and we do now let's say that you're like me and you realize that you just misspelled your API token because you just want to do that or maybe you accidentally committed it and you want to change it because now the whole world knows it all you need to do is go into your render server go to environment go to this environment variable and change this to whatever the new value is that hopefully is still spelled correctly wow okay and so now my new secret will be SK hyphen Hunter 2 the SK is just for me you can literally have this just be the word hello world if you like I'm going click save changes and what we should see in the logs of this service is our system will get rebooted but because we have that disk storage attached attached to render we actually will persist our data so we'll just have a new API key which is great okay so it looks like our container has restarted with those new environment variables so let's go and hit it from the API endpoint you can see that we get unauthorized this is expected because we changed our key so let's do Hunter 2 send it again and we get our data and our data has persisted because of our smart use of the storage device and the clever use of environment variables that chroma provides us likewise if you wish to turn off authentication you would just delete all of these don't remove the persistent directory it will mess everything up but in general your service is now live you now have a render server running chroma on its latest instance you can always update the image at any time update your API key update your credentials you have an https domain which is nice because some people require it and you are now good to go again let me just reiterate this is the quickest and easiest but the most expensive option AWS is a little bit more involved if you can't stomach AWS this is is the best option to run chroma aside from locally hopefully this documentation remains relevant for longer than the previous video did and you are good to go as always if you have any questions comments or compliments I would love to hear them and until then stay tuned for more videos from mlex labs