Make Your LangSmith Deployment Multi-Tenant

Key Takeaways

Make your LangSmith deployments multi-tenant in just a few lines of code. In this video, we're going to walk through how you can add user authentication to your deployed agents in LangSmith deployment so that each user will get their own scope threads, runs, and conversation history, all managed by the deployment itself. Let's go ahead and jump right into it. With every LangSmith deployment, you get access to over 30 plus API endpoints that come as part of our agent server. This agent server will include endpoints such as our assistant endpoints for creating and managing assistants, which are essentially ways to reuse the same graph architecture with different configurable fields. We can also create new threads, which are essentially conversations that your users are having with your agent. Then we can kick off new runs inside of those threads and stream back their outputs using a variety of different streaming methods. We can create and manage cron jobs on our deployment as well, kick off stateless runs for one-off kind of executions. We can interact with the store that comes as part of our deployment, and we can even interact with our agents via A2A or MQTT protocol. These all work great. However, by default, they will be locked down by just the LangSmith API key that is part of that workspace. In a lot of production use cases, you might want to use something to enforce multi-tenant access to all of these different resources so that users can't interact with other users' resources, including their conversations or their threads. And you can even manage which users have access to which types of resources. By default, all these endpoints will be locked down by a LangSmith API key. Let's go ahead and see what this looks like in practice. So here, I've deployed a full-stack application, and now I'm allowing users to sign up and interact with our agents via our LangSmith deployment. So you can see here we had a user sign in as test@gmail.com. And if I navigate over to these threads, we can see it has access to a lot of different threads. And that's a little suspicious since I haven't allowed them to create any new threads yet. So let me go ahead and give it a shot and say hello from test at gmail.com. And we'll see this kind of flows through and we stream back the response. And that works great. We're able to interact with our deployment, no problems there. What I'm going to do now is sign out. And I can sign in as another user. You can see now if I navigate up to the threads view, I can then select the most recent thread. We can see here we can see the thread from our test@gmail.com. Now this isn't ideal. In most use cases, you want to allow users to lock down their resources um and make sure that other users can't see the threads and conversations they're creating. This is all logic that you can define in your LangGraph deployments very easily with our custom auth handlers. So let's take a look at how this works in practice. Before we jump into the code, let's take a quick look at how this works architecturally. So on the far left, you'll see we have our client app. In the middle, we have our auth provider. Um and this could be any auth provider you choose. For this demo today, we're going to use Supabase. And then lastly, we have our LangSmith deployment's agent server. You can see here the first step is a user logs in. We then return the token back from the auth provider to our client app. Then when we make a request from our client app, we can pass through that token to our agent server. We can then validate that token with the auth provider here, confirm the validity, and then apply any access controls that we deem necessary based on their roles, based on their current permissions inside of their org. We can then return the resources back from our agent server. This may seem a little complex, but it's actually very easy to implement inside of our LangSmith deployments. So let's go ahead and jump into our project and we can explore how we can do this in code. Okay. So the project we're going to be using today is actually a full-stack application that we've deployed onto LangSmith deployment. We'll be doing a future video on how to do this in depth, but for today, we're just going to focus on the auth component. And the reason why we chose this project is since everything is self-contained inside of one repo, it makes it really easy to understand how this auth flows from the front end to the back end and vice versa. So let's go ahead and jump into the actual implementation. So the first thing I'm going to show you is our langgraph.json. And this is essentially our configuration file that our LangSmith deployment will look at. So you can see the first thing we do is point to our agent here and where it's compiled. For this example, we're going to be using a deep agent from our deep agents package. And this is actually pulled directly from that repo. There's an examples folder, and inside of there we have a deep research implementation. So I thought I'd just take that, copy it over, and that's what we'll be using today as our actual agent implementation. The next thing you'll see is this auth.py. This auth file that we're pointing to is how we can go ahead and define that custom authentication logic that we'll see today. And lastly, you can see we're pointing to this app.py and pointing to some custom routes. Custom routes are essentially a way to add additional API endpoints onto our agent server. So out of the box, we get those 30 plus endpoints. We can additionally add even more endpoints for custom logic. So here, we're going to actually be serving a front end through that custom route. That again will be covered in a future video. It's not important for today's video. What is important is the auth implementation. So let's go ahead and navigate over to this auth.py and we can explore this implementation together. Okay. Inside this auth.py file, we can see all the custom logic that we're using in order to authenticate users and enable this multi-tenancy. So the first thing you'll see is from the LangGraph SDK, we're importing this auth class. This is what provides that unified system in order for us to authenticate users and manage all of this logic. The next thing you'll see we import is this is_studio_user. And this just allows us to basically bypass all of this custom auth logic if we're sending requests from inside LangSmith Studio. The next thing you see is we initialize this auth class down here. From there, we initialize all of our Supabase environment variables and initialize an HTTP client. Okay. On to our first auth handler, you'll see here we have this auth.authenticate. And this is going to be able to register any function as the authentication handler. So essentially, we're going to decorate this get_current_user, which accepts an authorization uh token, and then from there, we're going to validate the user and then return back this minimal user dict. This is just a dictionary that will return all the contents of the user that we want to either be able to access inside of our graph or inside of our other authentication functions. So you can see the first thing we do is we grab out the authorization token. We then send that to Supabase to get the details about the current user. We'll go ahead and return the response. And then we'll return the identity of the user, their display name, in this case we're going to use their email, and then finally their role. Um and we'll get into why we need this role in a moment, but this will essentially allow us to scope certain resources based on roles as well. Great. Now on to the auth_on handler. This is the default handler that will run on every resource operation, and it's going to write the user's ID into the resource metadata as an owner, and then it's going to return that as a filter. So when a user lists their threads, the deployment will only return the threads that they own. This happens automatically without any additional logic from our front end. You'll see here we also include this is_studio_user in order to bypass the default auth if we're calling this from LangSmith Studio. The next area that you'll see is we have this final handler for auth_on_crons_create. And this is basically permission-based access that will only allow users with role of admin to create crons inside of our application. This is really powerful when you think about only allowing certain types of users to interact with certain resources on your deployment. This could be cron jobs. This could be the ability to create and manage assistants, right? Like different configurations of your agents. So just know that you have this ability to enforce permission-based access, and you can get as granular to the method as well. And that's it. There's no custom middleware needed, no database queries on every request, just three decorators, or really two if you just want to scope resources to your users, and your deployment is now multi-tenant with granular permission. Let's quickly navigate over to our front end so we can understand how we interact with our deployment and pass along these tokens for authentication. Here in our front end, we have a very simple implementation using Supabase. Again, you can absolutely use any auth provider that you choose. We just use Supabase here since they made it very easy for us to authenticate. So you can see here, we are just going to create this Supabase client in order to reuse that later. If I go over to our auth.tsx, you can see we have a very basic kind of login and sign up flow where we're going to use our Supabase methods here for sign up and sign in with password to basically allow users to sign up and sign in. That makes it very easy for us to just create a form in order to handle all of this logic for us. If I navigate over to our app.tsx, you can see how easy it is for us to interact with our deployment. The main thing to point out here is we're just going to be setting this access token as part of our default headers. And essentially, anytime we make a request back to our LangSmith deployment, we're just going to include that access token that we got from Supabase. Let's go ahead and see this in action. So we've already created our langgraph.json. We saw what was inside of our auth.py, and now we know our front end works and is ready to go. Additionally, if I navigate back down to our SRC, we can see that we have our agent created and compiled here. So we have that create_deep_agent function, and we've compiled it here and pointed to it in our langgraph.json. What we can do now is actually go ahead and test this locally. So let's go ahead and open up our terminal and give this a shot. So here we are in our terminal. The first thing I'll actually need to do is build our front end since we're going to build it and mount it onto that custom route. Again, we'll cover this in a separate video, but let's just go ahead and do that really quickly. So I'll navigate to our front end and run npm run build. Awesome. So now I can navigate back to the root, clear this out, and we can go ahead and run uv run langgraph dev. And that's going to build and package up our local server, and it's going to go ahead and open Studio for us. Now Studio's great for interacting with our agent itself, but I'm actually fairly confident that this agent is ready to go. So I can actually navigate over to our front end that's running locally now and give it a try from there. So for that, I'm going to go up here and go to our localhost:2024. And we're now in our locally running application. So, I can go ahead and sign up here. So, I'll give it just a dummy email here. And we can go ahead and log in. So, now I can make a request here and say something like, "Hey." And we can see our deep agent works and is able to respond back to us. If I go up here to our threads on the top right, we can see that we have this new thread created. What I can do now is sign out and go ahead and sign up as a new user. And now if I navigate up to our threads, you can see here we have no threads. So, I am unable to access the threads that our other user just created. So, that worked great. Now, what I can do is go ahead and actually deploy our agent. So, let's go back to the terminal and we can do that. All we have to do in order to deploy this agent from our terminal is just run UV run langgraph deploy. We can then give it a name. And then you can see it's going to go ahead and build our image. It's going to, you know, package up all of our environment variables and everything we need from our local project in order to actually deploy this onto LangSmith. So, this will take a few minutes to run. So, I'm going to let it run and then check back in. Great. So, our deployment is now finished and you can see it gave us a few different URLs while it was building this over the course of a few minutes. One is the link to our actual deployment in LangSmith. And you can see here we have access to everything in our deployment. I can go ahead and open up this connect tab. We can interact with our deployment in studio. We also have nice docs on how to use this via our LangGraph SDK. And also some nice ways to use this via A2A or agent to agent protocol from Google. Since we actually deployed a front end with this, let's go ahead and interact with it now and make sure this multi-tenancy is working as intended. So, we can see here we have our API URL as part of our deployment. I can go ahead and copy that. And let's go ahead and paste that with a slash app. And that's going to now take us to our full-stack application that we deployed. What I can do is now enter in the same email that I was using locally. And the reason why we're able to sign in into this deployed agent now is because we're still using the same Supabase project under the hood. So, we actually didn't need to modify anything there. We'll see now though that in the top right we don't have any threads because those threads were kind of running locally. I can go ahead and kick off a new thread down here. And we can interact with our agent again and see it responds back to us as intended. Great. So, I see this new thread was created. And that's going to be scoped to this user. Just to confirm, let's go ahead and sign out. And we can go and log in to that other account we created. And if I go back up to the threads over here, we'll see that we don't have any threads created. And that's the whole pattern. You can plug in your own auth providers and implement multi-tenant auth in your LangSmith deployments with just a few auth handlers. We'll be sure to link to the docs and the resources we used in this video in the description below. If you have any questions, please feel free to reach out to us on the LangChain forum so we can get you the help that you need. Thanks again for watching.