Building a Linear issue agent with Langsmith Agent Builder

Key Takeaways

Hi, I'm Sam and I'm a product manager at Langchain and today I'm going to give you a brief tour of the Langmith agent builder which we're launching as a public beta this week. Uh so these are the agents that I have available for myself in production right now. Uh you can see my support email filter assistant, my daily calendar briefer. Uh, the one I'm going to be showing today is the linear issue manager, which monitors a Slack channel that the engineering team that builds the agent builder spends most of its time in for feature requests or for bug reports. And it then takes that information and it it manages our linear board. So, uh, the board in which we manage our our issues and our feature requests. Um, it can create new issues. It can update existing issues. Uh, then it reports back in Slack with the work that it's done. But to start, I'm going to show you how to create such an agent. So I'll click the new agent button up here in the top right corner. Uh we want to make it really easy to build agents on the agent builder. And so we've allowed people to create agents using natural language. So let me just describe the task now. So I've written out I want an agent that listens for messages in my team Slack channel and manages our linear board when it receives bug reports and feature requests. So I send that message and that goes to our agent creator agent as we call it. Um and it searches the available tools, the available triggers. Um and what it does then is it it manages a bunch of files. So basically the way that memory works in the Langmith agent builder is that uh memory is a file system. Um the system prompt lives in a file. The tools connected to an agent lives lives in a file. And so all of those files can be uh managed by the agent itself. Uh so then what it does is it asks me follow-up questions to gather more information. having seen what tools it's going to need, it knows the arguments to those tools. Um, it then can can sort of know what it does not know and it asks that information from me. So, which linear team should issues be created in agent builder team. How should the agent distinguish between bug reports and feature requests? Um, this look for keyword suggestion looks just fine. Should the agent respond in Slack after creating a linear issue? Yes, reply with confirmation and link to the linear issue. Perfect. Should certain types of messages be ignored? Uh, yes. Only process messages that clearly look like bug reports or feature requests? So, I submit that additional information and now the creator agent is taking it and it's going to make the final agent configuration for me. Okay, so this process is completed. uh it's generated an agent configuration for me which you can see triggers off of receiving messages in a slack channel and it has a small number of linear tools attached to it and then also a tool Slack reply to message which it will use to communicate back to me after it does its work. Um so let's say I'm still workshopping this agent. So as opposed to being available to everybody in my Langmith workspace, I'm going to say this agent should only be available to me just to start. And then I'll connect my Slack channel to it. And once I do that, I can hit create agent which takes us to the agent editor page. Uh this is where you can update the configuration of the agent and also sort of see how it's configured. Uh so on the the left side here we have the instructions which is effectively like the system prompt for the agent. Uh bottom left we have the the trigger connected to the agent and then also we have the toolbox which has the same uh set of tools uh backed by MCP that I just showed you on the prior page. To show you how it works, I'm going to use the test chat right here on the right side and I'll ask it what issues exist today on agent builder team. So I've asked the agent a question what issues exist today on the agent builder team. It first calls linear list teams which it needs to to do to get the exact UU ID of of the team and then it searches for issues uh with pageionation. It calls this tool twice and then it finds 82 total issues on our team. um and has them grouped by status. So that was the agent that I just created for you on the fly. But to show you the one that my team uses in production every day, I'll go back to the linear issue manager, which is this agent here, which has a very similar configuration uh as the agent that we just created. It has a bunch of linear tools attached to it and then a single Slack tool to reply to a message and it is triggered on receiving a new uh message in a Slack channel. Uh to show you an example of how that actually works, here's just an example conversation from last week where our CEO Harrison gave feedback on our product. Um lead engineer on our team responded to that, said, "I'm working on a fix, but in the meantime, Linearbot, please file an issue." The Linearbot filed that issue automatically and then linked us to it right in Slack. And just briefly, I'm going to show you um the the use agent page or the chat page in the in the links with UI. Uh these are all the conversations with the agent. Uh really this is an ambient agent so it's running in the background. So I'm not super often looking at these threads but you can see see it's receiving all these Slack messages. Uh with those messages it is calling linear list labels list issues uh then create issue three times in this case. I guess it was uh three bug reports or feature requests that were filed in the same black thread and then it replies to the message uh with what it did. So thank you for watching. This has been a a brief demo of the uh linksmith agent builder and my linear issue manager bot.