Creating a GitHub Issue Resolver AI Agent with Haystack

Key Takeaways

Hello everyone and thank you for joining today's session. My name is Ree and I'll be your moderator. Today we're going to get started in a couple of minutes. We're just waiting for everyone to join. In the meantime though, please make sure that you've registered for today's session. You can do so by scanning the QR code that's in the top lefthand corner of the screen. Now, you can also find a link to it in the chat and in the video description as well. Uh registered for the session and we will send you the recording and the resources as soon as they're available in our resource center and that should be tomorrow. Uh if you attended yesterday's session, you should also have uh the recording for that and the resources in your inbox. Uh and it should have arrived about an hour ago. Uh today's code, we are going to be working in data camp data lab. Uh you do not need a premium account to code along with us live. You can do this all on a free account so long as you have a data camp account. Uh if you want to get started and have a quick peek at the slides and at the notebook, they are in the resources document that is pinned in the chat and also in the video description. Also, if you haven't already, please do subscribe to our YouTube channel and drop a like on this video. Helps push it out to people like yourself so that we can uh spread the learning and yeah, get everyone uh up to speed on AI agents and using Haststack as well. Um if you have any questions at any point throughout the session, let us know in the chat. We're going to be covering your questions for the last 10 minutes of the session. But also, if you're coding along with us live and you have any questions, let us know in the chat and we may stop and help you out and fix any issues. So, for anyone that's just joined, welcome and thank you for joining today's codeong. My name is Ree and I'll be your moderator. If you haven't done so already, please do register for the session using the QR code on screen now or the links pinned in the chat and in the video description. Uh we also have a resources document that's in the chat and the video description that will be your main uh source of content for this session. It includes the workbook that we're going to be working in in data lab and it also includes the slides as well as a few other relevant resources that we put together for you as well. So yeah, if you want to code along with us live, please do check out the resources link. If you have any questions throughout the session, let us know in the chat. We're going to be covering your questions for the last 10 minutes of the session. And if you're coding along live, let us know if you have any issues as well and we can help you get those sorted. With that, I think that's everything from me. Uh I will be sharing a code halfway through the session so that you can get 40% off uh data lab if uh that's something that interests you. So please do stick around and I will share the code in due course. With that I will hand over to your host for today's session, Richie. Richie please take it away. Um, actually I I don't know whether my camera's working there. Hold on. Let me just switch over cameras. Bear with me a second. Just having a few technical uh shenanigans here. Okay, there we go. You can actually see me. Wonderful. Uh, so sorry about that. Uh, hi there data scamps and data jams. This is Richie and, uh, today we are combining two of the hottest ideas in AI right now. So, uh, I'm sure you all know that AI agents are basically the biggest story around. And so, being able to create agents is about the most marketable skill you can have. Uh, so that means today we're going to create one. And secondly, we're also going to learn how to use Haststack. So, Hastack is one of the most popular AI development frameworks. So again, an incredibly important skill to uh have. So combining these two, we're going to be focusing on a use case of a time-consuming but boring task, which is the best kind of thing to automate with AI. So we're going to make an agent to resolve GitHub issues. Apologies to any software developers here like, "Yeah, I really like doing GitHub admin messing about like resolving issues." Actually, it can occasionally be quite therapeutic, but uh mostly a little bit uh little bit tedious. Now uh our guest today uh is uh exactly the right person for this. So it is uh Bila Eugel. Uh she's a developer relations engineer uh for Haststack at uh DeepSet. Uh so that is Bila spends uh all day teaching the world how to use Haststack. So welcome Bila. Hi thank you Richie. How are you? And how are people joining this session? It's I'm super excited to be here and talk about like agents, GitHub, and of course, hstack. Absolutely. So, uh yeah, Bill is an experienced software uh engineer turned educator and her work focuses on building and training a haststack developer community on GitHub, Discord, and in real life. And uh before her time at Deepset, Bila was a web developer and a uh at the product development company Hippo. So uh yeah uh uh very wellqualified and uh please take it away Bila. Yeah thank you. All right let's start because we have a lot to cover today. Um but first like you know AI agents are a bit big topic. So like these agents are can be these agents can be like hard to understand. So I prepared like a short presentation explaining you what like these things are. So let's go ahead and like talk about that. So um my name is Vila. I work as a developer relations engineer at Deepset which already gave a lot of introduction about me. But if you like to connect on LinkedIn, you can also find me there and we can have a further chat. I don't want to waste my time here like talking about myself. So here's today's agenda. First we'll talk about like agents and workflows. Then we will see how we can implement those agents in haststack. and I will show you a real demo of a GitHub issue resolver agent that we are going to build together through a data uh lab and we will also have some time uh in the end for question and answering but I will be happy to answer your questions during the live coding if you have like if you're stuck at some point so feel free to like also send your questions along the way all right so agents workflows and core concepts um so like in Like 2025 is the year of agents. And there's not like one definition of what an AI agent is, but here's my definition. An AI agent is a system that autonomously pursues a goal by interacting with its environment and using tools. So basically the idea behind agents is there's a human input, a query from the user. there's an LLM running behind the scene and this LLM takes some actions uh absorb observes the environment gets some feedback takes further actions or thinks that okay this is like this query this human query is resource so I can just stop so basically you see like there is some sort of a loop an autonomous system that can decide on its own what to do next should it like make another tool call or make like or stop But like in like past these systems were quite static, very sequential, very predefined. Uh but right now it's not the case. Agents kind of make some autonomous uh things and interact with environment get some inputs produces more tool tool calls etc. So this brings us back to the AI workflow. So if we know that AI agents these autonomous systems what are uh AI workflows? So an AI workflow is a system that completes predefined tasks by processing inputs through a structured sequence of operations using LLMs. So if you know like there's a deterministic steps uh like sequence of steps and if you know what to do for your tasks the AI workflow can be a great fit for you because usually in the AI workflow you know there are you know the steps you know what to do of course there might be branches it doesn't have to be linear like this uh but in the end the the path the the road is defined and there's nothing unexpected happening here. Um, for example, like an AI workflow can be a great fit if you have a customer request uh like service where like there are like requests coming from the customers and what you need to do is to like summarize the these customer requests, put some tags on them and then like direct them to the right team. Maybe it's the sales, maybe it's customer success team, like you kind of do it with the AI workflow. And this AI workflow, the steps are defined. There's nothing that that needs to be figured out along the way. And when that's the case, you don't need the AI agents. So like if you put them like side to side, uh AI agents make sense to use if you have a complex multi-step problems requiring diverse actions or if you need multiple tools or resources to resolve your use case. Maybe your goals are clear but the path to that goal is not predetermined. So you don't know exactly what to do. Whereas like if you're if your interactions I mean if you have a very common use case like such as like question answering so you if you can see the predictable pattern if you know the task and if if you can decompose them into clear steps then it makes sense for you to use AI workflows and on top of that I think it also boils down to stability robustness and efficiency. So if these are important for you uh independent from your use case it makes sense for you to use AI workflows and it's mainly because okay like agents are great systems but they are quite autonomous so they can do things that are unpredictable that are complex so they are like good size and bad sides but like when you give this much autonomy to an AI agent you also get some risks and risks are like generating ing wrong answers like picking wrong tools or maybe there is like a very high latency and you don't want to deal with that. So when these things are important to you, so if you need like if you're working in a very delicate domain, you might want to uh use AI agents less and have deterministic AI workflows more. But like if AI agents are different than AI workflows, how do you kind of achieve that? But it it's it's how I mean how you can do that is like choosing an AI agent or choosing an AI workflow is not zero or one decision. You can kind of meet in the middle and this is actually what we call agentic systems. So it means that you you give some you have some agents or agentic behavior in your system but there are also some deterministic things that you are doing. um this is meeting in the middle and I think this is the best way of taking your application into production because we don't see autonomous agents autonomous AI agents running on production all the time. They always need some sort of like deterministic steps or maybe human in the loop. So something that checks what these agents are doing. uh unless uh you have this agentic system. In this agentic systems you can do some undeterministic things but when it comes to a point you can say like okay thank you agent thank you for your output I will take it from there and produce the things that I want to do. Um this is important because we are going to do we are going to implement our agent in an agentic system. So we will have nondeterministic behavior and deterministic behavior together. Um before we go into the details of um before we go into the technical details of this uh let's talk about tool calling uh because tool calling is is core of agentic systems and AI agents as well. So tool calling, you might even heard of it uh as function calling. Tool calling refers to the ability of a language model to dynamically invoke external functions and APIs as a part of its reasoning process. So if there's a user query coming, LLM can decide to choose the tools uh to choose the relevant tool to resolve that query. For example, if the user is asking for what's the weather uh like in Berlin, the LLM can decide to call the weather API with the Berlin uh parameter so that it learns about the weather in Berlin. And it can also uh be for different tasks. The LLM can decide to do multiple tool calls or like one like sequential tool calls. This is this is the core of an agent. And if you are working with like very famous large language models out there, most of them are um supporting tool calling at this point. So like Entropic, OpenAI, Gemini or like other big LLM providers when you think about it. All right. So let's talk about Haststack and how we can implement these systems with uh Haststack. So, Hastack is an open-source LLM orchestration framework by DeepSet and it provides the tools that Python developers need to build real world agentic AI systems. And the idea behind Haststack is components and pipelines. Components are smaller unit. they have one responsibility such as like creating embeddings, generating text, maybe retrieving information from different data, splitting your uh files into smaller chunks etc. So everything that you think as one specific task can be boiled down into a component and by connecting components to each other you form pipelines. Um I will go back to that uh in a short time but before that let's take a look at the haystack ecosystem. So right now it has more than 21,000 stars on uh GitHub. It has more than 160 built-in components that you can already start using out of the box. It has 70 integrations. It means that you can use some external tools like monitoring tools, LM providers, evaluation frameworks out there and like put it into your pipeline. And if you are just a new beginner or if you're looking for ways to solve your use case, there are multiple examples, tutorials and recipes already there in the website. So you can check check them ahead and at least like get some ideas of how you can build your uh application in mind. Okay, now it's the code part. So this is an example component in haststack. Uh basically each component in haststack is a python class with the component decorator that comes in the haststack package. And all you need to do is to define a run function for this component and initialize the inputs and the outputs of this uh component. So each component you can think them as the nodes in a graph and of course like as an as a node it has some input edges and output edges. So you can have like multiple inputs and multiple outputs. The only important thing here is to define their types in your component when you're initializing that and that's it. And in in the component you can do whatever you want in a very custom way. So each component are implemented like this in haststack. But if you see that like you have a very custom use case or like very custom logic that you want to do or maybe there's a tool that you're using internally at your organization, you can go ahead and like build your own custom component and put it into your pipeline. And pipeline in haststack is actually a directed graph. This means that you can connect these nodes in any way you want. So you can create like your modular application with the component. You can connect the outputs of each node or each component to different components and you can have like branches. You can merge those branches. You can even have loops in your graph and it gives you a lot of flexibility when you're building your custom application with haststack. And here for example you see some yellow nodes and these yellow nodes are custom components. So just like any other co component that comes out of the box for with haststack you can implement your component and put it in uh in your pipeline in your directed graph basically. So I will start with a very simple use case then we will like implement a um more um complex pipeline together. So here's an example of a retrieval augmented generation pipeline implemented with haststack. So imagining that you are using embedding embeddings for your document. The first step here is to have an embed embedded component that gets the query user query creates an embedding by using the models out there and sending it to the receiver. Then receiver can connect to your database, your document store and retrieve relevant information by using that query embedding. Then we have a third component prompt builder that gets the user query and retrieve and retrieve documents and renders this rag prompt that we usually use in our applications. And as the last step, we can use a generator component that is your connection to an LLM. this generating component can get the prompt from the prompt builder basically send it to the llm and get the response that will be uh in the end the answer of this whole pipeline. So as you see like each step is covered by a different component but in the end it gives you a clear idea of what you are doing in this pipeline and you can even see what's happening between components. So it's e it's very easy to also like monitor and see what's happening in your pipeline in general. Um this was the pipeline and we also have this very special component agent component. So just like any other component you can put this agent component in your pipeline and use it in a pipeline uh in a rag pipeline even. Uh but this agent component is more capable than other components that we have out there because it can get the user request and it can connect to multiple tools. So component tool, MCP servers through MCP tool just like regular tools or tool sets and generate an answer um by using these tools. And of course like maybe something that I need to mention here. Of course this agent uses an LLM under the hood and the and this agent is um a model agnostic. So you can use entropic models, open AI models, Gemini models, open source models out there. Basically the only requirement is that this model that you want to use or this model provider that you want to use in your agent needs to support tool calling and that's it. If if tool support is there then you can use this LLM in your agent and this can be even a local LLM running through Lama for example. Um let's take a look at these tool abstractions. So the the simple thing that we have is tool. So basically if you have a Python function maybe this is like a function that is connecting that is connecting to an external API you can easily turn that into a tool and it's quite easy flexible and you don't need to know a lot about what's happening in that tool. If you just want to use that abstraction then we have component tool. This turns any haststack component into a tool for an agent. For example, there are like web search components we have in haststack or retriever components. You can easily connect convert these components into tools by using this component tool class. And another tool class that we have is MCP tool. This allows an agent to connect to any tool hosted on an MCP server. So if there's a ser MCP server running um outside like externally you can easily connect to that server and use the tools on that server in your haststack agent by using that abstraction. All right. So this one like since this part like till this part it was uh about the the theory of agents and how we implement stuff with hstack but let's see everything now in code. So together we will build a GitHub issue resolver agent. It already has a repository on GitHub. So if you go to github.comset.ai AI GitHub agent. You see the link in on the screen right now and follow the instructions in read me. Um you can do everything on your own. Um all you need is a GitHub personal access token and an entropic openai basically an LLM provider API key. Um since this is an agent and this is a complex or like uh maybe like very functioning agent um the free tier of most of LLM providers is not enough for us. So just keep that in mind that you might need to have a paid tier uh to run this agent for yourself. All right. So let me share my other screen and we can start coding together. All right. So um this GitHub issue resolver AI agent with hstack we are going to use the entropy cloud for sonet model. And what this agent will do is to giving a issue URL. The agent will fetch and parse the issue description and commands. It will identify the relevant repository, the directories and files. It will retrieve and process the file content and determine the next steps for resolution and post them as a comment. And for this we will use the agent component that we have. Uh and I already mentioned I talked about a lot uh this agent. And the good thing about this agent is you can use it as a standalone component or within a pipeline. And for this one, we will use it in a pipeline. So let me take you through this pipeline. So this is very similar to the rack pipeline that I showed you previously. So this is a linear one. I don't need to do anything like extra uh or anything uh any loops or branches for this one. So it will in the end have one, two, three components and two tools. Of course like if you like you can extend the tool list here like provide more tools to your agent. So first um we will have this GitHub issue viewer component. the um since we want to implement something modular here like every component will have some specific use case um and we will uh handle that and it will handle that on its own. So what this component will do it will get the issue description and commands of a GitHub of an GitHub issue. Then we will have this GitHub issue builder. It will just render the issue details as a prompt um so that um we can pass this prompt to our issue resolver agent and this issue resolver agent is the agent component that we have in haststack. It will have access to two tools. One of them is GitHub repository viewer. It will iteratively explore the repository GitHub repository and the other comp other other tool that we have is GitHub comment writer. It will basically send the generated comment to GitHub. So the idea is maybe let me share let me share one example that an example that we have. So what I did is to I forked haststack um haststack repository to my personal account on GitHub. Um and I have an issue here. So here I want to have like I want to have princing chunk by default in haststack agents and here I define what the agent should do and what we are going to do today is to our agent will get this issue description. It will go and explore the haststack repository try to understand how it can resolve this uh feature request and create a plan for us basically. So all right without further ado let's go back and start implementing. So the first thing that we need to do here is to um install entropic haststack and github haststack packages because we will use entropic sonet 4 uh for our agent and we will use some uh github components or github tools to uh for this um example and we already have them as integration. All right. So let's run this cell. It's running. All right. So the next step is to uh provide the entropic API key and GitHub token. So um basically you can get access to your entropic API key from your user profile on entropic. But if you already have a preaid uh like uh account for other LM providers, you can change the API key name here and use that instead. uh but just for this one we will use anthropic and for GitHub uh I mean I can also show you quickly how you can get that thing. So basically you go settings developer settings and under that you need to go to tokens classic and here by clicking this generate new token you can generate a new token just make sure that it has public repo access if you want to also like fork this fork a repository and like do this on your own and once the the token is generated you need to copy it um and copy it and like save it somewhere else otherwise you will lose the um the token. All right, let's go back here. So I will add my tokens here quickly. So if you use this get pass um package uh it will um ask you your API key but it will not expose it. So you can easily use it like in notebooks or in live streams like that. So I'm just passing the tokens that I have here. All right. So we have our tokens. Next step is to initialize a GitHub issue viewer component. So remember this was the first component that we needed and it's already implemented um in haststack integration as an haststack integration under this GitHub hstack package. So I will just import it from that package and what I need to do is to initialize it. So I say like issue and say that GitHub issue viewer and also what I need to do is to also like provide this uh GitHub token so this API key to this component. So what I and the way to do that in haststack is through secret. So I will say secret dot from and var and give this name here so that it picks up my token and what I can do is to then run this component with this example that I have here. So basically I want to show you how this works and I can already show you. Let me do that. So imagine we have this issue. It has like lots of comments like from people. It's a close one but it doesn't really matter. We just want to test if this component so this retrieval information like information retrieval works as we expected. So I will copy that link and get it here. So I what I'll do I'll just pass it as the URL and maybe I can say like issue comments here and we can run this cell. All right. So here you'll see that under this documents key there's a like a list of documents. So one document, two document, three documents. So the first comment, so the issue description is saved as the first document here and the rest are the issue comments that we have in in this issue here. And each document has content. So the issue description or issue comment. Then in the meta we have type, title, number, state, created at, updated at, author and URL information. So this will be useful especially when we are generating the prompt for the agent and let's go ahead and do that. So the if I go back here so this is check. So we created this component. it was already uh in the integration. So I will go ahead and create this issue builder component. So in haststack prompts uh can use ginger syntax. This means that you can have loops in your prompts. You can have if else statements in your prompt and this becomes quite handy especially if you want to like get information from different resources and put like more than the document content itself. And here's what I mean. So um we will pass these documents to the to the second component that we have. So this prompt builder component and the prompt template that we are going to use is going to look like this. So it will have this issue from URL and you'll see that URL here is the prompt variable. Then we will have a for loop running over all documents that are coming from the issue. And if it's the f first document, we will put the title that is coming from the meta meta information of the document. And we will put also the each document content here between these issue command tags. I'm using these tags because uh for entropic I'm going to use a specific prompt, but that's not uh necessarily needed for um other LLM providers. So since this is my template, I already have one. So I can just create my issue builder component by using the chat prompt builder component that I have as the template. I will use chat message from user. Where's that? So as you know like the way to communicate with LLMs are through messages. So we basically send user messages or system messages and get assistant messages back. So what I'm doing right now is to create a user message and for that I will use this issue template and the I need to make it a list and then I will also have required variables all this is just to get not to get the warning basically this is not mandatory so let's initialize our issue builder and make sure that everything works fine. All right. So, what I can do maybe on top is to yeah run this thing uh with the documents that we have here just to give it a try. I don't need to but document and I will pass the documents cell here and see what we have. Oops. An issue. Oh, URL. Yeah, we forgot about the URL because we have documents variable and URL variable here. So let's add that to and since this was from this issue I can just get it here put it and run it again. Yeah, this should be fine now. Come on. Yeah. All right. So, here you see that there is this prompt as a chat message and in the content you see like issue from like we use in our prompt the the URL of the issue and then we see the title and we see like each issue comment behind these like issue comment tags. Okay. So far what we did is we created this component and this component. So basically what we do is if there's a URL coming we can go go to GitHub check get that issue commands and render that as a nice prompt like as we want as we initialize here and we can pass this information to our agent and our agent will take care of the rest. Let's go to that step. But before that, we need to initialize the tools that we are going to use for this agent. So the first tool that we need is this GitHub repo viewer tool. What this tool does is like it retrieves content from GitHub repository and based on given repo and path. So basically what it does is it iteratively explores the repo. So it goes through the the directory and if there's another directory and if it sees if it thinks that it needs to go there it can go into that directory. So it helps us to basically explore every file that we have in the directory but we don't need to think about its logic because we already have a tool for that so we can directly use that tool. So what I will do is do repo viewer tool. And don't worry if you feel kind of like lost right now. In the end it will all make sense. So let me go ahead and write this. Let's go back and get this GitHub token part here too. Yes. So this was this is the first tool that our agent is going to use. Uh and we initialized that already. And the other tool that we are going to use is this u GitHub issue commentor tool and for that I'm going to do something different. There is this tool already available in the GitHub hack integration but I will I will not use that one just for the sake of this live coding. I will create my own tool for that and it's going to be a mockup tool but it's also nice because then you will see how to also like define a tool like a custom tool. So I will start with the tool decorator and I will define a function and can name as like GitHub issue commentor. This will have like command as a as command parameter as input and it will have it will return string and what I will do is I will not do much in this component. I'll just want it to return this command. So maybe let me go ahead and add a description for that too. For that I will go back to my notes and put this description. [Music] All right. So basically this is a function uh a Python function that I use and it will um it will just uh I will turn that into a tool by using this tool decorator and our agent will use that one as the last step. So if it can come up with a plan to resolve that issue, it will use this tool and by using this uh tool definition I will be able to see what my agent came up with. So this going to be handy. All right. So I am going to go quite fast from this part. So now that we have tools, we have our components. Uh the missing part is the agent. So for the agent we need a chat generator and a system prompt. Let's go ahead and create our chat generator. For that we are going to use entropic chat generator. So I'm going to go ahead and initialize that chat generator and I give a model name here and our model name can be this the latest cloud sonet model. Let's do that. Yes. And then as the system prompt, um you are free to do anything you want here, but there's already a system prompt coming from this integration. So I will use that one. And if you want to take a look at this system prompt without me making any typos, I'll just copy paste it here. And you see that like not like this. Let's see this prompt. Okay. Here in this prompt, it's not the whole prompt, but we are instructing our agent to uh how we are saying our agent on like how to explore the repository, how to resolve an issue. It's a pretty long um prompt. Uh if you go to the repository of this integration, you can see the full prompt there and you can also find the link of the prompt here. Okay. So we have our chat generator, we have our system prompt, we have our tools. Let's create this agent in the end. So uh for an agent we have like chat generator, system prompt tools. But we can also define something extra like exit conditions, state schema and streaming callback. And we are going to initialize them as well because they are going to be handing. So I am going to first provide this chat generator. So this will be the LLM. So the brain of the agent. Then as the system prompt um I should yeah system prompt. I will go back and get the system promp prompt coming from this package as the tools. We will I will go back. My first tool is this. And the other tool is this repo viewer tool. And then I will define an exit condition. I should put a comma an exit condition here. And it will be GitHub issue commentor. So what I do is to I let my agent to do anything at once. So I will let my agent to explore the repository multiple times by using this repo viewer tool. But when it thinks that it's time to use this GitHub issue commenter, it should stop because it means that the issue is already the issue command is already on GitHub. So it doesn't need to do anything further. Um I define that. Then I define the state schema and it's going to be documents because documents are actually my um issue comments. So I want to save them in state. So I will say that it's a list of document object that we have in heystag and streaming callback. Come on. Yeah. streaming callback is going to be the sprint streaming chunk. It that will allow me to see every tool result, every tool call when they are happening. Um, so let's initialize that. All right. So the last part is to bring them all together and create this whole pipeline. So here's my pipeline and what I do is to I add these components one by one to my pipeline with this add component function and let's start with the very first component that we initialize which is issue we go back. Yeah, I say issue viewer. Then I will copy this. I think the second component was issue builder component. Let me double check that. Yes, issue builder component that this is rendering the issue. And the third one is going to be let me copy and paste it. The third one is going to be this issue resolver agent. And I will add that and then the so like basically I add this components to my pipeline but I need to also connect them and and define how I want these components to be connected. So issue resolver connect without typos and I say like okay issue viewer component docu is returning documents and I want these documents to be connected to the issue builder documents because remember the prompt was waiting for these documents parameter. Then I know that issue builder is going to render a prompt. So I want the prompt coming from this to re go into the issue resolver agent as messages because my agent is expecting a list of messages as the prompt. There's small typo here. Okay. So let's run this pipeline. And if you see this message, it means that okay, at least like the types are fitting. So your pipeline is ready to use. So okay, this is the exciting part. Now we are going to try this whole pipeline and see if this works. Hopefully it will work. So I have an issue URL I that I showed you like previously. I want my agent to like come up with a plan to have this streaming call back function like streaming callback parameter as print streaming chunk by default and we will see if the if the agent will be able to do that. So what I do is to I have this issue resolver pipeline and I will run it with the issue URL that I have and since I enabled this sit streaming callback we'll be able to see the tool codes. So let's run this and cross our fingers. So here we will we see the thinking process of the agent. Um so it says that it's a feature request for the haststack framework. The user wants to enable streaming by default in agents to provide more visibility into tool calls and results etc etc. So it also says like how it needs to come up with that plan. So it says like let me explore the repository to understand how the agent components currently implemented where the streaming callback parameter is defined the print streaming chunk utility function how streaming currently works in the codebase. So this is just like the initial thinking of this agent and then it makes a tool call and we see the tool result here. Uh it doesn't allow me to scroll as it's like generating text here. But in the tool result you see a list of files. It's basically like exploring the haystack repository right now and coming up with the directory u directories that we have. Okay. So now it now it's too long to be displayed in full at least but to some extent it gave it's going giving us some idea uh on what our agent is doing and the cell is still running. So it's because it's trying to explore like the the repository for example it's looking for the agent component and where this agent component is. It thinks that it's like in this components folder. So it goes there. Um and for that it notices this agents folder. So it checks that. So so basically it does a lot of things and I see that it is it has stopped now. So but you see like the thing is quite huge because first it explores the content then it looks at the content itself tries to come up with a plan etc. And we won't be able to see the full thing here, but we will be able to see the full thing when we run this cell hopefully. All right. Apparently, things didn't work as we expected. So, let's see what happened here. What we can do is to check the issue resolver or agent and see if there are any messages. We see the documents. I don't need the documents. Okay. Something is wrong here. Let me go back and see what we have in this result first. So, and I noticed that my internet is not doing very well right now. So that's that might be one thing. A lot of content to debug. Okay. Let's see the keys here first. Okay. There is something weird happening here because this should have been more than the the agent output but all right we'll go with that and let's see the last message then this last message you All right. Okay. Then if I do text here, I should be able to see the thing that it came up with. And what I can do maybe nicely is to print this. Okay. So basically what this agent did is to it explored the whole repository and it located the agent.py Pi so where the agent component lives and it understand and it understood where this printed dream chunk function is came up with the plan and plan is also here I mean it's not exactly what we expected to work uh but this is the reason why uh you need to also be hands-on with the agents so if you uh I believe the solution is already shared with you so if you go there you will be able to also see the exact output that we are expecting from this agent. Um, so the idea is basically it needs to explore the repository, check the file contents and uh come up with a plan. Now it seems like it's it stops somewhere in the middle. Probably something is wrong. But that's the uh that's a thing with live demos, right? So you cannot always trust these LLMs running under the hood there especially if they are working on complex tasks sometimes like they can be quite they can work in a very uh unexpected way but maybe I can show you the the exact thing that we were expecting from this by sharing the solution. All right. So what we were expecting from this agent to come up with the the idea. So it was observing the current implementation um coming up with benefits of change implement implementation of the request and it can it was all even like suggesting like text testing consideration and it had this uh last part like recommendation explaining like why you need to why we need to do this. um feature request, etc., etc. All right, so that was it from me. I know I'm already over time. Um, and if you if there are any questions, maybe this is a like a great time to answer those questions. Uh, wonderful. Uh, thank you, Biler. That was very cool. I also like the impromptu debugging at the end. It's always nice to see what happens when things go wrong and how do you go about trying to figure out like uh just debugging things and uh retracing what happened there. U so uh we got time for a couple of questions from the audience. Um before we get to that, I've got a question for you. So it seems like um once you know what you want the agent to do and you can break it down into specific tasks, the actual the writing of the code seems fairly straightforward. I guess the hard part is how do you decide what are the different steps in in the process like how do you go about figuring out what the workflow for the agent should be? Well, I think first you need to like come up with a plan yourself and if you see that there's a deterministic part uh you can definitely remove that from the agent. For example, for this one, we knew that the input of the system is going to be a URL and the agent will only be responsible of like consuming the content coming from that issue URL and that's it. So like I could have just like put create like another set of tools and put that into a agent and say it like if there's like a URL coming use this like use these tools to get the content of the URL etc. But since I know that this is going to be like every time I don't need to like provide it to my agents. So overall I think like how I can say like if you can do separate the deterministic behavior from nondeterministic behavior. That's like the biggest thing that you need to do and like try different things. Try edge cases. um like when AI agents work they are great but when they don't work then like it's very hard to debug that's why like uh monitoring tools like lang fuels or arise are quite helpful there when you connect your haststack pipeline or haststack agent to those tools they help you to see like what's happening under the hood because like when there are a lot of tool calls when there are a lot of like thinking process happening when agent is working internally, you don't know what's going on there unless you're using like monitoring tools. Of course, like you can use streaming to help you when you're developing, but as you saw, like for example, in this editor, it cut off at after like 1,000 characters, which is like not always handy. Uh yeah, there's a maybe the last point is maybe like a a feature request for the uh the data lab team. Uh this provide more than thousand character output. Um, all right. So, um, it sounds like, uh, I guess the takeaway here is that reasoning is quite expensive. Thinking about what you should do is difficult. So, you want to try and narrow the scope as much as possible and then break things down to smaller steps and only make the AI uh, the agent part do simple things and do as much deterministic stuff separately with other tools. Yeah, exactly. And also one thing I need to remind everybody here is like if you're working with agents don't use like uh products out there like I mean in a way that for example when we are using this uh agent I forked the hstack repository I didn't want to use my own uh personal token I used like I created another account and use the token for that one. So like when you are giving access to production to those agents be mindful like test it with like forks and mockups and or staging environments first like this is quite important especially if you providing this right access to the to those agents. Yes definitely there testing stuff before you release into the wild seems very useful. Um and can you talk me through um about cost control as well? I don't know how much did it cost to run this agent like once through. So like for this one it costed like 30 cents I believe. Uh and it's in the end made a lot of tool calls because it was trying to like explore the whole uh uh repository. Um so like this is like a very heavy task I would say. So 30 cents is again like a reasonable amount of money for this task I would say. Okay. Yeah. So in this case um you probably want to only use uh an agent for uh tasks that are going to take up a lot of time for a human then like if it's just something you can solve in in a few seconds then don't bother with it. Exactly. Like imagine that it it runs for almost two minutes and in those two minutes it makes a lot of like calls and it thinking uh of course like there are ways to optimize that but like when you're doing like when you just started to work with agents I think like you kind of like it takes you it takes some time to optimize everything first you need to like make it work then you start optimizing it. Um all right super. Uh so uh we've got time for a couple of audience questions. Before we get to that, I just want to say uh we've got the radar conference coming up on June 26th. Uh please do register for that if you haven't already. It's one of the biggest events of the year. We've got a whole day talking about AI with some of the top experts around the world. Uh there's a link in the chat. There is a QR code on the screen to sign up for it. I'll also say uh I am going to be back tomorrow for a webinar on uh the data camp quarterly roadmap. So you find out uh what's just been happening with data camp and what's coming soon. And then next Tuesday we have a session on building and evaluating uh rag pipelines. So also uh sort of agent uh related uh retrieve augmented generation is another incredibly important uh skill to learn about. So please do come back for that. All right with that uh let's go to some questions. So uh Peter had a question about uh this is when you were presenting the uh the slides. So why do we need to embed the query again uh in the retriever if it's already been embedded once beforehand? Oh yeah, great question. Um so when you embed uh the things that you embed are actually your documents. So you don't embed the query. First you create embeddings for your documents. You put it into your document store for to your database. So wherever you are storing your information and then of course to calculate the similarity you need to create an embedding for your query as well because like you can like index your documents any time but query is coming in during like runtime. So you need to create another embedding calculate the similarity retrieve like most similar uh five documents, 10 documents, 50 documents. this is a setting that you need to define and then like do retrieval. Hope that answers Peter's questions. So basically it's the difference between indexing and query. Okay. Yes. So if you want to see um how similar or different two bits of text are. You got to convert them both into matrices. So you can measure distance there. So you need to do it with the documents and with your query so they're in the the same kind of number format. All right. Nice. Uh there was one more related question. Uh so this comes from NIS. So um when you're showing like the different uh tools that are used um say it looked like there's no support for Google embedding uh within Haststack. Is that the case or or not? Good question. I'm not sure. I think there's support for there's support for Google embeddings. Yes, but I need to check that. But if you want you can also go ahead and check it out by going into like hstack.deet.ai/inttegrations. So if you go to hastack website you will see the integrations tab and there we have a list of like every integration that we have every component that we have you can definitely see that but I think there's support for Google embers if I okay so like it's probably there or at least coming soon uh go and check out the documentation all right super uh thank you so much to that uh Bill that was very very cool stuff uh a lot of fun thanks thank you all right cheers and uh thank you to everyone in the audience who asked a Thank you to everyone who showed up today.