Gemini CLI Guide (1hr Workshop)
Key Takeaways
The video is a 1hr workshop on Gemini CLI, a terminal-based tool for AI agents, covering its features, usage, and integration with other tools like Cloud Code, and providing practical steps for configuring and using Gemini CLI for tasks such as content creation, search, and planning.
Full Transcript
All right, I want to get started. There's so much um exciting material to cover and I really want to make sure we have enough time and that we don't go over the time. There's a lot of like cool stuff that will be released today. I think one of those ones that I want to bring to your attention is um you know GPT I think 5 is about to be released today. So that's something you want to like keep track of. So that's going to be in a couple of hours. Um lot of rumors around that. very interested in myself to see what these new models are going to bring. But today the emphasis is going to be on um Gemini CLI specifically, you know, Gemini um the Gemini models and how to how to leverage um the CLI specifically. Um, so what I want to do is I want to make this as interactive as possible and for you to take away something from today and hopefully you know you're encouraged to go and try out a few of these tools. So two tools that I've been using recently are cloud code and Gemini CLI. Now I can talk about how these two compare. I will potentially do some other session on that for our academy subs um later on because there's been a lot of questions about that you know which of the CLI that I should I use is is it cloud code is it Gemini CLI um is it some other CLI there's so many of them out there um and there are a few others that will also be released very soon um you will see there's a lot of news around that you know the week is just getting started literally there's so much releases that are coming around this technology so there's going to be a lot of interesting stuff Um, so what what I want to do is um yeah, sure. I can I can start off with a quick introduction. I typically don't introduce myself because I'm I suspect you already kind of know me from like Twitter or something like that. But yes, I'm an independent AI researcher. I do a lot of research um for my own company and for other companies as well. I do provide some uh consulting on the side. Um I also love like teaching and doing training. So you will see me a lot of like doing these type of sessions. Um very passionate about that and I I'm curious I have a lot of curiosity about you know where this technology is going and I do a lot of experimenting um and I love sharing this stuff. I I think there's a lot of value that you can get either you know professionally or personally and that's the stuff that I think about you know um every day when I get up that's what I think about how do you I use this technology to you know to enhance my life and augment everything that I do. That's basically kind of the the backstory for me at least. Um, but yes, um, you know, I'm I'm in different places like LinkedIn. Um, I write a lot as well. I have like really popular newsletters. Um, I I share a lot of like research papers is something that I learned when I was doing my PhD. Um, I enjoy doing that. I still enjoy doing that u to some extent today. um you know I think I was probably one of the well among the the small group of people that started to share like AI papers but no it's everyone sharing AI papers in a way it's a good thing because there there's a lot of communication about these ideas so um but yeah this is something I enjoy doing like sharing uh insights and things like that um I've also worked on you know on on training these models um previously I was at at companies like Meta like helping to like shape up this technology ology language models, you know, like training these models, what kind of data you need, things like that, fine-tuning these models, evaluating the models as well. So, I have some history and that has been kind of my emphasis for for the last couple of years. So, that's a little bit about myself. Um, all right. So, I'm looking at the chat, right? I think what's important here is to make this interactive. So, what I've learned recently is that I could give you a demo and it could just me time for later for a discussion and I'm going to try to keep my demo actually short but at the same time that you take away something, right? So, that's kind of my goal. So, let's get started with this. So, I have here a terminal open, right? So, this is why it's called Gemini CLI because it's in the terminal, right? Some of you may come from the world of like chatpt, right? You use agents there. uh some of you are using things like init um and some of you might be developers that are using things like the you know agents SDK langchain whatever that might be um so there's something here for everyone I think and why I'm excited about the CLI is that um so the the trend seems to be for me at least that we're going from um you know we're going from these like chatbased applications into more agentic solutions right agentic applications meaning more work gets done for you automatically and asynchronously. So what does that mean? So you will see a little bit of what I mean by that in a bit when I show the demo. Um and this I think is the future of like AI agents, right? Things that are done for you on your behalf, but you still have a bit of control over the results um over any actions and decisions, right? You want to offload some decisions for the system and there's a lot that can be automated that you don't need to be involved in but you also have to build and use these tools with the idea that you know you want to build something that's useful and something that's actionable but you also have to build with you in mind right like you being part of of the process so this is why I think these tools are kind of really interesting now the the Gemini CLI I think the cloud code these these different systems are like they're in the terminal meaning they were designed to have a bit of control over your, you know, in this case your local machine, right? Your file system and things like that. Uh, so it can edit files. Um, and that means it can be pretty good at a lot of applications, but it can also be pretty bad. So you need to understand that you there's a lot of control that you're given the agent. So the more you can customize it for what you want it to do, the better uh the experience is going to be. You know, you're not going to like do something catastrophic with your system, dest files and things like that. So we want to start off with that because that's really important. Anyways, that's enough talking. I wanted to just give an intro on that. What's the inspiration behind you know covering something like Gemini CLI? Even if you are someone that's like not developer oriented, I think this is still going to be extremely useful for you. And the reason I say that is because look, I I have been experimenting with cloud code. Cloud code I use it like pretty much 60% of the day. I would use cloud code as my this is my top AI tool now. And I sometimes switch over to Gemini CLI because it has access to Gemini which means it has access to longer context which is useful for me for tasks that are you know where that require long context understanding and like like analysis of bigger files things like that. So I'm going to show you an example of what I mean by that now. Um so how do we start this? I have everything set up. Um, if you want to know how you can set up your, you know, your Gemini CLI, there'll be a lot of instructions over, you know, out there. Um, you can just go to the Gemini CLI GitHub repo. I think that would be the best place for you to go get started. And so, I think that that's the one I recommend. So, I've been just going through like their docs, you know, finding out what's the most possible that's the most capable thing that you can do with with Gemini. And I think I have a really cool demo for you guys today. So, let's go and start with that. So what I'm going to do is I'm just going to open Gemini. Um so this is basically how you start it once you have it installed. I think it's a node package. So you can install it with whatever um package manager you have. So this is how I've started it. Now some people ask me why are you using Gemini CLI inside your IDE? What is this about? Well, it turns out that for a lot of the tasks that I do, I actually love to interact with, you know, either Gemini CLI or cloud code. And I like to have this editor as well where I can sort of tap into, in this case, my ID, which is Windsorf, could be cursor, could be anything, right? Could be VS Code. Um, and the reason I like this like switching is because I love using the tools like autocomplete, like the tab autocomplete from Windsor, which has gotten really really good. Um, you know, so for instance, edit this file. I'll just like show you here. So let's say I wanted to add like an MCP server here, right? So if I wanted to add an MCP server, you can do it, you know, via commands. But if you don't remember your commands here in the Gemini CLI, um you know, you can always go and manually do it in the editor. So the cool thing about the editor, so let's say I wanted to add something new here. Um so let's say it's like just as an example here, I'm just going to use the same one there. But you see that autocomplete really like because this saves a lot of time. So here it might not be obvious because I'm just adding like a couple of lines. But imagine if you had something that was generated by cloud code and you need to refine it a bit more and you now want to be involved in editing that code because you saw something that the the system wasn't uh producing really well. Well, I can just switch to that editor, right? I could just switch to the file and do the edits myself. And I also like the the the diff view as well. So the diff view is really useful here. Um this is the reason why I use the CLI within my IDE. So if for those of you that use let me just undo this before my MCP um configuration here. So for those of you that are familiar with uh cloud code right so you can also just run cloud code and gemini at the same time as well. So you can also switch between the two. This is another I think advantage of using the ID here. So let me see if I can open up. Yeah, there we go. So I can open up cloud code really easily here. And now I have okay I have the tab autocomplete. I have the cloud code also working. So I can just switch between these things. Right? So normally what I do is I use um the ID which is here as my editor. I use cloud code as my like code um agentic system and then I would use Gemini for like like content creation on other types of tasks that I work on. Right? So it's like I have all of these systems right within the interface. And I really like this because I can just switch out switch off like switch on and whatever. Let me just close this one. Okay. So that's the idea. So just in case you were curious about this, but you can use it, you know, in the in the CLI if that's something that you prefer. Um, all right. So let's start off with something here. Um, I've prepared a few things here for you. Um, but let me take a pause here because I noticed a lot of questions. So I just want to answer a few questions and then get rolling with the demo. I've been holding the demo for a bit too now and too long now and I just want to get started with that. But let me just check the chat here. All right. So there's a lot of stuff coming in. All right, let me see. Oh, yes. So, any thoughts? Um, so there's a question here. Any thoughts on Quinn CLI? Um, it's a fork of Gemini CLI. Yes. Um, I think with Quinn, right, they're I believe it's called there's like a coder, Quinn coder or something like that. Um, all of these companies will probably have something like this. Um, I I I can see that pattern. I haven't tested it yet to be honest. Um I don't really use the Quinn models that much. I mostly would use um these proprietary models because I mostly work with the APIs. So I've moved a little bit away from fine-tuning models and fine tune models just because a lot of these proprietary models get a lot of the job that I do, you know, they they help me get done the jobs that I do. So I don't rely too much in these open source models that much today as I used to before. But anyways, I'm getting into it again. And so this is something I'm actually looking into um especially with like these new open source models like Quen, right? There's the GLM model. There's even OpenAI has been doing open source model. So it's something you want to be testing and you want to like be um you know on top of just to see if there is um you know if there's some potential there. Uh but I think like in my experience so far with using these CLIs these these coding um agentic systems I would you know it's it's either between Gemini um the CLI and then the cloud code. Cloud code is the one I use the most but those are the two I have the most experience with and I'll be doing some I'll be doing some comparison between the two here and there. I think that will be useful just to kind of ground this conversation and say well you know Gemini is not perfect. Gemini CLI is not perfect. It needs a lot of work. um you know cloud code I think at least had some kind of momentum I think starting from last year and they have you know I think they were one of the first ones to introduce this idea and so I think Gemini is still a little bit behind but there's still a lot of potential with it today just some thoughts on that yes and I and I like that Leonardo for bringing that up thank you for bringing that up so the reason and this is something you've been pushing that team right so interact with the team a lot give them feedback But I like this idea of giving free credits and they launch like that, right? This is why I think they gained a lot of momentum like they launch with free credits means you can use the system, you can experiment with it. That's super important. So I like that and and and they do give you more credits through the developer program for sure. Yeah, security issues is is always going to be um a real problem especially with the MCP stuff. Um, I'm going to show you like the MCP later and maybe we can have a deeper discussion about that. Okay. Um, um, Louis I Okay. So, your question is what's going to be taught can be can be used in cloud code as well. Definitely everything I'm going to teach you here um that I'm going to show in the demo can be done with cloud code. In fact, I already did it with cloud code. um most of the most of the workflows that I use today I do prefer to use cloud code is just a lot better and I'm going to be honest about that but I think there again as I said I hope to bring some light into like what are the advantages of using something like Gemini CLI um okay all right anyways so let's get started with the demo here so what I want to do is um I've configured ahead of time uh a few things here so if you do the slash right if you do slash like this um okay so just before that I want to show you that you see this little instru this little message here. It says I have one Gemini MD file and then I have um you know I have these MCP servers. This Gemini file here basically is the memory of your system, right? It's not the most like advanced method on how to do memory, but it's memory for the system in a sense that you can give it instructions if you want to tune the behavior of Gemini. And in fact, when you do start a project, it's a really good practice to um to start off with a with a Gemini MD file, I believe. I'm not sure. Okay, so they do have it, right? So let's say I were to want to start off with you know I have a new project and I want to start the project has some kind of structure and you want Gemini to always be grounded with that structure of that project and whatever information you want to give it. So basically what you would do is you would give it like init right and this init function u command what it does well the slash command I'll just use the right term here what it does is going to give it that context right so now when you're using gemini and you're using within the project um you know launch it within the project which in this case is for me is course I'll show you the the root folder here course everything goes under course this should be renamed this should not be course because there's so much stuff that I do here like my projects my open source projects my research my courses my workshop everything is here so I should definitely change the name but anyway so that's the project and within this project I have like different configurations for Gemini that will help Gemini get grounded on what how I want it to behave right you really need to pay attention to this otherwise if you use Gemini out of the box you you won't get the behavior that you want especially Gemini specifically I think really needs help with that context right the additional context so let me see if I actually have um I don't have it here but because this is my root folder right so if I go to a specific project inside that specific project I would have different Gemini uh Gemini files um but this one it doesn't make sense this is way too much uh folders here um so let me share with you um so this one will start cloud so this is what that MD file will look like it's the same for cloud code that is for Gemini CLI so I could just use this reuse this but anyways this one was an example I put together for cloud um for cloud um so this is the memory file this is what it looks like so it basically it's like an MV file it has like different information. So this one I put together this um these instructions for for um cloud which can I can also reuse here in in Gemini CLI if I wanted to create like let's say an an MCP to MCP tool or MCP server. So because I do build a lot of MCP servers um this the set of instructions is really important context for the system. So it knows what are the commands that it needs to use. It has an example of the code of how the MCP servers should be implemented. So this is sort of like a meta um application of DCLI tools like you're actually building tools with them for them essentially right so that's the idea here at least with this one but anyways just an example of what a memory file would look like this is just the example right here and it doesn't change that much with um with Gemini CLI it maybe there's a little bit of structuring that Gemini MD will include but basically roughly this roughly speaking is roughly the Any any questions on that? Did that does that make sense what I just said? If it doesn't, please let me know. Um I can definitely clarify more. Anyway, so that's the idea. Um yes, there recording for sure. All sort stuff has recording. Um so let's see. Um yes, and and and obviously you keep updating it, right? um if there's new stuff like as you work with the system this is this is why I said this is the lesson I've learned with cloud code gemini and all these CLI tools is that or even things like winer cursor to be honest so you always have today where it stands it the system is not intelligent in the in in the sense that it's going to like take the the you know it's going to incorporate memory uh because maybe struggle with something no that's something that you explicitly need to tell it so In essence, you are tuning that system by giving it some like memory or additional instructions, but you have to be the one tuning it, right? Ideally, it would be the system doing that by itself. But this is why I tell um a lot of like developers that I, you know, that that I work with, it's like you you actually have to give it the instruction. You actually have to tell it the context. This is why there's a lot of conversation around context engineering because you have to do that part. um later. I think what we're expecting with the newer models and the newer architectures that are coming soon is that this stuff is more self-improved in the sense that it improves itself. So let's say it made a mistake about an MCP server that I asked it to build um and it's making that mist mistake consistently. Oh, just take a note on its own. It's just say okay I need to make a like a like a little memory of this because I'm making mistakes and and I know I know how to like correct it. So the next time I know how to do it but you know this is stuff that you have to do manually today. So definitely um I think that will be improved over time. Yes. So again um the way you would um the way you would add it here, it doesn't make too much sense for me, but let me see if I can actually um just to give you a quick demo of that. So basically what you want to do is you want to go into a specific project. So I have a project here as an example. So I'm going to go into this folder here, right? Um and then inside this folder, it's just a readme file. And basically what I can do, let me see if I can yeah make this a little bit larger. Basically what I can do is I can go and start Gemini now in this one. And then here I would initialize again a Gemini file. Right. So okay when I zoom in it looks a little bit ugly. Okay. All right. So so what I can do is I can just tell it init something like that. Right. So I can start it. So notice it says now analyzing the project to populate it right. So it says okay I'll start by analyzing the current directory to generate the comprehensive and again this is an important step you want to be doing this. Let's say you have already have some kind of project or some coding or codebase. This is what you want to do. You don't want to miss this part. That's why it's calling it. Um I think the name speaks for itself. So anyway so I'm going to allow this. Let's see what it's going to do. So it's going to go into this folder here because this is the folder I'm working on. Okay. So it's doing something. Notice it's calling different tools. It's calling the shell. It's got this listing files. Um, so it's doing something, right? And then it's going to look at the workflow. So this one is something I was testing recently with Git Gemini CLI, which is this GitHub action stuff, which I won't be able to cover today, but it's something that recently came out as well. But anyway, so it says, okay, it looks like okay, I'm just going to say, by the way, it's calling the tools to do this to to create this file for you, right? So, um I'm just going to allow it and then let's see what it shares. Oh, did did it say that? Well, it did not say that for me. I'm not sure what's going on. Maybe you it didn't install it properly. You might need to uninstall that. But that should work. I mean, it hasn't really given me issues like I have done this. I think it might be problematic if you're doing this from your root folder, like a root directory where you have like a lot of projects. I think it's better to use these type of like commands within a specific project. This is why I switched to this project in particular because I noticed that it does um tends to not do it really well when you have too much too much of these files and so on. But that should be a recognized command anyway. So, it's taking a bit of time, right? like this is one of the things that you that you you will encounter with these like CLI tools like they are they're quite slow um I think Gemini in particular is a lot faster than cloud code but they're quite slow because they're using the reasoning models for all of this stuff right it is an agent at the end of the day it has access to tools it's reasoning about what it's doing so you can expect it that especially if it has a lot of f a lot of information a lot of context that it will take a bit of time to like complete that task and sometimes it's doing things that you shouldn't be doing by the way and that's where that um that's where that Gemini MD file actually comes into play because now you can go into this file and you can tell it um so this is an instruction I want you to follow you must answer with short responses something like that you can add more stuff here you can add more instructions about how you want it to behave right um that's the idea with the Gemini file uh with the Gemini MD file Okay, that's how I utilize it. I use it for storing facts about the project, um, helping the system when it makes mistakes and offering it some corrections. Um, adding some highle instructions about how I want it to like respond and things like that. Okay, let me just close this. I might not save that. So, it's still going with that and it hasn't finished. Um, I'm just going to cancel that because that wasn't the demo I wanted to do. I just wanted to show you that quick example and I'm going to go back to my to my other stuff that I was there. Okay, hopefully that's clear. Anyway, so that's one one important part, right? And then what I what I want to do here is actually I want to focus on on the demo that I had prepared. Still I still haven't went to demo. It's already half through the hood but but it's okay. I'm answering some questions. That's fine. Um anyway, so what I will do is I'll just open um so I want to show you here like you have tools, right? So if you look at tools um this command here so it will give you the set of tools, right? So you notice that sometimes it would call the shell. Well, that's just going to be able to like remove files, right? It's going to um list files in your directory, things like that. It has access to those things. It can also do a web fetch. So I can I can ask it something like uh can you tell me the latest rumors from GPT 5? That's the thing that's most top of mind for a lot of us today. Um so basically what it will do is again this is a this is a high level a it's an agent that's set up with all of these tools. That's basically what it is right the interface you know shouldn't really matter too much here. It's just an agent. It's an agentic system. It has all these tools and it'll just use this tools to like perform tasks. So here out of the box, it's going to use um it's going to use Google search, right? It says searching the web for this. It figures out that it needs to search for GPD5. So it's doing the the GP GPD5 search here. Then it's going to return some results. Okay, there we go. So we have a summary of it. Notice it, right? Um if you guys are like familiar with perplexity, this is this is basically what a perplexity response looks like. So this is nice and I I like having this essentially right like why do I need something like like perplexity search when I can just do it within here in this context maybe there might be some use cases where I can use proplexity but I think you know having it like this is really nice so something I want to call out here with Google search um and this is the same for cloud code so you get a summary of it but you don't get the actual raw information um from the sources where the websites that were scraped, right? You don't get that here. If you wanted that, if your application needs, you know, the actual source or the website information that was scraped from from, you know, from the web, then in that case, you probably need a custom tool for that. And there's a way to do that within cloud code and Gemini CLI as well. So, what you will need for that is something called um an MCP server. Um, so these are like MCP tools. Um, MCP servers are like tools basically that you can configure. They can be scripts. They can be whatever you want them to be. They can be literally they can just be prompts like prompts that you would maybe u want to reuse everywhere. So this is something I've been experimenting with. I've actually just stored my templates like prompt templates in an MCP server and then I just connect this MCP server everywhere, right? Because MCP is like um a standard way on how to create and give these agents tools. So spend a lot of time understanding what an MCP uh server is, how to configure it because I think in the future if whatever tool you're using, you're going to be taking these MCP servers everywhere. So learn how to use this. That's why all of these tools support MCP because it's really an important concept to be familiar with. So here what I will show you here is like my custom tools that I've created, you know, via MCP servers. So I want to show you the list. Um okay, don't get scared too much about that one. This is something I've just added. So this one has uh for this particular project I have a few of them configured. I was just testing some examples here because it works. But I have this thing called ex search right. So this ex search here is essentially like web search but it's configured and customized. You basically have more control over you know how the search behaves. So if your agents need all that context and all that wrong information coming from the web search results then this is the tool that you want to use. You either use excess search um sharp API or you would use firewall. Firewall is another popular one and I know a lot of developers use it but these are the two that I use the most. Excess search and firewall will be the ones that I use. So let's say if I wanted to use in this case I wanted to use let's say can you tell me the latest rumors from catch use ex search for this. So you can just tell it like that. Again it's an age just expecting it in give it the instructions. Let's see if this actually works. what I found with Gemini CLI, a little like piece of advice here. Um, so notice what it did, right? So look at this. Look at this. So this is I think a very very very bad experience that I've gotten with Gemini CLI. And the reason is because I'm expecting that this system understand right that I am telling it to use the excess search tool that it has here to do the search but somehow it prefers to use its built-in tools over these um MCP ser MCP tools that I've configured for it and this is not great because now when you want to build an like an agentic system or an agentic workflow here more complex workflow it will not um you know it will not do what you want it to do because it's not using the right tools. It's just using its own built-in tools. I really don't like that experience. Um, so let's say I want to like I want to actually clear this out. So I just can do clear like this and it just clears up my my terminal here. And and I actually want to track a command again. Let me see. Search about 5 using the exa search MCP server. Let's see. I insist. I think I need to use this. Um, and and this is this is what I'm saying. Like it's not perfect. Now it's going to do it. Look at that. So what was the difference here? Why? How can I fix this? By the way, this is a question for you all to think about. So now it it I mean I switched around the query, but how how did it now figure it out? Okay, it you have some kind of tool here already configured. So let me just like say yes here. So now I know for sure it's using Okay, so now you see I'm getting this entire raw data. This is what I use the APIs for because now it's like I'm I'm sure that I'm going to get a lot more details, right? And this is what I love about using my own search APIs. I use them everywhere because the the responses are just a lot better when you have full context. Um and Gemini, there's no reason why Gemini should not use full context. Gemini has this one million context window. Use it like use it and the responses will be a lot better. That's a that's a good point. Google bans other search engines. No, I don't think that's what's happening. I mean, I think the the the the takeaway here is that it doesn't matter the tool that you're using. It's really important to diagnose and to debug what's going on behind the scenes. This is what I tell everyone. Like yes, it's an agentic system and you probably want to like use it to get work done really fast, but it's really it really pays off to understand why is it not using the tool this way? Why is it like not giving me the thing that I wanted to give? Can I customize this fully? That's the question you should be asking. Can I customize the behavior of the system? Right? What do I need to do to customize it? And I found that the more context the system has, right? The like about its tools and things like that, the better it's going to do. So I'll go back to the question initially that I asked. This is such an important one. Uh it doesn't matter if you use Gemini CLI, cloud code, whatever it may be. How do we fix this? Any There we go. So I asked and I got an answer. So I think Lonardo is right. I think you want to put instructions in that gemini.md file, right? About what is your preferred, you know, what is your preferred set of tools. And it will also help, by the way, to tell it that it's a tool that does this and it has an MCP server that does this and the MCP server has all of these tools. That context, if you add that to the Gemini MD file, that is going to make that um it's going to make Gemini a lot better. From my experience, this is what I'm learning um about this tool. So, I want to share a little bit also about cloud code here. I know it's not about cloud code. I keep bringing it up, but there's something interesting also about cloud code. when you use the default web search functionality. Um in fact what I found out is that I'm not sure how much of you are using cloud code but actually they're using um their latest model which is GPD uh sorry cloud cloud oppus 4 right to do the search and basically it searches and do a sum and does a summary of the search. So basically what you get is a summarized version but it's using GPD4 which uh sorry not GPD cloud oo 4 which means it's going to be a super expensive task. Um so if you're doing a lot of uh web search in cloud code be very careful if you're using the standard APIs because it's going to cost you a lot and this is something I learned recently. I didn't know about this. I mean I would have preferred to use a cheaper model for search but for some reason the quality of search is not as great if you use a cheaper model. So you want to combine that um if you're doing summaries with the more so this is an insight that's great to share because I think if you are building with web search tools that's something you want to do you want to use like your best model for summarizing because the data is kind of like mixed sometimes you know you get a lot of like like really really messy data with with search from search results. you want to have some way to kind of summarize that nicely and you want to use a really good reasoning model for that. So that's kind of the reasoning behind that. Um I think but anyway so so this is great. Um um and and we discussed again you want to like maybe tell in a Gemini MD that you have all these tools and so on. So tell the agent what it has access to so that it does better at the stuff and it does behave a little bit better know for the things that you want. You don't want to always be explicit at this because this is not really efficient. So I want to actually share something right after this which is let's say I want to use uh the search tools within a workflow that uses other tools as well right how do we do this in Gemini so well the standard way of doing this let me just clear this so the way let's say I wanted to do okay um I have this idea where I want to do um I have this query something like latest news latest news from from open AAI Right. Something like that. So I have this query. I I have this idea that I want this to be like a like let's do a little bit deeper research into latest news from open. I just don't want a single answer, right? Like when you go to to Google it gives you like a overview. I don't want that. I want something to do something a little bit deeper than that, right? Let it be like a deep research agent. So how do you do this within Gemini? Um I think it's a cool cool uh demonstration. This is something that you you can test really easily. So if you want to do that then you can tell it this is that's this is a query query is um latest news from openi and then you can tell it something like uh what else you can tell it latest news from let me see what profit so they're kind of related so how how do you how do you actually do this and then after that I wanted to do something else I wanted to um use search tool But before that I wanted to please plan a set of search subtask before searching the web. Use X search. I'm really curious. I haven't really tested but I'm really curious what it will do. Use access search MCP server to search the web and then summarize the results in uh nice one pager something like that. Okay. So with this agent you can ask it things like that. Um and and I'm really curious how it so what I'm expecting it to do is like okay it has two different it has two different queries that it needs to go and search right so this is a more complex search it needs to plan a set of like search subtask which means it's going to take this and then it's going to okay latest papers might make sense latest models might make sense these are the kind of news that I want right and it should be personalized for me so once it does that then after that it needs to use this exos search and to do the search because I don't want it to use the default search. Let's see if Gemini say I can do this right out of the box. Let's see if it can actually do this task. So, I'm I'm pretty curious. I haven't really tested this one. Uh but I want to make Okay, so it it did it did figure out. Okay, I need to use XAR, which is a good this is a good thing to see. I'm just going to allow this. All right, so it's doing something. But what what I wanted it to do um I saw that it didn't really do this first part here which is the thing that I really don't like about about Gemini, right? So I'm getting the results, but is this I mean the results doesn't matter too much. But I noticed that the results the generation stuff is pretty good. I'm not going to complain about that. But did we miss something here? Like did it actually miss something from my was I not clear about what I wanted it to do about how I wanted it to perform this this task? What what was missed here? Any any clues on that? What was missed? What did we miss here? What did the agent miss here? It did the search, right? It definitely did a search. You can see it there, right? It did not do planning. That's that's that's the that's the answer, right? It's it's not planning. Like that's really important for this task. Um maybe I could have been a little bit more specific like hey um you know plan plan ahead do some s subtask like four subtask search uh subtask for each one of the the queries and then do the search the reason why this matters for this use case is because again I don't want it to just focus on like model releases which I think that's what it's going to do just focus on model releases and the reason I need that is because I have a feeling that that's not going to give me good results it's not going to be you know robust trust in the results it's going to give me. I needed it to be a little bit more comprehensive. So I want more control over that and planning helps agents to do that really well. So that planning is really important. Here you saw a very clear example that the system ignored that instruction because like probably it didn't have a tool to do planning. Hey, I don't have a tool a tool to do planning so I would just skip this stuff. So that's why I'm suspecting and and the other thing is like it like there's no instruction about how we should deal with these queries. So maybe adding something on Gemini will help. But there's actually an easier way to do this, which is going to be the demo I wanted to get into from the beginning. So if you stayed um you know till till right now, you're going to you're going to love this one. So how do I fix this? Well, in cloud code, the way you can fix this is you just use sub agents for this stuff. You can just use sub agents. um make an agent for planning, make an agent build up agent for doing the search stuff and you can even have an agent or sub agent for doing the summarization which is this part. But here like Gemini CLI is a little bit like not doing the thing that I want. So what do I do here to fix this? Now I want to share with you um something that was recently introduced um from from um from this team which is the slash commands. The slash command actually solves this problem. So anytime you want to do something more customized and something a little bit more advanced in terms of the workflow um and maybe you are using different tools or you want the agent to use different tools this is the way forward to go as it stands in the future we will have probably sub agents and I heard from the team already uh that they're building that out so I'm expecting sub agents to be introduced into Gemini CLI very soon I'm very excited about that because I think one thing I found about Gemini Gemini CLI because it uses Gemini Gemini has really good long context understanding um there's going it's going to be useful for a lot of tasks a lot of analysis type of task where you do analyzing analyzing lots of data like heterogeneous data things like that if you have reports PDF files things like that um that you want to analyze it'll be a really good use case to use Gemini so even today as it stands it's actually really good for that but let me go ahead and show you what I mean by the slash command so inside of this folder here notice I have a cloud and I also have a Gemini so in that in cloud code basically you can set up agents like this This is not available in in Gemini unfortunately, but I I pretty sure when it does get introduced, I'll just kind of like copy and paste my my templates. But anyway, so they have these little templates that you can set up called slash commands within cloud code and and I'm just going to show you really quickly how it looks. So it looks something like this right here. So basically it says please follow the carefully the following. It's like an orchestration that you're doing. It's like a specific instruction of steps or a set of steps that you want the agent to perform and you basically tell it hey you have an agent that does the planning do that with the planning um and you can even tell it if I give you multiple queries you can run them in parallel as well you can you can just instruct it like that and cloud code does knows how to do that really well unfortunately in gemini you cannot do this there's no concept of agents first of all there's no idea of paralization as far as I'm concerned as far as I know so how do we actually get something like this right that mimics this workflow that we have um we're using with cloud code. So we are going to use slash command for that and I'm going to show you how to do that. Um I'm going to show you how to do that in gemini. So in gemini this is the workar around. Um so I've created this folder Gemini in the root folder and then there's another folder called commands. This is something you have to do yourself and then uh inside of the commands folder there's a deep research homl file. So the difference is that cloud code uses MD files and uh Gemini uses TOML files. Um I think the TOML files I actually prefer the TOML files for some reason you know I I'm a developer so I work a lot with like YAML files to files things like that. So I actually like this because it's it has a nice structure like you can tell it um so in this case I tell it like perform a deep research. This is the description for just think of it as this is like the agent, right? This is the orchestration that's going to happen, right? And then the prompt tells it your task is to perform deep research to answer the user query. These arguments, these arguments, we're going to pass it from Gemini CLI. Uh let me just open it here. It'll be more clear. Um so from Gemini from here, I'm going to pass it the arguments and it's going to take the arguments and it's going to say, okay, follow these specific set of instructions in the exact order. So then it's going to say step one, right? So this by the way, this little template is something I have to refine. I spent, you know, like an hour or two refining this. For some reason, like um Gemini just insisted in using its internal tool, built-in tool, and it just refused to use um my MCP tools that I've configured for this stuff. And so hopefully in this demo, nothing changed from yesterday to today and it works because I tested it. It worked really well. Um so don't be surprised if actually it doesn't work when I do test it now. Um but anyway so it says like then use the exos search MCP server. So now I tell it I have two server MCP servers. Um one is for planning specifically. So this is a tool for planning. Um and then I have another MCP tool for searching right the exo search which I just showed you. And then finally you will just summarize the stuff. This I'm not using a tool for this. I notice Gemini is really good with summarization. So I'm not going to mess with it. I'm just going to use its default uh capability to do summarization. So essentially I'm just using tools two tools here. So slashcomand basically orchestrates or gives the system some instructions um and it can you can tell it to interact with different tools. You can even pass it the arguments as well. So it should be smart enough to actually do this really well. Again, this is a concept that I haven't like really um like I only experimented with this with the with the use case I'm presenting to you here. I don't know how robust this is. If you test it and and you get some good results, let me know. Um but if you're struggling with it, also let me know and we'll see how I can help you with this. I'm very fascinated um to use this type of slice commands to to get the results that I want, but there might be some issues with it if you if you try it out yourself. Anyway, so let me see if I can now do this query. So once I have this configured, I don't need to do anything else. But the cool thing now is I can go to slash then I could go deep research and there we go. So we have this deep research command. This by the way is just the um it's just a pre like this is has to do with the um because basically what it expects is going to have like multiple folders. So if you have like like it's just the structure of the folder, right? You might have like commands for DP search. You might have commands for I don't know analyzing documents. You might have commands for like chat customer support. Whatever that may be, right? Like you will have different folders for that. That's just a structure that it's expected here. Um and it's okay. It's a good to follow that especially if you have multiple of these slash commands within the uh within within each of these like with use cases. Okay. All right. So now let's just test this. Um hopefully this actually works. All right. So I'm going to take this query here. I'm modify it a little bit. Actually it won't use the two queries. I'm just going to use one. Uh and this I'm going to tell it please plan a set of search subtask. Okay. So what's missing here now is I need to well do I need to how how would I query this? Do I need to do all of this stuff? Do I need to do all of this? How how would you query this? What's a good example for how to query this? Um I want to query this uh the same I want to do the same thing but I want to do it with slash command stuff. How do I do that? Any ideas? Yeah, this is this is going to be it can definitely break if they um but but I think like I have it on auto upgrade. So if there was some release that came out yes uh this morning it it probably will definitely screw up things and no because um Gemini still has been a little bit like all over the place for me anyway. So basically what I need to do is deep research. All right. Um let me just this stuff. I'm going to delete this. Yeah. So when you use it in the ID by the way it's not perfect like by the way this is not I'm using I'm basically using bash here but the issue is that there is no official um extension. So cloud code has this nice extension and it works really nice, right? You can paste code, you can do a lot of things. You can even paste images into into cloud code when you have like cloud code open like this, you can paste images into it. So that doesn't exist for Gemini CLI. Hopefully they fix this because it's actually quite clunky to work with here in the ID um especially if you're just using match. But anyways, so that's why you saw that it it was giving some issues here. I'm going to clear this because um I don't want that information to influence. So I'm starting again with 100% of the context, right? So I'm going to say um slash. Okay, so that's my slash command. And now all I need to give it is just the argument which is just a query, right? So I'm going to tell it uh test rumors. Maybe not the perfect query, but let's see if actually I'm curious what it's going to do. So let's do it. Yes, definitely. You got that right. All right. So, this is great. So, we're seeing that it's using the planning tool, which is the planning tool here that I gave it, right? If you can see that um use a deep research MCP server plan. I I needed to tell it specifically this this is this is something I really I don't like this this kind of like behavior because this can be a lot better. Like with cloud code, you don't need to do this stuff. Like you just tell it, hey, there's a it's just smart enough to understand that there's a tool. I have a feeling that I think cloud code specifically cloud codes uh no sorry cloud sonet 4 and cloud oppos 4 were specifically trained um you know like with identity capabilities and tool calling capabilities and things like that. So um it just knows that it has tools it's smart enough to know I don't need to tell it explicitly this this actually is not a great um
Original Description
We are doing more live sessions for our academy subs:
- GPT for Building AI Agents
- Prompt Engineering in the OpenAI Playground
- Ambient AI Agents
- ChatGPT Agents
We also have a recording of the Claude Code sub-agent workshop we did this week:
https://dair-ai.thinkific.com/
Use code YOUTUBE20 to get an extra 20% off.
#ai #tech #artificialintelligence #gemini
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
Playlist
Uploads from Elvis Saravia · Elvis Saravia · 0 of 60
← Previous
Next →
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
101 ways to solve search (by Pratik Bhavsar)
Elvis Saravia
TLDR Generation of Scientific Documents | ML Interview #1 with Isabel Cachola
Elvis Saravia
Sentiment Analysis: Key Milestones, Challenges and New Directions
Elvis Saravia
Discriminative Adversarial Search for Abstractive Summarization (by Thomas Scialom)
Elvis Saravia
Question Understanding: COVID-Q: 1,600+ Questions about COVID-19
Elvis Saravia
Getting Started with NLP
Elvis Saravia
Building tools and frameworks for large-scale social media mining (by Dr. Juan M. Banda)
Elvis Saravia
TextAttack: A Framework for Data Augmentation and Adversarial Training in NLP
Elvis Saravia
Dive into Deep Learning (Study Group): Introduction to Deep Learning | Session 1
Elvis Saravia
Dive into Deep Learning (Study Group): Multilayer Perceptrons | Session 4
Elvis Saravia
How I read and annotate ML papers
Elvis Saravia
Keep Learning ML (Session 1) | DSV, CompLex, Modern tools for emotions
Elvis Saravia
Dive into Deep Learning (Study Group): Preliminaries | Session 2
Elvis Saravia
Keep Learning ML #2 | Language-conditioned policy learning, Effective ML Testing, EagerPy
Elvis Saravia
Dive into Deep Learning (Study Group): Linear Neural Networks | Session 3
Elvis Saravia
Dive into Deep Learning (Study Group): Multilayer Perceptrons | Session 4
Elvis Saravia
Keep Learning ML #3 | Contrastively Trained Structured World Models
Elvis Saravia
Dive into Deep Learning (Study Group): Deep Learning Computation with PyTorch | Session 5
Elvis Saravia
Dive into Deep Learning (Study Group): Convolutional Neural Networks | Session 6
Elvis Saravia
Dive into Deep Learning (Study Group): Modern CNNs | Session 7
Elvis Saravia
101 ways to solve neural search with Jina
Elvis Saravia
(Hopefully-Reusable) Life Lessons for PhD Students in NLP
Elvis Saravia
How to save the world and forward your career in 5 easy steps | Women in NLP Talks
Elvis Saravia
Prompt Engineering Overview
Elvis Saravia
Getting Started with the OpenAI Playground
Elvis Saravia
LM-Guided Chain of Thought
Elvis Saravia
Elements of a Prompt
Elvis Saravia
Reasoning with Intermediate Revision and Search with LLMs #chatgpt #ai #llms #science #programming
Elvis Saravia
General Tips for Designing Prompts
Elvis Saravia
Efficient Infinite Context Transformers #ai #machinelearning #research #llms #science
Elvis Saravia
Best Practices and Lessons Learned on Synthetic Data for Language Models #ai #machinelearning #genai
Elvis Saravia
Reducing Hallucinations in Structured Outputs via RAG #chatgpt #ai #llms #programming
Elvis Saravia
Basic Prompt Examples for LLMs
Elvis Saravia
LLM In Context Recall is Prompt Dependent #llms #ai #chatgpt #machinelearning
Elvis Saravia
Zero-shot Prompting Explained
Elvis Saravia
RAG Faithfulness #llms #ai #gpt4
Elvis Saravia
Understanding LLM Settings
Elvis Saravia
Llama 3 is here! | First impressions and thoughts
Elvis Saravia
Llama 3 is Here! #ai #llms #llama3
Elvis Saravia
Microsoft introduces Phi-3 | The most capable small language model?
Elvis Saravia
Microsoft introduces Phi-3! #ai #llms #microsoft
Elvis Saravia
Make Your LLM Fully Utilize the Context #ai #llms #machinelearning
Elvis Saravia
When to Retrieve? #ai #llms #machinelearning
Elvis Saravia
Training an LLM to effectively use information retrieval
Elvis Saravia
State-of-the-art open-source LLM judges #ai #machinelearning #gpt4
Elvis Saravia
Better and Faster LLMs via Multi-token Prediction
Elvis Saravia
AlphaMath Almost Zero #ai #science #machinelearning
Elvis Saravia
SWE-Agent | An LLM-based Software Engineering Agent
Elvis Saravia
[LLM NEWS] AlphaFold 3, xLSTM, OpenAI's Model Spec, DeepSeek-V2, OpenDevin CodeAct 1.0
Elvis Saravia
LLM-powered tool for web scraping #ai #chatgpt #engineering
Elvis Saravia
Learn about LLMs in this NEW course #ai #chatgpt #engineering
Elvis Saravia
[LLM NEWS] KANs, Gemma 10M Context, OpenAI Updates?, Automatic Prompt Engineering, Tokenizer Arena
Elvis Saravia
[LLM News] GPT4-o, Project Astra, Veo, Copilot+ PCs, Gemini 1.5 Flash, Chameleon
Elvis Saravia
Enhancing Answer Selection in LLMs #ai #machinelearning #engineering
Elvis Saravia
On exploring LLMs #ai #promptengineering #chatgpt
Elvis Saravia
Transformers Can Do Arithmetic with the Right Embeddings #ai #machinelearning #engineering
Elvis Saravia
[LLM News] xAI Series B, Codestral, LLM Guide, AutoGen Course, Symbolic Chain-of-Thought
Elvis Saravia
PR-Agent #ai #gpt4 #software
Elvis Saravia
Extracting features from Claude 3 Sonnet
Elvis Saravia
Has prompt engineering been solved?
Elvis Saravia
More on: LLM Foundations
View skill →
🎓
Tutor Explanation
DeepCamp AI