Gemini 1.5 Pro for Code - Part 01

Sam Witteveen · Beginner ·🧠 Large Language Models ·2y ago

Key Takeaways

This video tutorial series covers using Gemini 1.5 Pro for code and building LLM Agents, with a focus on setting up CrewAI in Colab and creating simple bots.

Full Transcript

okay so in this video I'm going to look at using Gemini 1.5 Pro for doing a number of tasks around Cod and the first thing that I thought I'd take a look at is something that people were asking in the comments is could you get it to look at a repo and to basically ask it to build some examples based on that repo that kind of thing so the repo that I've chosen is crew AI some people mentioned this in the comments and also this is one of the projects that I've been meaning to make a tutorial for or much longer form tutorial for over time so basically crew AI is a framework for multi-agent apps or Bots and it uses Lang chain underneath in here you can see here there's some docks about it in the repo which I'll talk about which we'll bring in um telling you what it is telling how to install it telling it some various things that you can do with it that kind of thing as we go through so I've gone through and run this in Gemini I'm going to walk you through exactly what I did and just sort of show you I've gone back to running this as a free form prompt just because I was trying to get some of the formatting out and I found that this was nicer for that okay first of all I basically take the repo and uploaded it coming back and looking at the repo I'm not really interested in these parts I was interested in the docs not really interested in the tests although that could be useful later on and I'm interested in the actual source code for building crew AI in here I've brought that in as one of the folders that you see here this is the crew AI 30 files and the other one that I've brought in are the docks in here so this is basically just the docs of how to do various things I guess there's some images in there I didn't realize that at the time and then this is basically just sort of like the how-tos the markdown files and stuff like that so it's not really extensive docs it's just I'm not sure if they actually have the whole doc site but anyway what I've done is just bring in those two things so in total we've just got just under 50 files we're looking at around sort of 35 37,000 tokens in total so I've just put in first off tell me about crew AI what it can do so it says okay crew ai orchestrating ai Frameworks for collaborative intelligence and then it basically goes through and tells me a little bit about what it can do here's what Crea can do create and manage AI agents assign and execute tasks Foster collaboration and delegation integrate with various llms enhanced with tools and it's got some benefits it's got some different things like that so I was kind of curious at the start that I didn't see that it actually mentioned Lang chain here so I was curious to ask it okay what's it made with and sure enough J had no problems being able to look at this and say okay it's got python it's got pedantic it's got Lang chain and then by default it's got open Ai and it includes some other additional libraries for different kinds of things as well all right so what I want wanted to do was basically set this up in collab and I wanted it to do all the work and I'm going to try and do as little intervention as possible here so this is the whole idea is to see okay can we get it to basically write some stuff and get it working and stuff like that without me having to make changes I do know how crew AI works I've used it and tried it out for a few different things and stuff like that so I felt when I did this that okay yeah I've got a basic understanding of okay where it might go wrong that kind of thing so I asked it okay what package do I need to install into collab to use this and so it basically tells me that I'm going to need crew AI I'm going to need Lang chain I'm going to need Lang chain Community I'm going to need unstructured Etc and then it just gives me the nice pip installs so I thought okay let's just take those pip installs so you can see the first thing I've done is just copy those over to collab and run them all right next thing it gave me some information about that once it's installed you can do different things and stuff like that and it gave give me some additional tips okay so next up I basically ask it can you write me the code using crew AI for a simple bot that has two agents that talk to each other where one is the customer and one is the salesperson the salesperson is a great saleswoman who is selling air conditioners the customer has a chain of hotels and is thinking about upgrading them but wants to make sure he gets value for his money all right so we can kick this off and start it running and you'll see that the inference times for this actually reasonably short because we're only looking at something that's around 40,000 tokens in here okay so we can see that the code that it's given back it's basically obviously using open AI here by default but it's bringing in from crew AI import agent task crew process it's then basically set up our hotel chain owner with the information the goal for them the back story for them setting them up to be verose and our sales person here now it is funny that I specified that it was a saleswoman that the salesperson was a woman yet when it goes to write it it kind of makes it gender neutral I don't know if that's intentional or just random here so it's basically set up the task so the task is going to be that the customer expresses their need for air conditioner upgrades and their desire for value for money and we can see that the task for the salesperson is to introduce themselves inquire about the customer specific requirements Etc all right and then it's got an explanation of what it actually is doing here as we go through this okay so I've copied this over I've replaced it a little bit so I can actually get the open a I key from here and then I've just pasted in the rest of the code exactly like Gemini wrote out here and we can see if we start looking at the output of it that sure enough it starts working right that basically it decides off it's going to use a tool for this it's going to basically have a coworker action air conditioner salesperson research it's got a whole bunch of things in there it's doing this and we don't have any sort of recursive criticism Loop here going on which probably would make it better output but anyway this is a really sort of basic thing here and we can see the final answer is there are a few options to consider it's got some things about the different kinds of air conditioners and it's gone through and done these it's printed things out in yellow here which is a bit of a challenge anyway for the second task we can see that it's used the tool again that the person's introduced themselves we can see that the salesperson is basically asking for specifics about numbers of rooms numbers of floors that kind of thing and we can see that okay we've ended up with a dialogue or at least extracted out the needs and stuff like that for these particular people in here and in the end we can see that we've basically gotten the sort of thinking process of each of these agents it's it's worked as a clearly the code has worked for the agent framework in here all right so the next thing that I wanted to do is not use open AI so let's get it to use a Gemini model in here okay so you can see that I basically asked it can you make some changes to use Gemini Pro for the llm Via check Google generative AI change this code and tell me what packages I need to install via pip so it's saying okay here is the modified code it's generating out okay so it's finished generating out let's have a look at it okay so we can see that it's basically kept most of the code for this it's told us that the package that we need to install is Lang Ching Google which I don't think is exactly right but then it's also told us okay set the llm up to be chat Google generative AI to set the llm to be for Gemini Pro and stuff in here okay so it's made a number of mistakes in here or it's made a couple of small mistakes in here basically the package that we needed to inst I think is this Lang chain Google gen AI because we want to use the AI Studio One in here and then we need to actually P import that in like this chat Google generative AI in here as opposed to what it originally came out with and then basically just set up the llm like that I've changed the names to be Gemini llm in here the rest of the code is basically the same from what it's done and now we can run it and we can see sure enough it's using Gemini to basically give us outputs for this and the outputs are definitely different than open AI I should point out that the Gemini Pro that it's using here is the 1.0 version and not the 1.5 version in here all right we can see that there's a bunch of things that we can do okay so now I want to try and see can it generate something that uses a tool as well so I you know can you write me the code using PR for a simple bot that has as search agent that uses uses the tool duck Dogo to get information about new AI releases and then another agent that takes that info and rewrites it from a perspective of an AI Doomer and the AI is evil and will end the world I use the Gemini llm model all right so basically I'm trying to recreate the mainstream press for a lot of this sort of stuff they always seem to either hype things up or pull things down all right so let's see can it actually generate the code here so that we will actually get something that's using the tool as we go through this okay so I've taken the code in it seems to be using the old way of doing that so I'm just going to comment that out I'll leave it in for there and I'll update I'll just make this to be llm in there it hasn't given me the PIP for actually running this so let me just get that okay so it didn't give me the PIP to install duck. goo so I'm just installing that myself looking at the rest of the code apart from it sort of getting the sort of old way of doing the Google models the rest seems to be pretty good in this so let's run it and see I like it's basically AI doomsayer warn the world about the dangers of AI the task is search for recent news about AI releases rewrite okay so it's given them the tasks and stuff like that let's run it and see how it goes okay so we got an error out I copied it across to Gemini it looks like that basically here it needed a back story and this had left out the backstory here so let me try again so it gave me one for one of these let me try and get another one okay so it looks like it basically just left out the backstory for both of them I got Gemini to rewrite this one with a backstory and this one I just wrote myself and stuck it in there if we run it now the agent's actually working it goes and does a search for this we can see that it's getting back information about Sora from open AI about a text to speech API from Google a Gemini AI model for generative Ai and Android's new personalization features and then basically you know passes along to the doomsdayer you know who rewrites it these AI releases are just another step towards the inevitable AI takeover and we've got some nice text about how Sora will spread misinformation and propaganda etc etc etc so clearly it's been able to write something now that's using the tool of search and using you know this to actually go along so one of the things I've actually found is that it can be really interesting to basically not include docs just put in the source code and then start asking it questions about the source code and that allows you to sort of get a good understanding of what actually is going on in various code bases so I've tried this for some other code bases and I found that it was actually quite good at working out what were the key classes what were the key inputs and outputs that you would expect from different classes and different functions as you go through this so I think this is definitely a tool that you know is going to get better and better over time so obviously we've seen things like the GitHub co-pilot like duet AI for Google stuff like that these things you know are going to get a lot better and you can actually then go and ask it to write things like you know tests or other things like that and maybe even write some docs or to be honest getting it to WR docs I found not to be you know amazingly great that often you would need to tell it sort of a little bit about what the actual code is doing for it to sort of work out some understanding and then be able to write you know different docs for different parts of the code and stuff like that okay so I think I'll finish up there with this video as you can see that Gemini 1.5 Pro really can do a lot of cool things with code they'll probably lots of other things that we could have done of going through different code bases that kind of thing I will try and do another video in the future of looking at CSV files and looking at how we could do some of those things anyway as always if you've got any comments please put them in the comments below questions anything like that put them in the comments below if you found the video useful please click like And subscribe and I will talk to you in the next video bye for now

Original Description

Gemini Blog - https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/ CrewAI: https://github.com/joaomdmoura/crewAI For more tutorials on using LLMs and building Agents, check out my Patreon: Patreon: https://www.patreon.com/SamWitteveen Twitter: https://twitter.com/Sam_Witteveen 🕵️ Interested in building LLM Agents? Fill out the form below Building LLM Agents Form: https://drp.li/dIMes 👨‍💻Github: https://github.com/samwit/langchain-tutorials (updated) https://git hub.com/samwit/llm-tutorials ⏱️Time Stamps: 00:00 Intro 00:00 CrewAI 00:57 Getting the Repo into Gemini 01:59 Hey Gemini what is CrewAI? 03:30 How to set up in colab 03:46 putting it into Colab 04:05 Creating a simple 2 Agent bot 05:36 Running the bot in Colab 07:08 Adding in a Gemini model 08:53 Making a Bot with a Search Tool 09:35 Running the Search & Rewrite Bot 11:40 Using code understanding a code base
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Sam Witteveen · Sam Witteveen · 0 of 60

← Previous Next →
1 LangChain Basics Tutorial #1 - LLMs & PromptTemplates with Colab
LangChain Basics Tutorial #1 - LLMs & PromptTemplates with Colab
Sam Witteveen
2 LangChain Basics Tutorial #2 Tools and Chains
LangChain Basics Tutorial #2 Tools and Chains
Sam Witteveen
3 ChatGPT API Announcement & Code Walkthrough with LangChain
ChatGPT API Announcement & Code Walkthrough with LangChain
Sam Witteveen
4 Trying Out Flan 20B with UL2 - Working in Colab with 8Bit Inference
Trying Out Flan 20B with UL2 - Working in Colab with 8Bit Inference
Sam Witteveen
5 LangChain - Conversations with Memory (explanation & code walkthrough)
LangChain - Conversations with Memory (explanation & code walkthrough)
Sam Witteveen
6 LangChain Chat with Flan20B
LangChain Chat with Flan20B
Sam Witteveen
7 LangChain - Using Hugging Face Models locally (code walkthrough)
LangChain - Using Hugging Face Models locally (code walkthrough)
Sam Witteveen
8 PAL : Program-aided Language Models with LangChain code
PAL : Program-aided Language Models with LangChain code
Sam Witteveen
9 Building a Summarization System with LangChain and GPT-3 - Part 1
Building a Summarization System with LangChain and GPT-3 - Part 1
Sam Witteveen
10 Building a Summarization System with LangChain and GPT-3 - Part 2
Building a Summarization System with LangChain and GPT-3 - Part 2
Sam Witteveen
11 Microsoft's Visual ChatGPT using LangChain
Microsoft's Visual ChatGPT using LangChain
Sam Witteveen
12 Building a Summarization System with LangChain - Part 3 Using ChatGPT Turbo
Building a Summarization System with LangChain - Part 3 Using ChatGPT Turbo
Sam Witteveen
13 LangChain Agents - Joining Tools and Chains with Decisions
LangChain Agents - Joining Tools and Chains with Decisions
Sam Witteveen
14 Investigating Alpaca 7B - Finetuned LLaMa LLM
Investigating Alpaca 7B - Finetuned LLaMa LLM
Sam Witteveen
15 Comparing LLMs with LangChain
Comparing LLMs with LangChain
Sam Witteveen
16 Running Alpaca7B in Colab
Running Alpaca7B in Colab
Sam Witteveen
17 How to finetune your own Alpaca 7B
How to finetune your own Alpaca 7B
Sam Witteveen
18 How to make a custom dataset like Alpaca7B
How to make a custom dataset like Alpaca7B
Sam Witteveen
19 Understanding Constitutional AI - the paper and key concepts
Understanding Constitutional AI - the paper and key concepts
Sam Witteveen
20 Using Constitutional AI in LangChain
Using Constitutional AI in LangChain
Sam Witteveen
21 Talking to Alpaca with LangChain - Creating an Alpaca Chatbot
Talking to Alpaca with LangChain - Creating an Alpaca Chatbot
Sam Witteveen
22 Text-to-video-synthesis with Diffusers and Colab
Text-to-video-synthesis with Diffusers and Colab
Sam Witteveen
23 Meet Dolly the new Alpaca model
Meet Dolly the new Alpaca model
Sam Witteveen
24 Checking out the Cerebras-GPT family of models
Checking out the Cerebras-GPT family of models
Sam Witteveen
25 A Step-by-Step Guide to Fine-Tuning Your Dolly Model (tutorial)
A Step-by-Step Guide to Fine-Tuning Your Dolly Model (tutorial)
Sam Witteveen
26 Is GPT4All your new personal ChatGPT?
Is GPT4All your new personal ChatGPT?
Sam Witteveen
27 Raven - RWKV-7B RNN's LLM Strikes Back
Raven - RWKV-7B RNN's LLM Strikes Back
Sam Witteveen
28 Talk to your CSV & Excel with LangChain
Talk to your CSV & Excel with LangChain
Sam Witteveen
29 Vicuna - 90% of ChatGPT quality by using a new dataset?
Vicuna - 90% of ChatGPT quality by using a new dataset?
Sam Witteveen
30 Koala Revealed: The ChatGPT Alternative You Need to Know! 🔍
Koala Revealed: The ChatGPT Alternative You Need to Know! 🔍
Sam Witteveen
31 Running Koala for free in Colab. Your own personal ChatGPT? (tutorial)
Running Koala for free in Colab. Your own personal ChatGPT? (tutorial)
Sam Witteveen
32 BabyAGI: Discover the Power of Task-Driven Autonomous Agents!
BabyAGI: Discover the Power of Task-Driven Autonomous Agents!
Sam Witteveen
33 Auto-GPT - How to Automate a Task Based AI with GPT-4
Auto-GPT - How to Automate a Task Based AI with GPT-4
Sam Witteveen
34 Improve your BabyAGI with LangChain
Improve your BabyAGI with LangChain
Sam Witteveen
35 Generative Agents - Deep Dive and GPT-4 Recreation
Generative Agents - Deep Dive and GPT-4 Recreation
Sam Witteveen
36 GPT4ALLv2: The Improvements and Drawbacks You Need to Know!
GPT4ALLv2: The Improvements and Drawbacks You Need to Know!
Sam Witteveen
37 Dolly 2.0 by Databricks: Open for Business but is it  Ready to Impress!
Dolly 2.0 by Databricks: Open for Business but is it Ready to Impress!
Sam Witteveen
38 Red Pajama - Operation: Freeing LLaMA
Red Pajama - Operation: Freeing LLaMA
Sam Witteveen
39 Investigating Open Assistant - Models, Datasets and Addons
Investigating Open Assistant - Models, Datasets and Addons
Sam Witteveen
40 Investigating MiniGPT-4 - The Secret behind GPT-V?
Investigating MiniGPT-4 - The Secret behind GPT-V?
Sam Witteveen
41 Stable LM 3B - The new tiny kid on the block.
Stable LM 3B - The new tiny kid on the block.
Sam Witteveen
42 Bard can now code and put that code in Colab for you.
Bard can now code and put that code in Colab for you.
Sam Witteveen
43 Checking out Bark: a Text to Speech system by Suno AI
Checking out Bark: a Text to Speech system by Suno AI
Sam Witteveen
44 Fine-tuning LLMs with PEFT and LoRA
Fine-tuning LLMs with PEFT and LoRA
Sam Witteveen
45 Master PDF Chat with LangChain - Your essential guide to queries on documents
Master PDF Chat with LangChain - Your essential guide to queries on documents
Sam Witteveen
46 Using LangChain with DuckDuckGO Wikipedia & PythonREPL Tools
Using LangChain with DuckDuckGO Wikipedia & PythonREPL Tools
Sam Witteveen
47 Building Custom Tools and Agents with LangChain (gpt-3.5-turbo)
Building Custom Tools and Agents with LangChain (gpt-3.5-turbo)
Sam Witteveen
48 StableVicuna: The New King of Open ChatGPTs?
StableVicuna: The New King of Open ChatGPTs?
Sam Witteveen
49 WizardLM: Evolving Instruction Datasets to Create a Better Model
WizardLM: Evolving Instruction Datasets to Create a Better Model
Sam Witteveen
50 LaMini-LM - Mini Models Maxi Data!
LaMini-LM - Mini Models Maxi Data!
Sam Witteveen
51 Finding the Best Free ChatGPT
Finding the Best Free ChatGPT
Sam Witteveen
52 MPT-7B - The First Commercially Usable Fully Trained LLaMA Style Model
MPT-7B - The First Commercially Usable Fully Trained LLaMA Style Model
Sam Witteveen
53 LangChain Retrieval QA Over Multiple Files with ChromaDB
LangChain Retrieval QA Over Multiple Files with ChromaDB
Sam Witteveen
54 LangChain Retrieval QA with Instructor Embeddings & ChromaDB for PDFs
LangChain Retrieval QA with Instructor Embeddings & ChromaDB for PDFs
Sam Witteveen
55 LangChain + Retrieval Local LLMs for Retrieval QA - No OpenAI!!!
LangChain + Retrieval Local LLMs for Retrieval QA - No OpenAI!!!
Sam Witteveen
56 Transformers Agent - Is this Hugging Face's LangChain Competitor?
Transformers Agent - Is this Hugging Face's LangChain Competitor?
Sam Witteveen
57 StarCoder - The LLM to make you a coding star?
StarCoder - The LLM to make you a coding star?
Sam Witteveen
58 Testing Starcoder for Reasoning with PAL
Testing Starcoder for Reasoning with PAL
Sam Witteveen
59 The New Wizards - Unfiltered & Unaligned
The New Wizards - Unfiltered & Unaligned
Sam Witteveen
60 Camel + LangChain for Synthetic Data & Market Research
Camel + LangChain for Synthetic Data & Market Research
Sam Witteveen

This video series teaches how to use Gemini 1.5 Pro for code and build LLM Agents, covering topics such as setting up CrewAI in Colab and creating simple bots. It's a beginner-friendly tutorial that provides hands-on experience with LLMs and AI agents.

Key Takeaways
  1. Set up CrewAI in Colab
  2. Create a simple 2 Agent bot
  3. Add a Gemini model to the bot
  4. Make a Bot with a Search Tool
  5. Run the Search & Rewrite Bot
  6. Use code understanding to analyze a code base
💡 The key insight from this tutorial is that LLMs can be used to build powerful AI agents that can understand and generate code, and that Gemini 1.5 Pro provides a robust platform for building these agents.

Related Reads

📰
Top AI Papers on Hugging Face - 2026-07-23
Discover the top AI papers on Hugging Face, featuring trends in world models, diffusion, and agents, to stay updated on the latest advancements in AI research
Dev.to · Y Hành Nhan
📰
Future of AI: How Artificial Intelligence Will Transform Our Lives in the Next Decade
Discover how AI will transform lives in the next decade, from work to communication, and why it matters for professionals and individuals alike
Medium · ChatGPT
📰
BrainStem: A Neuro‑Symbolic System That Learns Language Through Structured Cognition
Learn how BrainStem, a neuro-symbolic system, learns language through structured cognition, offering a new approach to AI language learning
Dev.to AI
📰
Building LORE: An AI Native Platform for Story Development | Week 1
Learn how to build an AI-native platform for story development, transforming ideas into production-ready story universes
Dev.to AI

Chapters (12)

Intro
CrewAI
0:57 Getting the Repo into Gemini
1:59 Hey Gemini what is CrewAI?
3:30 How to set up in colab
3:46 putting it into Colab
4:05 Creating a simple 2 Agent bot
5:36 Running the bot in Colab
7:08 Adding in a Gemini model
8:53 Making a Bot with a Search Tool
9:35 Running the Search & Rewrite Bot
11:40 Using code understanding a code base
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →