Turn ANY FOLDER into LLM Knowledge in SECONDS

Key Takeaways

the simplest way to convert an entire code base into an llm knowledge that means an llm prompt that you can fit in an llm context window a lot of websites as part of their document started offering something called an lm. txt that works only with documentation you can't give an entire code base so in this video I'm going to show you how you can use a tool called code to prompt and then give the entire code base as an llm prompt so if you've got a model like Google Gemini with a longer context window this is Extreme helpful for you to write unit test cases documentation like whatever you want to do with the code base you can do that I'm going to show you the entire process of how you can take any GitHub repo convert them into an llm text send them to an llm and then ask some question to start with this is the tool that we're going to use which is called code to prompt it's a rust based tool I think so if you wa to install it you need to have rust available so you can also if you already have rust available you can install it from from cargo so I've got my terminal so I'm going to just do cargo install code to prompt and once I do that it technically is going to install this library from crates.io as you can see here it is getting installed while it is getting installed this is the repo that we are trying to convert into um NM Tech so this is my very old repo like I had it a few years back probably like you can see that I started the 6 years ago it's a very simple R Library that at that time was quite helpful for people to get cryptocurrency prices to do analysis at the time I'm not actively maintaining this but what we are going to try to do is we're going to use this repo convert this repo into a full-fledged llm text send it to our Google Gemini model and then ask some question either we can ask her to give a a documentation or a unit test or something that we can do with that so now we have successfully install this library at this point just type that command to see if it has been successfully installed so let's go ahead and then do code 2 prompt and as you can see here it says the following requirement arguments were not provided path that means code 2 prompt has a command successfully worked it just does not have whatever we had to give as a path so now what we can do is we can just go to this repository which is what we need to use to convert into the text llm text and we can go here and then just clone this repository I'm going to just do get clone and paste this repo so you can okay no matches for that's quite weird because I just copied the URL that's a stupid thing to do um I'm going to just go here copy this and paste it and you can see that it is copying the repo and it has copied into my current working directory so which is my home folder so if I go inside my home folder I have got a folder called coin market cap R so now what I'm going to do is I'm going to use it with the very basic setup which is just simply do code to prompt and just do coin market cap R and you can see the code has been successfully copied to the clipboard so everything that is so I can go here right now I can just go here and paste it and when I paste it as you can see here it is already available there so I'm going to just say um let me go here and then say I'm going to ask you a couple of questions about this are you ready send it and you can see that the entire thing that we have given is 101,000 tokens so 101,000 tokens very simply copied and then we have pasted into our current system prompt so and Google jini already told me yes I'm ready ask anyway okay ask away ask anyway okay I'm going to say can you give me the sample code to extract all crypto prices and make sure I can draw a bar as well first of all I'm going to see if it can understand R and it should give me the code as well so okay it is giving me the right code and uh it is giving me the code from the source code instead of just using the using the library so it is actually giving me the source code which I've written as part of the library but you can see that it understood the entire context and then it is giving us whatever we want so not every time you want to copy the code to the clipboard so you can copy the code to the clipboard and then you can save it in a notepad file or a text file but the easiest thing is to actually write a code for that so what I'm going to do is I'm going to Simply write a simple command that is going to convert the entire code into a text file an llm text file and then we're going to also save it as an output file so that it is easier for us to just use it as part of rag semantic search or even give it as part of llm text so I'm going to just do code to prompt and the repo coin Market cap R and D- output is equal to um coin um prompt. txt once I do that the code has been copied converted into a prompt and the prompt has been saved in my coin prompt. txt as you can see here the output has been saved into coin prompt. txt so that is already available for us to use so if you want you can just go see uh nanoc coin prompt. dxt and then you can see the entire content here so the entire content of the file you can just see and now you can use this txt file within your chat GPT within your rag system or if you have got like a semantic search system within your computer you can use it it'll be more effective I'm going to just exit this one more interesting thing with this is you can also get to see the number of tokens so this is also something a lot of people try to understand because if you have got a local llm setup you want to understand how many tokens your code base is going to occupy one of the easiest way to do this is if you do code to prompt and then coin market cap R and then all you have to do is you have to get the token count so you can just do dash dash tokens and then it is saying that okay we have got 85,3 194 token and uh for the chat gbt models and for the text embedding like Ada 002 and the token has been copied so this is another easy information that you can get and finally the most advanced setting that I would say is if you want to skip certain files sometimes all you want to do is you don't want to use the code prom you want to just use one particular set of files so for example this has all the r files let's say I want to just use R files I can say include is equal to dot star R so if I do this thing it will just do one R files now it's been copied I can go create a new session let me go create a new s create a prompt paste it now and um I'm going to ask some questions are you ready and you can already see here it is 16,000 tokens because we just used the r code we did not use anything else we didn't use documentation we didn't use any other file format so this is again very easy for you to take only a smaller part of the code base and then use it so overall I think this is one of the easiest ways or probably the simplest way for you to convert your entire code base into an llm prompt and this should ideally work work with any other code base as well like for example if I've got a different folder in my computer U for example recently we did the kokoro TTS implementation so I can just go here and then say code to prompt cooro and then if I do that as you can see here it's got multiple directories it took everything and everything has been copied and pasted I can go click create a new prompt paste it and once I paste it you can see that entire thing is available here so this technically works for any repo that you have got let's go here and then ask a simple question can you give me a simple description of this so I've just pasted the entire thing and I've asked it to give me a simple description okay it is breaking down breaking it down for me it's a text to speech model and it is giving me the key parts of the project and you can see different files available here and here is how it works run this thing and then it will ideally work okay fine so can can you give a me a gradio code to run this so we don't have a gradio code but because I've given this repo so it is going to actually give me a gradio code which will help us to run it as a full frontend application so in just a couple of seconds or minutes probably we managed to give an entire repo ask question about the repo and get a new code that will help us run it as a full stack application I think like you can use this creatively as much as as you want code to prompt is an amazing CLI tool that can convert any repo or any folder into an llm prompt which you can either use it as part of the current in context learning or you can save it as a text file and then use it for posterity but either way this is an amazing tool I wanted to share it with you and then let you know that this is something that you can use to improve your llm knowledge let me know what you feel about this tool if you have got any other tool that you're using for the same purpose I would love to know about it see you in another video Happy promp Day