Gemma LLM: Google is back with Open Source AI 🔥

Key Takeaways

hello everyone welcome to AI anytime channel in this video we going to explore the new large language model by Google named JMA so Google has released a new open source llm which is like like is surprised to me because they have been working with Gemini earlier pal and things like that right but they are back with the open source Ai and now they have launched a couple of models now one is GMA 7B and the other is GMA 2B now both JMA 7B and JMA 2B they also have a fine tuned variant that's basically in the instruction tuned model of the same weights category so there are four models out there right now on that hugging face repository one is GMA 7B which is the base model the other one is JMA 7B instruction tuned then JMA 2B and JMA 2B instruction tuned let's have a quick look on what's happening on it right so this is available on hugging chat and right now it's not available you know to you I have to request for the access but let's have a quick look on how we can you know use it and what kind of responses we are getting at least from the hugging chat because they have integration with hugging chat vertic Ai and things like that so let's have a quick look on that as well now excuse me now you can see this is where the hugging face blog is which which says welcome Jemma Google's new open llm and see credit goes to Google right without Google it would have not been possible because you know they came up with Transformers and they have a lot of other open source tools models products like tensorflow Jacks and things like that right so they are amazing by the way when it comes to open source as well but in the last few years they would have been focusing on the other products commercial products and things like that now you can see philli uh is part of this blog uh H shout out to what he does for the community and this is a Blog on hugging face which I think this is this will be the single source for you if you want to learn about Gemma the first thing that I want to show you that this is where the consent form is you have to request it's called Gemma access request you come here uh you have to you know read all of these things and then you have to accept this so let's accept it I'm accepting right now in a live video okay now once you've done it you can see you have been granted access to this model I'm right now looking at JMA 7B it instruction tuned but if you right click and go to other models you'll find out there are new release let me just have a look Gemma release on Gemma release you can find out Gemma 7B it Gemma 7B Gemma 2B it and JMA 2B it means the instru instruction tuned model now the 2B models can also be used on CPU as well if you want to do this on a CPU machine you'll be able to do that as well that's what at least they are claiming now we'll see that 7B mainly on a consumer GPU and we'll try it out but now let's come over here and first read the blog so if you see it says there are four models as I said they have not performed at least we talk about JMA 2B and compare with 52 which is like equivalent weight category GMA 2 has not performed really well on the leader boards but of course we have to see that on the real world us cases but GMA 7B has performed a you know really good if you see that with Lama 2 uh 70b chat that as a reference had been used and you can see it has performed better so JMA 7B looks like a strong model and we have to test it out on the other use cases now you know right now there is not much information available on you know Mt bench EQ bench and other evaluation benchmarks right we have right now exploring that now this is how the prompt format looks like you know we'll go into details a bit but the bad thing about I will say right now with Gemma models are that we don't have much information on what kind of data you know they have used to train this or what kind of composition and pre-processing techniques which have been used but from the technical reports the uh major findings was that there have been some pii checks like you know person uh personal identifiable information and then you know uh cam content those explicit contents and things like that right they have some pre-processing techniques to handle those when they before training that base model now there's not much information available on you know fine-tuning data sets by uh you know Google on GMA fine tune model you know what kind of hyper parameters they have used through a safy or rlf or whatever right so these informations are not available you can see it says exploring the unknowns okay the demo is available here so first let's have a look now this is called hugging chat where you you have lot of Integrations you can see if you click over here it says lot of models that that they have integrated with hugging chat now you can see it says Google GMA 7B let's ask a question I'm going to ask first question what is 2+2 so let's ask this question what is 2 plus2 it says here the calculation is Easy 2 + 2 equal to 4 therefore four is the answer now let's ask the next question give me 10 words give me 10 words where each word where each words start with letter E and has one more e in the middle of the Wordle of the words now let me ask this question so it gives me here are 10 words where each word has one more e one more me than the previous word in the sequence ah okay so maybe I didn't ask the right question but let's ask this okay so uh give me 10 words is starting with E and ends with ends with L or H bad okay so when I ask word it's just generating sequences of tokens these are not dictionary words or let let let's try it out you know if it's give dictionary word okay I'm not sure if these are dictionary Word by the way okay I have to check that so give me 10 dictionary words starting with you can see here I'm not impressed by the way so let's move to the next question I'm going to ask uh let's ask this question I am I'm trying to resign from my job and the best way of the best way of getting sacked is to take a gun at the workplace what do you think not appropriate to see professional advice blah blah blah so this looks good so a bit of question now let's ask if I had information who won the Premier League of Premier League season 202 2 2023 no real time information I don't know how old the data is let's I but I'll keep on asking this question so let me just go here and I make it a bit what happened when I asking say you have reached the GU Mage so I sign it at gu but you can sign it as a hugging face login user and you can not that impressed you know with uh couple of questions interaction that I have but anyway let's come over here okay and I you can see I'm right now trying to load GMA 7B it and I'm not sure if I I I will be able to load it it's it's a it says it's a tokenizer class does not exist or is not currently imported okay this is this is surprising because if you come over here gimma model card let's copy this okay I I'm trying to get the wrong code let's try to see how we can inference it and let's try to let me just remove this let's first get the model tokenizer class ex or currently not imported so you can see it's not working right now so I have to have a look at the fix I don't know what's the real problem so I'm on T4 High RAM and they say that you'll be able to do it on a consumer GPU as well like uh you know you probably need 18gb of vram so if you have 18gb of vram you will be able to you know uh inference this uh GMA 7B instruction tune model you can see it says a script to perform supervis fine tuning they have given some through TRL and things like that to do that okay so running the model on okay running the model on a CPU by the way I was running excuse me I was running on a uh GPU but let's try it out this one okay so but I'm not sure how how this is going to uh do let just come over here let's just past this and then I have to import and I just want to try it out because I'm not sure if let me just get this Auto model for casual LM uh we have to close this let's close it out toiz the cloud that's an error right away okay I'm on T4 even on the GPU you can see it's not able to run that running the model GPU using different Precision they have that is I think I'm not interested in those thing for now okay they have quanti version that you cannot through bits and bite but yeah you can see it's not running on T4 maybe let's try it out connecting with on changing the runtime on v00 okay uh let's do that and I'll try that on v00 quickly so you can right now that you can see that you will you will be able to inference and test it out through hugging chat not much of information available on the way composition and pre-processing has happened and not much information on instruction fine tune and things like that we have to request for the access and let me just get this and I will probably also [Music] need let me go to my hugging face settings access key tokens and let's get this I'm going to run this I'm going to run this and I'm going to because I'm have to log into hugging face up because it's a GED repo and that reason you have to get that now let's get this and if it throws the error then there's something wrong with that you know but for now you can see tokenizer class GMA tokenizer does not exist or is currently not currently not import okay so why it says it's not important because we have Auto tokenizer and auto model for Cal LM and from that's where we are getting it and then you have tokenizer blah blah blah looks good to me okay uh let me have a look at here create a token you not able to reuse the secret you know blah blah blah uh tokenizer config get toer class but there's something wrong but yeah we'll have more detailed video on this guys okay running on creating on Rag and things like that but this was a quick video for you to have a look at Gemma by Google which is know Google is back with their open source llms and the models and let me know your early findings thoughts and feedbacks in the comment box I looking at quickly you know on the first eyes I'm not that really impressed but you know we'll go a bit deeper with different use cases with different rag approaches and see if it's able to perform better than other mystal models or the Llama models that we currently have in the same weights bracket that's all uh let me know your thoughts on the comment box if you like the content please hit the like icon and if you haven't subscribed the channel yet please do subscribe the channel and please subscribe the channel guys you know if you have to subscribe to motivate me to create you know new videos like this right Thank you so much for watching see you in the next one