Introduction to Jina Embeddings V2: 8K Sequence Length Open Source Model

Key Takeaways

hello everyone welcome to AI anytime channel in this video we are going to experiment with a new embedding model Gina embedding version two so it's a new embedding model in the town uh that has been you know created by Gina AI uh it's not a new name when it comes to AI Community Gina uh is Gina AI is a Berlin based or a German based organization I will say and they have been active in the AI Community or the AI machine learning ecosystem mainly on the natural langage part and they later came up with Gina Cloud as well personally I have used Gina in few of my projects earlier even in the production level projects earlier a few years back when they had know different models and integration or some kind of orchestration Frameworks when it come to deploying applications or the microservices and they have a lot of templates so now they have come up with uh version two of embeddings model they have V1 as well and the good thing about version two is that it supports more than 8K plus you know sequence like the sequence length is around 8,192 uh to be precise so it 8K plus sequence length which tends to outperform add 002 which is by open AI you know Ada 002 embeddings tends to perform uh on some benchmarked on mte leaderboards which is for embeddings model The Benchmark leaderboard it has outperformed uh other embeddings model on few uh benchmarks and so let's let's talk about it and you can find it out the hugging face thing over here the repoat as usual you know G AI embedding some information not much of information is available about right now because they haven't released the paper while creating this video so I don't know a lot about this but we experiment this how we can try it out in collab notebook or something and then we'll see now how it's working uh it has going to be three different variants you can find out over here in the version one they had which was based on T5 and if you talk about from an architecturally standpoint and the version two which is you know based on something the based on a Bo architecture and based on Gina bir which provides more than 8K sequence the smallest one having 33 million parameters and the thing that we're going to try today is 137 you know comparatively small when you compare this with other embeddings models like BGE embeddings by Bing AI Academy or sentence Transformers embeddings model like mpet or Min LM or mult multilingual E5 or any other embeding now comparatively really small and we'll see how it performs or something like that okay now an inference code has been given we of course we going to depend on this inference code and then we'll talk about it so here is the hugging face repository the link will be given in the description you can try it out yourself now let's come back over here and uh let's go inside this folder called Gina V2 I'm going to copy this over here and let's copy I'm going to open my anaconda prompt uh yes anonda prompt you know nobody knew that uh Intel Z the Creator Anaconda is so advanced in future now we're talking about prompt engineering right so now let's go CD into it and I'm going to just open my jupyter notebook over here so let's do that jupyter notebook and it will open a notebook uh in the web browser for me it opens in Microsoft Edge and you can see here I got my uh viewer and all here the notebook so let me just name it here guys quickly I'm going to call it Gina V2 experimentation uh okay while creating this video I don't see that there is an integration with langin but uh in the next few hours or a couple of days you will see an integration with langin for sure now how do we import this guys so we have to import this from autom model class of Transformers so from Transformers there are two dependencies if you want to install few things you have to install pych torch and then you have to install Transformers these are the only two dependencies that you need and you need numpy by the way so you also need numpy so from Transformers I'm going to import a Auto Model Auto model and then I'm going to also I also need uh numpy the linear algebra module of numpy because we're going to you know I mean do some cosine similarity thing over there so we need that so from numai do linel Algebra import normalization so import norm and you can see I am you know importing I imported success fully now let me just do one thing let me create a markdown over here and let's first go in little bit of mathematics here because that's what it has given on their documentation or the code s it that they have given on the hugging F thingy here now what I'm going to do create a variable same variable that they have given cost same and they have used Lambda to execute this in a single line of code so Lambda a python keyword and then a comma B which basically these are two your two vectors now imagine if you this if you have these two vectors A and B and then that's how you go comma and then you define your uh calculations or the computations by the way so here I'm going to just do a DOT product between the both the vectors A and B we'll go in a bit of mathematics very soon now so how do we write it so a and at the rate where the button goes okay now this is for basically your Dot and then I do a transpose if it's more than n two dimensional or n dimensional where n is greater than one I will explain that now B transpose this is how the formula you define and then divided by the uh normalization of both a multiplied B okay so let's do that Norm and then go to your B thingy over here now let's understand this and if I just do a c Sim right you just print a Lambda thingy function here now let me just bring up my uh this let me just so you know how we can do that okay let me uh bring my tablet here guys okay I want to explain that a bit so uh let me open my tool cast here okay and I'll just make my mic upside here it's okay not required for now let me just okay and here okay let's write it down I don't know why it's not okay now it did cool so here the let's write it down so if you look at the formula here right what the formula that we have written so the formula is COS of cos same of the formula that we have written is of Lambda and there are two vectors A and B and this is how you write then you go your calculations and it is a dot product B in transpose basically it's used for matrix multiplication that at theate that you see right and now then divides by your normalization thingy so Norm of a and then multiplied with Norm of B that's how you define the formula now what do we mean by this guys you know if you're not familiar with this let me make it simple for you so here we have two vectors A and B is your vectors I hope you can see it at least and let's break it down step by step I can see it here as well now the first thing that you see here in the formula is your A and B do T I hope you get the taxonomy right that's what I'm explaining here now this that you see is used for matrix multiplication so if you you know if you have uh in your high school we have all you know or even in our engineer ing or wherever you were learning your graduations right you would have studied about matrix multiplication and then this B that you see is mainly your transpose that's how you define it now this entire thing that you see here that denotes that denotes the dot product between two vectors so let me just write it denote the dot products between a and b that's what it denotes now so that b do B dot that you see here that let me just cover it also B do e if it's for one dimension okay if you only talk about you know 1 D so for example for this you can say for one dimension you can only write B it is okay you don't have to write don't do you don't have to do a transpose of that because it doesn't matter both both are same B and B dot is same in this context by the way in this context that we are talking is same now if for 2D or let's write like this for ND where n equals to or not n when n not equals when is n is greater than one then it's different then it matters then it's more relevant okay that that we talk about okay now here if you talk about cosine similarity right let me just write it bigger and also a simple calculation for you let me see if I can see it now cosine simulative formula goes like this if you if you look at over here it's a dob and it's it's this so This become your dot product this becomes your dot product and then you have magnitude okay of both the vectors so this this is how you write magnitude of both the vectors right so let me just write magnitude and you would have studied in this in physics as well the scalar the vector and then we tensors okay so scalar vector and tensor okay it's really intuitive it's all mathematics that we talking here right we' been talking mathematics and let's take an example here guys now now suppose if you have a equal 1 23 and you have B = 23 4 how would you calculate and I have also shown this in my other video uh build or create a vector store from a scratch I have shown this uh mathematical intuitions behind the vector stores or the cosine similarities Etc in that video in details as maybe you can check it out I'll give the link in description now here you have two vectors A and B where you can see the elements 1 2 3 2 3 4 how do we calculate so let's first calculate the dot product over here so in this dot product how will you do guys a and this is how you would do right B and T so let's first do that okay so here one Co Plus 2 cross 3 and then + 3 cross 4 how how much is this total so this becomes two this become six this become 12 so it becomes 20 now this is your first thing that that get gives you the dot product the next thing is the norm of a so let's get Norm of a so Norm of a how will you define like this and how much is this how do you calculate it what is a value 1 2 3 so this is how you have to do you have to do it in a root here so 1 + 2 + 3 and then this becomes 4 9 uh and this becomes 14 we'll calculate that I have my phone handy where I will calculate then normalization of B and this becomes B and how much is this let's count it out what do we have here we have 2 3 4 so this becomes 2 s + 3² + 4 S this becomes 4 + 9 + 16 and this becomes 29 now let's just fit it in the formula now we are just calculating cosine similarity here so cosine similarity this becomes 20 as we got 20 and then this becomes 14 cross 29 let me just use my phone okay this came as 0.997 okay now if like it's closure to like one right and this is how you calculate so I'll give this in description now what we have done here if you look at this formula that uh you know they have given on their hugging face Lambda of a b which is two vectors A and then you have this matrix multiplication B do transpose like transpose of B and then you have normalization of A and B that you multiply and get the cosine simility so for these two vectors we got around your uh 0.997 which is very closer to one now this is what I wanted to show you from the formula stand standpoint I hope you got a very high level view about that now let's jump in and you know see that how we can you know export and all do the modeling and all here okay so let's uh come back here in the collab not notebook not collab by the way I'm doing it locally now let's load the model here so for model equals Auto model again we're going to use from pre-trained what will we do if there is no from pre-trained now we are relying on pre-train models like anything guys okay from pre-train and here let's use the Gina thing okay so there's an Hindi song as well on Gina or something okay maybe it's okay uh Gina and then this is how you get the model repository and you have to do trust remote code equals to true because you're going to use incode here the incode function and for that you need to have keep it as true so trust remote I don't know why it's not showing me when you need it the Auto intelligence or completions whatever let's make it true it will take a bit of time if you're running it for the first time it will take more time because it has to download the model WS Etc keep tokenizer the config Etc the model W and keep it in a uh your cash or anywhere you are doing it and then it will take more time and let let it uh let it happen right then meanwhile I'll just uh unplug the charge of my MO [Music] okay now let's print the model thingy here so once I print the model you can find it out it's a Gabbert model which has your uh you can see the padding the word embeddings has been used the token type embeddings the normalized layer the dropouts you know encoder everything you can find it out in this veros description of this model that you have printed now let's have an embedding variable here and in this embeddings I'm just going to do model incode so we're going to use incode function model. incode and it takes a list of like sentences that you can have so let's ask how is the weather today or something like this okay how is the weather today and the other goes like this within the same list what is the current weather let's what is the current weather uh current weather like today okay something like anything okay now here let's just print the embedding thing here you can find out uh list of array here right this in the array that numerical representation of your vectors now let's use this C seam guys okay the COS Sim that we have the formula I know very basic formula let's use cos Sim you can use different formulas as well if you want embeddings and but we are here using the same recommendation that they have given embeddings first one and the am adding second one okay let's give it like that and you can find out the score over here with 93 no uh this is this is what cosine similarity is guys you know it doesn't look at the contextual meaning of your two different sentences it's still more semantic not that advanced as well it looks still at your sentence similarities okay or the word similarities or if you come on the tokens level as well now let's create a function quickly guys you know to Define comp compute similarity just to quickly try it out four five different thing here now compute similarity and I'm going to give sentence one and I'm going to give sentence two because this will be my input as a parameters and then just use the embedding sing here so embeddings equals we already have model. incode so let's use that code from there model incode and then just pass inside a list which is sentence one and sentence two that's it okay and then you have your result thing over here so result then use the C same thing here and the cost same and then it's a function pass the Mings first and the aming second you can have n number of thing here okay that's how you write a function and let's just do a return result here okay now let's use this uh similarity here something like this similarity compute similarity and ah compute similarity and here I'm just going to pass let me show you something I love apple and then I like banana just look at the result here guys oh I have to print similarity and look at the result here like 78 okay I love Apple I like banana I mean uh that's that's the we will talk about it so I'm creating a playlist of evaluating llms and racks approaches and where I'll talk these things in detail that you should not always look at cosine similarity depends on what you want to do but here it is okay so let's look at something else as well quickly after changing it so I I have I love Apple maybe I'll just copy this and here I'll just write I like for example uh let's write I like open Ai and I'll just here write I don't like open AI so I'm being neutral here guys okay I don't like open Ai and then similarity again look at the score 89 or something right so you can find it out here what I'm trying to say so if you look at the contextual meaning then you have to look at the matrixes in some different ways okay and here let's try something last uh quickly uh let's write uh like let's write the same thing uh India is going to win the ODI World Cup 2023 and then India India might be the winner of the ODI World Cup and then just do similarity and you can out you can find it out over here right so this is how by the way it works you know uh it's not much detailed information about Gina but you should uh expect a few more videos on Gina very soon because I'm waiting for some Integrations and how we can use it in our rag approaches Etc and then evaluate the performances this video was a quick one to show you a quick 20 minutes video that how you can you know inference it understand cosine similarities Etc how it works and that might help you to be updated with different things happening in the generative AI ecosystem so the code will be available on the GitHub repository the link will be given in the description let me know if you have any thoughts feedbacks questions you can comment it down you can also reach out to me through my social media channels the links are available on YouTube banner and also on the channel about us uh if you haven't subscribed the channel yet guys please do subscribe and if you like this video please hit the like icon and share the video and Channel with your friends and to peer thank you so much for watching see you in the next one