Dirichlet Energy Minimization Explains In-Context Learning (Harvard)

Key Takeaways

hello community emerg and phase transition in in context learning this is now part two remember in my last video we looked at how to override knowledge in an llm within context learning and now I want to give you the result of today's video so the study here by Harvard University about in context learning of representation showed us and I quote Harvard an intriguing breakpoint that is reminescent of a second order phase transition and Howard found that this behavior is extremely Rob across the graphs of different sizes and shows here an explicit power law scaling Trend with an increasing graph size so we work here with a Transformer architecture we explore different graph sizes and we look for the perfect learning when is ICL the most efficient learning and training procedure for your llm now in my first video I showed you if we have an llm and we want to overwrite the knowledge it is not that we just touch on the not on some edges we have to maybe overwrite here compl complete subgraph sub networks here in our knowledge presentation if we make this simplified visualization here of our deep neural network connect to so let's start as we SE said Howard increasing the amount of context leads to a sudden reorganization of the internal the Transformer internal representation in accordance here with the graph connectivity and I showed you last time here they suggested llms can manipulate their internal representation in order to reflect here concept semantics that were specified entirely in in context learning great so if we accepted LMS can reorganize here the concept representation here then we AIM now how to study this Behavior we need a mathematical model we want to understand how is the context if we have a scaling so is there a continuous monot iic Improvement toward the context specified structure as context is added and it turns out new now you please refer here maybe you enjoy this study if you want to go to the edge of knowledge if you want to go to The Fringe here in EI research and please know this is research this is not just you can copy a code and this is it this is really research with a lot of some deep thinking involv but if you enjoy this this is the paper for you and I've showed you here in my last video yesterday today that if we accept that the representation are not just the token Tings but they are evolving here dynamically with contextual activation vectors within the lm's residual stream of each single layer then we understood that the self atten mechanism is here the main driver and you know this but now let's look at this and let's frame this ICL representation learning now from a different point of view because we want to build an internal understanding of it and whenever we want to build an understanding of how a system is learning we want to model this system we want to build a mathematical prediction model to understand if it is really working and here we go now now Harvard is telling us hey we tried an energy minimization process and this seems to work fine so let's follow this yeah in my simple terms is hey how to model this process mathematically now we have here this is really we look back I don't know 200 years plus to Peter gusta directly and he was German mathematician and he did something beautiful also he was looking here for a particular problem which involves finding a harmonic function give me a specific boundary condition interestingly look at this this is 100 200 years back and we use it today in the latest AI research to try to understand new mathematical models and he was applying this here to electrostatic Fields this is this is the field that we apply now and it's amazing now was that electrostatic potential is described by on a boundary of a region it extends to a potential in the interior that is harmonic when the condition the electric field is in a stable equilibrium now this electrostatic field configuration corresponds now to a state of the system of a minimal directly energy so we take this idea this 200y old idea and we appli to the latest research Harvard University on eii I I'm always fascinating to learn this yeah he himself was not so perfect but it took here some years then David Hilbert provided here real justification here under certain absum he showed yeah the this is valid therefore we can use it today if we want to learn a little bit before we jump into the explanation by Harvard I would recommend to you here if you want to look uh about investigate a little bit this smood Vector field design especially if you look here at the simple discretization of the vector directly energy this is my go-to publication that I highly recommend it is from 2020 and they introduce here exactly this uh corresponding Vector directly energy function here where U is your smooth Vector valued function on Omega and they're working here with a specific franus Norm so if you want to have an idea what we're talking this is the paper for you they go here and they explain in Chapter 2 that theoretical background so if you're really into mathematics and you say hey I really want to understand this on a deep level I would highly recommend here especially 2.2 here where they show you how easy it is to derive here the mathematical equation of the vector directly energy function this is what we are going to work with this is what we are going to calculate with and yes it has a beautiful Rich history in mathematics and theoretical physics and now we apply it in EI now just to make sure this Vector dly energy that is just defined here quantifies now the smoothness of a vector field and this is what we are talking about no just like the scalar did yeah for the scalar function and now this makes the vector directly energy and it's Associated L plus equation useful for the application requiring here a notion of a vector field smoothness so this is what we want to experiment now with the smoothest the smoothness of the vector field is you're given by certain constraints beautiful just give you this example imagine you want to fit a smooth Vector field to the red vectors you can see we have here one two three red vectors and now you have to build a fitting here a smooth Vector field here to the vectors here and they give you all the mathematical details I do not understand in detail for this particular topic but we are just luckily just looking here at the AI implementation if you want it in an EXT extreme simple idea this is just a feeling I can give you here uh imagine we have this network imagine we have here complexity a high dimen end dimensional complexity and now we want to overwrite knowledge and this is the llm that is shipped by Meto or whatever and now we want to have in context learning and we want to activate in context learning and we are aware there is a maybe a discontinuity here for this so therefore what we want want in simple picture we want to extract here a sub Network and we want to substitute this sub network with the in context learning Insight with the in context learning examples with the examples given here in the in context learning here there few short examples but they have to have here a smooth Vector field that fits into the greater Vector field representation and everything should be smooth and beautiful and everything should have here a beautiful continuity there should be no spikes or no empty region where do you have no connectivity so if you go for the real simplest I would go with this one yeah if you want to go a little bit more complicated I try with this simplification this is just me somehow trying to make this understandable if you encountered this the first time so we want to find you as smooth back to field given that we have certain boundary conditions remember the electrostatic field butly this is it so we need to find now a mathematical function that exactly describes this behavior and we have directly so we want now to find a smooth solution in our if you want complex nend dimensional knowledge manifold and it turns out this smooth solution is a minimizer of the vector directly energy if you want to see this in further detail tells as I told you great but luckily Harvard in its paper also kind of simplified this here to this representation so if you just want to focus here on the directly energy equation 1 two and three of the horard publication is here a simple entry into this topic so Harvard now tells us okay great so the measures above indicate whether the neighboring tokens are node in our graph and the ground truth graph have a similar distance or have a small distance between the representations th as the model now are learned ICL model now correctly infers the correct underlying structure what we want to see now mathematical model is now a decrease in the dly energy as I showed you so we have now a beautiful model and we can see does it work does it fit and how it calculated this now and here you have it beautifully so let's on the x-axis we have the contact length of our llm and here how calculated for and this is here a llama 3 llm and normalized de clear energy yes we have normalization factors but this would be too much detail just look at what happened so we have here in blue a mid layer layer number 18 and then here in whatever pink a layer 30 so the one of the last layers here and then in green we have here the accuracy so if you look at accuracy just look at accuracy we start here somewhere at I don't know 20% 40% we are relatively stable if we increase the context length but then at a certain point the context length starts to go up so we have here if you want your a breakpoint we have here stable and then we have kind of a linear this is here for the grid configuration I showed you in video number one yesterday if we go to the ring topology of our data representation that we want to find same structure accuracy normalized the clear energy you see just look at here the green the accuracy from 0% to 100% of our system you see there's here an extreme jump here to 100% And then more or less we are stable close to 100% accuracy this is exactly what we want to reach in our model we don't want to have to stop with the ICL in the rec system the a the augmentation part here somewhere here with this context length on examples that we provide but we want to have here of course the best accuracy that we can get isn't this interesting and now as you see there are this region that are really here where you transfer here from a real low performance of 20% and when you come out here you're above 80% so this region here is one of the most interesting region if you want to understand this if you want to model this and how tells us hey this leads us to the claim that as the amount of context is scaled there's an emergent reorganization of the internal llm representation that allows the model to perform well on our in context graph tracing task I showed you in video one so there you have it there is something absolutely fascinating happening and if the theory of har it is right we can describe this system Behavior with a mathematical model that is based here on the minimization of the directly energy of the system and they say building on the results from previous section here Howard tells us we now put forward a new hypothesis for why we are able to identify such structured representation from a model I showed you the ring the hexagon or maybe the grid representation in last video and Howard tells us we now say that the model in internally runs our llm with the in context learning internally runs here an energy minimization process in search of the correct structural representation of the data and this is mindblowing just to be sure no when we talk about energy this is not the the electricity that the energy that that a computer cluster uses no this is here a very grph specific mathematical expression of the dly energy so this is an abstract that we've chosen in a particular representation but if we do this if we see this as a minimization problem then interly the llm builds here in it internal representation remember here PCA the correct structural representation of the data set therefore we are now left with the main question here yeah of course this does not come here suddenly because if you look back here at the beginning of 2023 you had this study here the Transformer from an optimization process and they looked here into also is is possible to have an association between an energy function minimization and the deeper layer of self attention so for years there was this idea can we can we describe this in a mathematical model but of course there is one paper this is the Bible if you want this is from Yale University Spectrum and algebraic graph Theory now if you read incomplete 2019 you see it's an old paper forget it this 400 Pages if you want to know anything about spectral algebraic graph Theory here you'll find it so maybe you have a look at this this here is the link so whatever I ever had open question this was the paper to read and if we take this now and especially may I put your attention here a little bit to the spectral part here the spectral embeddings now often used to draw here a graph on a plane on a two-dimensional manifold and in many cases it will preserve here the structure of the graph look here at the publication we see here exactly here the spectral embedding and we have it a ring graph and we have it a grip graph from video one but please notice this is not the embedding that you know from the Transformer this is not an internal representation here given here the activation factors or the tza weight structures this is now a spectral embedding something completely different and Howard tells us hey notice how such spectral embeddings are similar to The representation see my video one from our MS and they have now a beautiful Theory and they say here this is in fact expected if you are a professor of mathematics at Harvard University if our energy minimization hypothesis is true because if the representations that we have seen in video one from the model really minimize here the directly energy of the system and a non degenerated mathematical detail then the first two principal components of PCA will exactly produce the spectral embeddings of our system maybe you need a minute to understand this and then in the second minute you say can you give me a mathematical proof yes in Annex B of the paper by Harvard University you have here exactly this Theory in perfect mathematical Beauty and aove so if you want to have a nice weekend you don't know what to do with your time me I recommend here Annex B of the paper if you're new to this listen if there are some simple ideas you can go to chat with for om whatever you go it doesn't really matter because those ideas are hundred of years old more or less no so a graph representation what is a graph what is a laan what is the laas Matrix of a graph The adjacency Matrix the degree Matrix then you go to the value and Vector the decomposition here and then you have here the embedding if you want so this is for beginners the easy explanation how is this all with have adjacency Matrix uh la plas Matrix EG Vector value how is this all connected if you do not want to read this 400 beautiful paper of Yale University great here I just want to show you listen this is really happening this is really happening in dependent if you have a llama 8 billion a 1 billion instructed noninstructed a small jamama 2 2 billion a 9 billion look at all the different layers it is happening the blue layers here all this going down the calculated directly energy that Harvard calculated for each and every single model is really minimizing itself and then we have here really a minimum somewhere and you see this is exactly where the learning the in context learning process starts so if you want again we have here kind of Lu durations here about kind of a stable Plateau at let's say 40% and then when we have really the directly energy minimum we have here the jump here in the performance over 80% let's see this here in a ring topology see video one if we have here 100% you see this beautiful here the energy comes down down down we have here then here kind of the minimum Plateau here and this is exactly where the the real learning is happening now and we have a satation at close to 100% so this mathematical model this is really kind of describing here the behavior of this real system where we have an llm and we have in context learning I want to know how can we optimize our in context learning how can we understand the inner working do we have a model to calculate this and to my knowledge this is the first model that Harvard gives us here to calculate exactly that we see at what context length with what modalities we can expect here from the modeling of the system where our in context learning really takes place and if we are somewhere left here you know you don't expect ICL to work at all you can calculate this for each and every model see maybe my next video but let's come back to the main question when if you want to take away something simple when does this in context structure this in context learning the structures we provide with ICL now really override the semantic prior of our llm semantic prior this means this is the model from meta this is the model from ex or whatever when will when you have your data that you want to insert or maybe you want not just to add but to alter a little bit of the reasoning conic to when does it happen this is also the reason why my last video was called how to override knowledge in an llm but now we have if you want a mathematical model Harvard academic solution but hey we are interested here in a real solution we can Implement immediately so this is the said true Howard did here exactly this the number of example that is provided here when the in context structure in context learning overwrites here the semantic priority pre-encoded knowledge of an llm so if you look at this in detail this here demonstrates yeah we have the accuracy and the number of example X and y- axis demonstrate the accuracy when give an in context task that is now in a contradiction to the what the llm Learned contradiction to the semantic prior we first observe that the model makes prediction that reflect the original semantic prior so even if we have I don't know 10 or 20 in context learning examp examples the llm will stick here with its own pre-trained reasoning although it is going down my goodness you see from 70% accuracy it's going down down down down and here at I don't know whatever this is the number of examples here we're really here the linear fall comes down and then we have also a different curve but you know what is is not interesting when does the new Inc context learning takes place and this is here the blue line so the accuracy here in pink drops very quickly as the M captures that the semantic rule that is learned is not being followed in these new ICL examples so with more ICL examples given on the prompt we see here a slow decay of the remaining semantic security and a transition to the model's Behavior as it begins to make prediction that reflect now the newly defined ordering of our ring structure this is not a blue line the ring structor is an example from video one so you see between the behavior that the llm still tries to hold on to the pre-trained value originally from meta and when you provide here now examples in your prompt when it catches up you see let's say 40% accuracy you see that there's a value of death and this is where hallucination happen like hell because the system doesn't the llm doesn't know what to do it's pre-trained knowledge the more provide you the more examples you provide you see this pre-trained knowledge is wrong I cannot apply it and the LM ask itself so what knowledge should I apply and if the new in context learning is not there yet what should the LM do it hallucinates but there's another Point look at how slowly we increase here with the number of examples 600 and we start to come close to let say 100% in the best case so you see this is now a behavior of in Contex structure overriding here the semantic PRI of an llm that is showing us my goodness ICL to be really effective look these are the number of examples you should provide so it is not at all simple easy trivial to override the knowledge the prior knowledge of your llm out of fresh out of of the meter and you know what if you are a subscriber by the way which would be great you remember that you have seen this curve before and we see a similar Behavior where I showed you in my video how to fine tuning behaves exactly in the same case but not within context learning but really when we fine tune the Alm and now you see what you want to tell me that the in context learning learning Behavior with this this value of death and the fine tuning they behave similarly yeah you know if you really want to go to the eye to the edge of research look at this particular data and can you explain this to me that the energy minimization here with the in context Learning Happens here in the dimension that really correspond to the in context structure do you know what this means can you can you get I Can Only Imagine the implications of this but yeah this is beyond today's video so let's come to an end what is here the result harbard tells us hey these results just showed you suggest now that we have here a a context scaling in our in context learning then this can unlock new capabilities of our llm and more broadly this axis may have as yet been under depreciated for improving here a large language model and absolutely fascinating so this might be a a path to better llms and maybe even a path to have a better in context learning that we know how to build in context learning prompts because we want that this scaling is happening happening and unlocking new capabilities no now at the end if you are really one or two of the of my subscribers who come to the end and still are here with me I want to give you some personal reflection and insights at this point in time I feel that if we do a pre-training of an llm this pre-training data set this pre-training knowledge that we imprint on a large language model on a Transformer model is the dominant Factor because if we then add here fine tuning you know this very tiny little um if you want sphere of fine-tuning knowledge fine-tuning information fine-tuning data new data what we really modify is just a minor really tiny little sphere and you think but this is more or less true also if you want to go with in context learning I just showed you when we have an override of the knowledge given the semantic prior so in context learning although we sort it is so easy in reality if you really examine it it is not easy at all and there are a lot of second phase Auto transfusion and you know what we do I have the feeling that we do not care about the optimization of this system you know what we do we build complexities that are outside of the core of the llm methodology and functions we build rag system that have a complexity with an indexer a re indexer then we have here the main retriever we have a pre- retriever we have a r retriever we have three agents showed you in my last video for to optimize the re system because the learning is not really happening but I have a feeling we don't touch here the real core of the problem to me currently in this time it seems we just building construct outside and we try to to get a RP to proove the accuracy to get a better model but we are just building more and more complexity outside of the core of the LM and you know this reminds me a little bit if you think about the current research in Quantum error corrections I also have a feeling there we are not really tackling here the main issue of the system so outside we build complexities that are just amazing you know but maybe in research this is not the path forward but okay let's say for this video this is end of part two part three is in the making if I get some positive I mean one or two likes of this video otherwise it will get even more complicated but what we have achieved today we have looked at a new mematic model we understand that the model now or llm internally with this hypothesis of Harvard University runs an energy minimization process we can describe here the features of the process and this energy minimization is now in search of a correct structural representation of the data and this is mindboggling I mean can you imagine this we have now a connexs between a structural data representation a neural network architecture at transform architecture and what we do on all of this complexity we run an energy minimization process and this gives us the right results I find it fascinating so if you a little bit into mathematics or a little bit more to theoretical physics you're going to enjoy the future of eii and today we only looked at in context learning there's a lot of happening currently at the pre-training of llms that I would like to show you maybe this is even more important because it turns out we are just building complexities that are not really tackling here the real problem of AI research so if you're interested maybe you're seeing you would like to subscribe and it would be great to see you in my next video