99.3% of ChatGPT Performance with OpenSource AI - [QLoRA paper]

Key Takeaways

so there's some big news today with the open source AI progression paper called Q Laura efficient fine tuning of quantized LMS now in order to really understand what this means we first have to understand a lot of the terminology and Tech jargon this paper uses so first things first here's a Quick Clip explaining what all these words mean what is Laura in AI you may have heard of a concept called Laura referring to Ai and large language models but what is it imagine you have a giant box of Legos you can build all kinds of things with this giant box houses cars spaceships but it's so big and heavy that it's hard to carry around and most of the time you don't need all these Legos to build what you want to build so instead you build a smaller box of your favorite most useful Legos this smaller box is easier to carry around and you can still build most of the things that you want in this analogy the giant box of Legos is like a large language model for example gpt4 it's powerful and can do lots of things but it's also big and heavy it requires a lot of computational resources to use the smaller box of Legos is like a low rank adaptation of the large language model it's a smaller lighter version of the model that's been adapted for a specific task it's not as powerful as the full model there might be some things that it can't do but it's more efficient and easier to use Laura stands for low rank adaptation low rank in this context referred to a mathematical technique used to create this smaller lighter model you can also think of low rank as just reading the highlighted parts of a book full rank would be reading the entire book and low rank would be reading just the important highlighted bits why is Laura important let's say you have a large and advanced AI model trained on recognizing all sorts of images you can fine tune it to do a related task like recognizing images of cats specifically you do that by making small adjustments to this large model you can also fine-tune it to add behaviors you want or remove behaviors you don't but this can be very expensive in terms of what computers you would need and how long it would take Laura solves this problem by making it cheap and fast to fine-tune these smaller models Laura is important because one efficiency using Laura can greatly reduce the amount of resources used to train AI models to perform these tasks two speed these lower ranked models are faster to train but also they can provide faster outputs this can be crucial in applications where results need to happen in real time three limited resources and many real world applications the devices that are available to run AI models may have limited computational power or memory your smartphone may not be able to run a large language model but a low rank adaptation can be used for specific tasks you may need for stacking and transferring low rank adaptations can be helpful for transfer learning we're a model trained on one task can be adapted to a different but related task this is much more efficient than training the large model to do something from scratch the updates and new skills learned by these low rank adaptations can also stack with other such adaptations so multiple models can benefit each other as well as the original larger model Q Laura Q Laura is a similar concept the Q stands for quantized so kelora is quantized lowering adaptation quantized refers to data compression quantization is converting a continuous range of values into a finite set of possible values imagine if you're an artist mixing paint you have an almost infinite range of colors you can create by mixing different amounts of colors together this is like a continuous signal in the real world but if you're working with a computer Graphics program it can't handle an infinite range of colors it might only allow each color component red green and blue to have one of many levels of intensity this limited set of possible colors is like a quantized signal here it can apply to reducing the number of decimal places we need to express a number for example Pi is an infinitely long number but we can use 3.14 as an approximation when doing calculations so back to the paper key lower so basically killora trains these models through a 4-bit quantized pre-trained language model into low rank adapters Laura what does this mean well it means it's now very cheap and very accessible to training your own AI models here's a blog post on hugging face about this paper the majority of this paper deals with how we store data and the most common data types it explains why training these models can be so expensive and requires such Advanced and powerful and expensive Hardware the common data types used in machine learning and what quantization means but in a nutshell what this means is you can get the latest Nvidia card on Amazon for about 1600 bucks and with that you can start training your own AI models more specifically fine-tuning them to your own needs now if you recall there was a leaked document out of Google it was the we have no moat document and in it there was a whole paragraph talking about Laura and what an incredibly powerful technique this was here's what it said it's in the paragraph titled what we missed the innovation that powered open source recent successes directly solved problems we're still struggling with paying more attention to their work could help us avoid Reinventing the wheel Laura is an incredibly powerful Technique we should probably be paying more attention to Laura works by representing model updates as low rank factorizations which reduces the size of the update matrices by a factor of up to several thousand this allows model fine-tuning at a fraction of the cost and time being able to personalize a large language model in a few hours on consumer Hardware is a big deal particularly for aspirations that involved incorporating new and diverse knowledge in real time the fact that this technology exists is under exploited inside Google even though it directly impacts some of our most ambitious projects curious to know what they mean by some of our most ambitious projects retraining models from scratch is the hard path part of what makes Laura so effective is that like other forms of fine tuning it's stackable improvements like instruction tuning can be applied and then leveraged as other contributors add-on dialogue or reasoning or tool use while the individual fine tunings are low rank there are some need not be allowing full rank updates to the model to accumulate over time this means that as new and better data sets and tasks become available the model can be cheaply kept up to date without ever having to pay the cost of a full run by contrast training giant models from scratch not only throws away the pre-training but also in the iterative improvements that have been made on top in the open source World it doesn't take long before these improvements dominate making a full retrain extremely costly basically resetting whatever improvements and advancements they've made we should be thoughtful about whether each new application or idea really needs a whole new model if we really do have major architectural improvements that preclude directly reusing model weights then we should invest in more aggressive forms of distillation that allow us to retain as much of the previous generation's capabilities as possible large models aren't more capable in the long run if we can iterate faster on smaller models Lora updates are very cheap to produce around 100 bucks for the most popular model sizes this means that almost anyone with an idea can generate one and distribute it trading times under a day are the norm at that pace it doesn't take long before the accumulative effects of All of These Fine tunings overcome starting off at a size disadvantage focusing on maintaining some of the largest models on the planet actually puts us at a disadvantage I don't know what this means for you but for me the nvidia's GeForce RTX 490 just now became on my buy list and of course it's going to be 100 written off for tax purposes because I will use it strictly for work and business related purposes and certainly not for gaming thank you for watching