Foundation Models Explained | ft. Raja Iqbal

Key Takeaways

[Music] Foundation models are large language models that are trained up on massive amounts of well not necessarily public data but some of those that available they are actually well they are called Foundation model and uh you know GPT series models gpt2 gpt3 GP 3.5 3 or 4.0 uh all of these are Foundation models and each of them is good with different different tasks so when we talk about this large language model if someone next time says you know what is a foundation model well some Foundation um Foundation models are actually a large language model is essentially as a as a is a foundation model um some of the foundation models that are there U are you know there is openi entropic is there um uh Facebook recently had I think uh llama 2 should be here hugging face has one replicate has won this space is also um I would not say very crowded I mean there are some key players and the um and the barrier in this is um likely there's going to be some degree of Monopoly in this space because the amount of compute it takes it is non-trivial uh uni or small smaller companies cannot afford to actually train Foundation models from scratch and the way Foundation models are used as uh I will explain when when I get to the in context learning idea that since um none of us have that those resources to actually create those Foundation models from scratch uh we actually rely on the foundation models for their language uh capabilities or the semantic capabilities and we bring in the external sources of knowledge and somehow combine those uh that context or the knowledge with the language capabilities so it speaks like a human while it has some uh the model has been augmented with some context