Quantization Fundamentals with Hugging Face

Generative AI models, like large language models, often exceed the capabilities of consumer-grade hardware and are expensive to run. Compressing models through methods such as quantization makes them more efficient, faster, and accessible. This allows them to run on a wide variety of devices, including smartphones, personal computers, and edge devices, and minimizes performance degradation. Join this course to: 1. Quantize any open source model with linear quantization using the Quanto library. 2. Get an overview of how linear quantization is implemented. This form of quantization can be ap…