How to Fine-tune Mixtral 8x7B MoE on Your Own Dataset

In this video, we show you how to fine-tune Mixtral, Mistral's 8x7B MoE (Mixture of Experts) model, on your own dataset. You'll be directed to another video where we fine-tune Mistral 7B (standard Mistral) on your own dataset, but you'll be using the notebook here: https://github.com/brevdev/notebooks/blob/main/mixtral-finetune-own-data.ipynb My explanation on how QLoRA works: https://brev.dev/blog/how-qlora-works Fine-tune Mixtral MoE on a HuggingFace Dataset: https://youtu.be/zbKz4g100SQ More AI/ML notebooks: https://github.com/brevdev/notebooks/ Join the Discord: https://discord.gg/NVDyv7TUgJ Connect with me on 𝕏: https://x.com/HarperSCarroll