Fine-Tune EmbeddingGemma: 5% to 77% RAG Accuracy (Free Colab)

Shane | LLM Implementation ยท Beginner ยท๐Ÿง  Large Language Models ยท2mo ago
๐Ÿ““ Notebook: https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/EmbeddingGemma_(300M).ipynb โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ Fine-tune embedding models 2x faster with Unsloth. This tutorial shows you how to fix your RAG retrieval by training embeddings on your own domain data. ๐Ÿ”— RESOURCES Unsloth Embedding Docs: https://docs.unsloth.ai/ EmbeddingGemma-300M: https://huggingface.co/google/embeddinggemma-300m Medical Dataset: https://huggingface.co/datasets/tomaarsen/miriad-4.4M-split โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ ๐Ÿ“š GO DEEPER: Extension Materials Annotated notebook + slides explaining the metrics, loss functions, and production configs: โ†’ Join Discord (free): https://discord.com/invite/KpnJQbgpjt โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ โฑ๏ธ TIMESTAMPS 00:00 Intro & Results Preview 00:42 Why Retrieval Quality Matters 01:24 Unsloth Features & Speed 02:09 Setup & Loading Model 02:47 Adding LoRA Adapters 03:07 Medical Dataset Prep 03:44 Baseline Model Performance 04:32 Training Configuration 05:14 Fine-Tuned Results Evaluation 05:49 Real-World Inference Test 06:09 Saving & Exporting Models 06:22 Metrics Guide & Outro โ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌโ–ฌ #unsloth #embeddings #rag #finetuning #machinelearning #llm #python #colab #tutorial
Watch on YouTube โ†— (saves to browser)
Sign in to unlock AI tutor explanation ยท โšก30

Related AI Lessons

โšก
How Prompt Context Changes LLMs (Layer by Layer)
Learn how prompt context changes internal representations in LLMs layer by layer and why it matters for improving model performance
Medium ยท Machine Learning
 โšก
How Prompt Context Changes LLMs (Layer by Layer)
Learn how prompt context affects LLMs layer by layer and why it matters for AI applications
Medium ยท LLM
 โšก
How I Built an AI-Powered Field Service Dispatch System Solo; Full Architecture
Learn how to build an AI-powered field service dispatch system solo, from sensor alerts to LLM-generated manager summaries
Medium ยท Machine Learning
 โšก
Your ChatGPT market analysis is lying to you
Identify potential biases in ChatGPT market analysis due to workflow issues, not the model itself
Medium ยท AI

Chapters (12)

Intro & Results Preview
0:42 Why Retrieval Quality Matters
1:24 Unsloth Features & Speed
2:09 Setup & Loading Model
2:47 Adding LoRA Adapters
3:07 Medical Dataset Prep
3:44 Baseline Model Performance
4:32 Training Configuration
5:14 Fine-Tuned Results Evaluation
5:49 Real-World Inference Test
6:09 Saving & Exporting Models
6:22 Metrics Guide & Outro
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch โ†’