Fine-Tuning Wan 2.1 VACE to Swap Costumes

Links + Notes 📝 https://www.oxen.ai/blog Join Fine-Tune Fridays 🔧 https://oxen.ai/community Discord 🗿 https://discord.com/invite/s3tBEn7Ptg Use Oxen AI 🐂 https://oxen.ai/ Oxen.ai offers one click fine-tuning or fine-tunes for you! Built on top of the worlds best data versioning tool, we offer tools to automate model evals, generate synthetic data, and effortlessly fine-tune models. -- Chapters 0:00 Intro to Fine-Tune Friday: Training Wan 2.1 VACE 2:19 The Use-Case: Replacing actors’ costume after a shoot 5:19 Why do you need to train a model and such a complex Comfyui workflow? 7:29 Why a Hollywood professional fine-tunes for professional use 8:04 The Model: Wan 2.1 - VACE 10:24 Wan 2.1 - VACE’s Architecture 12:27 The reference video, aspect ratio, and video length 13:35 Tips for masking your characters and pose estimation 15:50 Training the LoRA on Oxen.ai 17:27 Preparing the training data 19:32 Fine-tuning the Wan 2.1 VACE LoRA for the costume 24:45 Plugging in the LoRA into ComfyUI and going over the flow 37:54 How to think about feeding the sampler 40:01 Tips for prompting and labeling your training data 43:36 Compositing the output back into the frame 44:48 The final results 46:15 Questions and final thoughts