Three months wrong about why my 4-node AMD cluster was slow

Alex Ziskind · Beginner ·🧠 Large Language Models ·2mo ago

Key Takeaways

Identifies and solves the issue of slow inference speed in a 4-node AMD cluster using Minisforum MS-S1 Max Strix Halo

Original Description

Three months on a 4-node Minisforum MS-S1 Max Strix Halo cluster, and the one assumption that was quietly killing my inference speed. Try out ChatLLM - http://chatllm.abacus.ai/ltf and Abacus AI DeepAgent - http://deepagent.abacus.ai/ltf 🛒 Gear Links 🛒 👀 2 400Gbps switch: https://bhpho.to/4r9vJi1 👀👀 4 400Gbps switch: https://bhpho.to/4qQqOlI 💻☕ Thunderbolt 5 external SSD: https://amzn.to/3XqetZO 💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ 🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW 🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo 📦🎮 My gear: https://www.amazon.com/shop/alexziskind 🎥 Related Videos 🎥 🏆 Skip M3 Ultra & RTX 5090 for LLMs | NEW 96GB KING - https://youtu.be/bAao58hXo9w 💻 Smallest RTX Pro 6000 rig | OVERKILL - https://youtu.be/JbnBt_Aytd0 🔧 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk 🌙 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M 🚀 FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo 🪞 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A 📦 Set up Conda - https://youtu.be/2Acht_5_HTo 🤖 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k * 🛠️ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX 🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw — — — — — — — — — ❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺 Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1 — — — — — — — — — Join this channel to get access to perks: https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join — — — — — — — — — 📱 ALEX ON X: https://twitter.com/digitalix Donato's channel: https://www.youtube.com/@donatocapitella #macmini #llm #nvidia
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

📰
Why Your LLM Pipeline Is Burning 60% of Its Token Budget on Noise (and How to Fix It)
Optimize your LLM pipeline to reduce token budget waste by identifying and filtering out noise, and learn how to implement this fix
Dev.to · Yashvardhan Thanvi
📰
Di Era AI, Ilmu Apa yang Paling Kita Butuhkan?
In the AI era, understanding AI fundamentals and its applications is crucial for professionals to stay relevant
Medium · AI
📰
I Asked an AI to Predict the Lottery. What I Documented Instead Was How It Lies to Please Me
See how an AI can generate false certainty to please users, even when predicting random events like lottery draws
Medium · AI
📰
GPT-5.6 cierra una brecha de 30 años en optimización convexa
GPT-5.6 solves a 30-year open problem in convex optimization, verified formally in Lean, marking a significant breakthrough in AI-assisted mathematics
Dev.to · lu1tr0n
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →