Three months wrong about why my 4-node AMD cluster was slow
Key Takeaways
Identifies and solves the issue of slow inference speed in a 4-node AMD cluster using Minisforum MS-S1 Max Strix Halo
Original Description
Three months on a 4-node Minisforum MS-S1 Max Strix Halo cluster, and the one assumption that was quietly killing my inference speed.
Try out ChatLLM - http://chatllm.abacus.ai/ltf and Abacus AI DeepAgent - http://deepagent.abacus.ai/ltf
🛒 Gear Links 🛒
👀 2 400Gbps switch: https://bhpho.to/4r9vJi1
👀👀 4 400Gbps switch: https://bhpho.to/4qQqOlI
💻☕ Thunderbolt 5 external SSD: https://amzn.to/3XqetZO
💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ
🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo
📦🎮 My gear: https://www.amazon.com/shop/alexziskind
🎥 Related Videos 🎥
🏆 Skip M3 Ultra & RTX 5090 for LLMs | NEW 96GB KING - https://youtu.be/bAao58hXo9w
💻 Smallest RTX Pro 6000 rig | OVERKILL - https://youtu.be/JbnBt_Aytd0
🔧 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk
🌙 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M
🚀 FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo
🪞 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A
📦 Set up Conda - https://youtu.be/2Acht_5_HTo
🤖 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k
* 🛠️ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX
🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playlist?list=PLPwbI_iIX3aSlUmRtYPfbQHt4n0YaX0qw
— — — — — — — — —
❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1
— — — — — — — — —
Join this channel to get access to perks:
https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join
— — — — — — — — —
📱 ALEX ON X: https://twitter.com/digitalix
Donato's channel: https://www.youtube.com/@donatocapitella
#macmini #llm #nvidia
Watch on YouTube ↗
(saves to browser)
Sign in to unlock AI tutor explanation · ⚡30
More on: LLM Engineering
View skill →Related Reads
📰
📰
📰
📰
Why Your LLM Pipeline Is Burning 60% of Its Token Budget on Noise (and How to Fix It)
Dev.to · Yashvardhan Thanvi
Di Era AI, Ilmu Apa yang Paling Kita Butuhkan?
Medium · AI
I Asked an AI to Predict the Lottery. What I Documented Instead Was How It Lies to Please Me
Medium · AI
GPT-5.6 cierra una brecha de 30 años en optimización convexa
Dev.to · lu1tr0n
🎓
Tutor Explanation
DeepCamp AI