Spark cluster switch

Alex Ziskind · Beginner ·🧠 Large Language Models ·4mo ago

Key Takeaways

The video discusses stress-testing various devices including NVIDIA DGX Spark, Dell Pro Max GB10, ASUS Ascent GX10, and MSI Edge Expert, and explores the use of a switch, specifically the Microte CRS 804, to handle QSFP56 ports for running a cluster of these devices.

Full Transcript

To run a cluster of these sparks or GB10s, let's put them that way. Uh, that's what they are. You need some kind of switch that'll be able to handle QSFP56 ports like these. And these are some of the switches that can do it very inexpensively, let's say, compared to some of the more enterprisey switches. Well, this one just came out. This is the Microte CRS 804. Yeah, CRS 804. And this one has four 400 gig Whoa. What the hell happened there? Yeah, these come out, of course. I thought one was broken for a second. All right, these 400. And you can use breakout cables like these to basically drive two sparks per port. So that'll be 4 * 2, that's 8. So that could work theoretically.

Original Description

Stress-testing the NVIDIA DGX Spark, Dell Pro Max GB10, ASUS Ascent GX10, and MSI Edge Expert revealed the real limiter behind the “throttling” headlines. My USB-C portable hub: https://amzn.to/4kw0hrf 👀 My favorite external drive (dependable): https://amzn.to/3Os9Wi3 👀 Thunderbolt 4 dock: https://amzn.to/3yVRicC 👀 Thor on NVIDIA Marketplace: https://bit.ly/44j0acY 👀 Spark and Thor on Amazon: https://bit.ly/4pCjBpJ 👀 M4 Pro Mac Mini deal: https://amzn.to/3Mw8dNx ⚡ *Other gear I use:* https://www.amazon.com/shop/alexziskind 🎥 Related Videos 🎥 🧬🐍 Mac Studio CLUSTER vs M3 Ultra 🤯 - https://youtu.be/d8yS-2OyJhw 🧳🧰 Mini PC portable setup - https://youtu.be/4RYmsrarOSw 🍎💻 Dev setup on Mac - https://youtu.be/KiKUN4i1SeU 💸🧠 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk 🧪🔥 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M 🍏⚡ FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo 🧠📉 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A ⚡💥 Thunderbolt 5 BREAKS Apple’s Upcharge - https://youtu.be/nHqrvxcRc7o 🧠🚀 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k 🧱🖥️ Mac Mini Cluster - https://youtu.be/GBR6pHZ68Ho * 🛠️ Developer productivity Playlist - https://www.youtube.com/playlist?list=PLPwbI_iIX3aQCRdFGM7j4TY_7STfv2aXX — — — — — — — — — ❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺 Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1 Join this channel to get access to perks: https://www.youtube.com/channel/UCajiMK_CY9icRhLepS8_3ug/join — — — — — — — — — 📱LET'S CONNECT ON SOCIAL MEDIA ALEX ON TWITTER: https://twitter.com/digitalix — — — — — — — — — #dgxspark #nvidia #llm
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

The video teaches how to stress-test llm devices and how to use a switch to handle QSFP56 ports for running a cluster of these devices. It also explores the limitations of these devices and how to overcome them. The video is useful for beginners who want to learn about llm devices and cluster computing.

Key Takeaways
  1. Choose the devices to be used in the cluster
  2. Select a suitable switch to handle QSFP56 ports
  3. Use breakout cables to drive multiple devices per port
  4. Test the cluster for throttling and other limitations
💡 The Microte CRS 804 switch can be used to handle QSFP56 ports and run a cluster of llm devices, and breakout cables can be used to drive multiple devices per port.

Related AI Lessons

⚡
How We Translate 300-Page Books Using Claude Without Hitting Token Limits
Learn how to translate long documents using Claude without hitting token limits by breaking them into overlapping chunks
Dev.to · 龚旭东
 ⚡
Building HITL Feedback RAG: Embeddings, Retrieval, and Reranking
Learn to build a Human-in-the-Loop (HITL) Feedback RAG system using embeddings, retrieval, and reranking to improve model performance
Medium · AI
 ⚡
Building HITL Feedback RAG: Embeddings, Retrieval, and Reranking
Learn to build a Human-in-the-Loop (HITL) Feedback RAG system using embeddings, retrieval, and reranking to improve LLM performance
Medium · LLM
 ⚡
A simple way to test model fallbacks with RouterBase
Learn to test model fallbacks with RouterBase using a simple fallback wrapper and OpenAI-compatible API surface
Dev.to · routerbasecom
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →