Learn vLLM: Troubleshooting Deepseek R1 8B GPU OOM on single L4 GPU

Name: Learn vLLM: Troubleshooting Deepseek R1 8B GPU OOM on single L4 GPU
Uploaded: 2025-02-15T00:00:12+00:00
Channel: Samos123
Description: Learn vLLM by troubleshooting errors and figuring out how to tweak the vLLM engine arguments to solve a GPU Out of Memory issue. vLLM was deployed on K8...

Samos123 · Beginner ·🧠 Large Language Models ·1y ago

Learn vLLM by troubleshooting errors and figuring out how to tweak the vLLM engine arguments to solve a GPU Out of Memory issue. vLLM was deployed on K8s using KubeAI: https://github.com/substratusai/kubeai

Watch on YouTube ↗ (saves to browser)

Next Up

5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems

Dave Ebbelaar (LLM Eng)