SGLang Serving Tutorial: Build Structured Agentic LLM Applications

Ready Tensor · Intermediate ·🧠 Large Language Models ·6mo ago

About this lesson

In this video, we explore SGLang, a powerful LLM serving framework designed for structured and agentic applications. You’ll see how SGLang compares to vLLM, when it makes sense to use it, and why it shines in use cases with predefined input/output structures. We walk through running SGLang with LLaMA 3.2 1B, attaching LoRA adapters, defining structured generation functions, and benchmarking performance against a baseline. You’ll learn how to: Install and run an SGLang server with Hugging Face models Serve multiple models simultaneously (base + LoRA adapters) Use structured generation functions for agentic workflows Build summarization and sentiment analysis pipelines with strict output constraints Track generation state across multiple steps Use the OpenAI-compatible endpoint exposed by SGLang Compare base vs LoRA-adapted model outputs Benchmark serving performance (TTFT, TPO, ITL) and analyze results Key timestamps: 0:00 - What is SGLang and how it compares to vLLM 0:47 - Launching the SGLang server with LLaMA and LoRA adapters 1:34 - Structured generation functions for agentic use cases 2:25 - Building a sentiment analysis pipeline with constrained outputs 5:37 - Running the sentiment classifier and inspecting state 6:03 - Using the OpenAI-compatible endpoint 7:09 - Benchmarking performance and interpreting results This video is ideal if you’re building agentic systems, structured LLM pipelines, or high-throughput serving backends where performance and control matter. This video is part of the LLM Engineering and Deployment Certification Program by Ready Tensor. Enroll now: https://app.readytensor.ai/certifications/llm-engineering-and-deployment-DAROCXlj About Ready Tensor: Ready Tensor helps AI and ML professionals build, evaluate, and showcase intelligent systems through certifications, competitions, and real-world project publications. Learn more: https://www.readytensor.ai/ Like the video? Subscribe for more content on LLM serving, agentic sys

Original Description

In this video, we explore SGLang, a powerful LLM serving framework designed for structured and agentic applications. You’ll see how SGLang compares to vLLM, when it makes sense to use it, and why it shines in use cases with predefined input/output structures. We walk through running SGLang with LLaMA 3.2 1B, attaching LoRA adapters, defining structured generation functions, and benchmarking performance against a baseline. You’ll learn how to: Install and run an SGLang server with Hugging Face models Serve multiple models simultaneously (base + LoRA adapters) Use structured generation functions for agentic workflows Build summarization and sentiment analysis pipelines with strict output constraints Track generation state across multiple steps Use the OpenAI-compatible endpoint exposed by SGLang Compare base vs LoRA-adapted model outputs Benchmark serving performance (TTFT, TPO, ITL) and analyze results Key timestamps: 0:00 - What is SGLang and how it compares to vLLM 0:47 - Launching the SGLang server with LLaMA and LoRA adapters 1:34 - Structured generation functions for agentic use cases 2:25 - Building a sentiment analysis pipeline with constrained outputs 5:37 - Running the sentiment classifier and inspecting state 6:03 - Using the OpenAI-compatible endpoint 7:09 - Benchmarking performance and interpreting results This video is ideal if you’re building agentic systems, structured LLM pipelines, or high-throughput serving backends where performance and control matter. This video is part of the LLM Engineering and Deployment Certification Program by Ready Tensor. Enroll now: https://app.readytensor.ai/certifications/llm-engineering-and-deployment-DAROCXlj About Ready Tensor: Ready Tensor helps AI and ML professionals build, evaluate, and showcase intelligent systems through certifications, competitions, and real-world project publications. Learn more: https://www.readytensor.ai/ Like the video? Subscribe for more content on LLM serving, agentic sys
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Related Reads

📰
RAG in Laravel: Embeddings and pgvector for a Knowledge-Base Bot
Learn to integrate RAG in Laravel using embeddings and pgvector for a knowledge-base bot, improving its ability to answer questions about your specific data
Dev.to AI
 📰
The AI That Can Re-Write Its Own Brain: Why Inkling is the New Frontier for Open Weights
Learn about Inkling, the AI that can re-write its own brain, and why it's a game-changer for open weights
Hackernoon
 📰
A Beginner's Guide to the Seedream-5-pro Model by Bytedance: What You Need to Know
Learn about Seedream-5-pro, Bytedance's text-to-image and image-to-image generation model, and its capabilities
Hackernoon
 📰
Kimi K3, and what we can still learn from the pelican benchmark
Learn about Kimi K3, a 2.8 trillion parameter AI model, and its potential impact on the field, plus insights from the pelican benchmark
Simon Willison's Blog

Chapters (7)

What is SGLang and how it compares to vLLM
0:47 Launching the SGLang server with LLaMA and LoRA adapters
1:34 Structured generation functions for agentic use cases
2:25 Building a sentiment analysis pipeline with constrained outputs
5:37 Running the sentiment classifier and inspecting state
6:03 Using the OpenAI-compatible endpoint
7:09 Benchmarking performance and interpreting results
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →