LLMOps
Operate LLM applications in production — evals, prompt versioning, and observability.
0%
Confidence · no data yet
After this skill you can…
- Set up LangSmith or Langfuse for LLM tracing
- Version and test prompts in a CI pipeline
- Monitor token costs, latency, and quality metrics
Prerequisites
Watch (10 videos)
Kuzu in Python: Build a No-Server GraphRAG Retriever
→ Build a local graph retriever→ Create an in-memory entity graph→ Retrieve ranked supporting context for LLM answers
What Separates a Side Project From a Real AI Product
→ Build scalable AI products→ Deploy AI models in production→ Manage LLM API costs
💡 6 Agentic AI Projects ideas to Work as AI Engineer. #agenticai
→ Deploy LLM models to production→ Implement LLM ops pipeline
Softr Vibe Coding, Docker’s AI Fix, & The Rise of Dify (Week 49)
→ Deploy AI models in production environments→ Use Docker for AI model deployment→ Monitor and maintain AI models in production
Don't Waste 2026 Competing Against PhDs (MLOps vs ML Engineer)
→ Deploy machine learning models→ Monitor and scale AI systems→ Optimize AI workflows
23. LLM Ops: Building a Quality Gate for Retrieval & Generation (Regression Detection)
→ Build a quality gate for LLM retrieval and generation→ Implement evaluation as a continuous monitoring discipline→ Detect regression in LLM system performance
Skills in LangSmith Fleet
→ Deploy skills in production environments→ Manage skill versions→ Monitor skill performance
Trace OpenRouter Calls to LangSmith — No Code Changes Needed
→ Deploy LangChain models to LangSmith→ Integrate OpenRouter with LangSmith→ Use Broadcast feature for no-code tracing
Checkpointing Flax NNX Models with Orbax (Part 1)
→ Save Flax NNX models→ Restore Flax NNX models→ Implement Orbax checkpointing
LangSmith vs Langfuse: Build vs Monitor LLM Apps Explained
→ Build LLM applications→ Monitor LLM models in production→ Optimize LLM app performance
DeepCamp AI