Model Monitoring
Detect data drift, model degradation, and trigger retraining.
0%
Confidence · no data yet
After this skill you can…
- Set up drift detection with Evidently AI
- Define and monitor SLAs for model performance
- Build a retraining trigger pipeline
Prerequisites
Watch (10 videos)
Monitor your AI applications in production using W&B Weave
→ Monitor AI application performance in production→ Track quality metrics with W&B Weave
MLOps Essentials: Enabling CloudWatch Logging & Monitoring for AWS ML APIs
→ Enable CloudWatch logging for AWS ML APIs→ Monitor machine learning model performance
Production monitoring for AI applications using W&B Weave
→ Create online evaluations for AI applications→ Monitor AI application quality over time
Model Monitoring for Generative AI applications
→ Monitor LLM performance→ Implement model monitoring techniques
Model Monitoring for LLMs
→ Monitor LLMs for performance and accuracy→ Evaluate LLMs using industry expert techniques
Model Performance Monitoring and Why You Need it Yesterday // Amit Paka // MLOps Coffee Sessions #42
→ Monitor model performance in production→ Detect model drift and concept drift
How Benchling's Team Actually Looks at AI Traces | Max Agency
→ Track AI model traces→ Analyze AI model performance
Shreya Shankar: Example of sudden data drift
→ Detect data drift→ Validate data schemas
Production ML on AWS: Monitoring, Troubleshooting, and Cost Optimization
→ Troubleshoot errors with log streams
Optimizing AI Applications for Production with Observability | OD548
→ Analyze AI component performance→ Capture token throughput metrics
DeepCamp AI