Model Monitoring
Detect data drift, model degradation, and trigger retraining.
0%
Confidence · no data yet
After this skill you can…
- Set up drift detection with Evidently AI
- Define and monitor SLAs for model performance
- Build a retraining trigger pipeline
Prerequisites
Watch (10 videos)
Monitor your AI applications in production using W&B Weave
→ Monitor AI application performance in production→ Track quality metrics with W&B Weave
Automate, Validate, and Promote ML Models Safely
→ Monitor model performance→ Detect model drift
MLOps Essentials: Enabling CloudWatch Logging & Monitoring for AWS ML APIs
→ Enable CloudWatch logging for AWS ML APIs→ Monitor machine learning model performance
Production monitoring for AI applications using W&B Weave
→ Create online evaluations for AI applications→ Monitor AI application quality over time
Model Monitoring for Generative AI applications
→ Monitor LLM performance→ Implement model monitoring techniques
Model Monitoring for LLMs
→ Monitor LLMs for performance and accuracy→ Evaluate LLMs using industry expert techniques
[Evals Workshop] Mastering AI Evaluation: From Playground to Production
→ Monitor AI application performance in real-world scenarios→ Implement logging and feedback systems
Orchestrate, Analyze, and Evaluate AI Deployments
→ Analyze telemetry data→ Investigate error spikes
Production ML on AWS: Monitoring, Troubleshooting, and Cost Optimization
→ Troubleshoot errors with log streams
Optimizing AI Applications for Production with Observability | OD548
→ Analyze AI component performance→ Capture token throughput metrics
DeepCamp AI