A Decade-Scale Benchmark Evaluating LLMs' Clinical Practice Guidelines Detection and Adherence in Multi-turn Conversations

1. 1Develop a dataset of multi-turn conversations related to healthcare scenarios
2. 2Implement CPGBench, an automated framework to benchmark LLMs' clinical guideline detection and adherence capabilities
3. 3Evaluate LLMs using CPGBench to identify areas of improvement
4. 4Fine-tune LLMs to enhance their ability to detect and adhere to clinical practice guidelines

This research benefits AI engineers, ML researchers, and healthcare professionals working on LLMs for healthcare applications, as it provides a framework to assess and improve the models' ability to follow clinical guidelines

💡 CPGBench provides a framework to assess and improve LLMs' ability to follow clinical guidelines, ensuring evidence-based decision-making in healthcare

💡 New benchmark CPGBench evaluates LLMs' ability to detect & adhere to clinical practice guidelines in conversations