Agreement Between Large Language Models, Human Reviewers, and Authors in Evaluating STROBE Checklists for Observational Studies in Rheumatology

1. 1Collect a dataset of observational studies in rheumatology
2. 2Evaluate compliance with STROBE checklists using large language models, human reviewers, and original manuscript authors
3. 3Compare the assessments from the three groups to determine agreement
4. 4Analyze the results to identify areas where large language models can support or replace human evaluation

Data scientists, AI engineers, and researchers in the field of rheumatology can benefit from this study as it explores the potential of large language models in evaluating compliance with reporting guidelines, which can improve the efficiency and objectivity of the evaluation process

💡 Large language models can achieve high agreement with human reviewers and authors in evaluating STROBE checklists, potentially improving evaluation efficiency and objectivity

🤖 Large language models show promise in evaluating STROBE checklists for observational studies in rheumatology 📊