AirQA: A Comprehensive QA Dataset for AI Research with Instance-Level Evaluation

1. 1Develop a comprehensive QA dataset with instance-level evaluation
2. 2Utilize the dataset to train and evaluate LLMs based agents for question answering workflows
3. 3Apply the trained models to automate QA workflows for scientific papers
4. 4Continuously update and expand the dataset to improve model performance and adapt to new domains

AI researchers and ML engineers on a team benefit from AirQA as it provides a realistic benchmark to evaluate the capabilities of large language models (LLMs) based agents, and helps train interactive agents for question answering tasks

💡 A comprehensive and realistic benchmark is necessary to evaluate the capabilities of LLMs based agents for question answering tasks

📚 AirQA: A new QA dataset for AI research to improve question answering for scientific papers!