CRAFT: Grounded Multi-Agent Coordination Under Partial Information

1. 1Formalize the problem as a multi-sender pragmatic reasoning task
2. 2Decompose the task into smaller sub-problems using a diagnostic framework
3. 3Evaluate the performance of large language models in constructing a shared 3D structure under partial information
4. 4Analyze the results to identify areas for improvement in the models' communication and coordination capabilities

AI researchers and engineers working on multi-agent systems and natural language processing can benefit from CRAFT to evaluate and improve their models' ability to coordinate and communicate effectively

💡 CRAFT provides a framework for evaluating the ability of large language models to coordinate and communicate effectively in multi-agent settings with incomplete information

🤖 Introducing CRAFT: a benchmark for evaluating pragmatic communication in large language models under partial information 📚