Towards Self-Evolving Benchmarks: Synthesizing Agent Trajectories via Test-Time Exploration under Validate-by-Reproduce Paradigm

1. 1Identify the limitations of existing agent benchmarks
2. 2Propose the TRACE framework for synthesizing agent trajectories
3. 3Implement test-time exploration under the Validate-by-Reproduce paradigm
4. 4Evaluate the effectiveness of the TRACE framework in creating self-evolving benchmarks

AI researchers and engineers benefit from this framework as it enables the creation of more challenging and dynamic benchmarks for evaluating agent abilities, allowing them to better assess and improve their models

💡 The TRACE framework enables the creation of more challenging and dynamic benchmarks for evaluating agent abilities

💡 Introducing TRACE: a framework for self-evolving benchmarks via test-time exploration