Benchmarking Large Language Models on Floating-Point Error Classification

1. 1Build a benchmarking framework using InterFLOPBench to evaluate LLMs on floating-point error classification
2. 2Run experiments on 14 LLMs to assess their performance on six categories of floating-point errors
3. 3Configure the evaluation framework to treat floating-point error detection as a classification task
4. 4Test the LLMs on 1,130 test samples across 90 C kernels
5. 5Apply the benchmarking results to identify areas for improvement in LLMs' floating-point error classification capabilities

Software engineers, AI researchers, and developers working on numerical computations and high-performance computing can benefit from this benchmarking framework to evaluate and improve LLMs' performance on floating-point error classification

💡 InterFLOPBench provides a comprehensive benchmark for evaluating LLMs on floating-point error classification, enabling the development of more reliable and stable numerical computations

🚀 Benchmarking LLMs for floating-point error classification: a new framework for evaluating numerical stability and reliability in software code 💻