JavaVulBench: A Java Vulnerability Benchmark with Realistic Splits, a Unified Multi-Backend Harness, and a Leakage-Aware Evaluation Mode
📰 ArXiv cs.AI
Learn about JavaVulBench, a benchmark dataset and evaluation harness for Java vulnerability detection, and how to apply it to improve vulnerability detection models
Action Steps
- Download the JavaVulBench dataset and evaluation harness from the arXiv repository
- Configure the harness to use the desired split strategy and backend
- Train a vulnerability detection model using the dataset and evaluate its performance using the harness
- Compare the performance of different models and split strategies to identify the most effective approach
- Apply the findings to improve the accuracy and robustness of Java vulnerability detection models
Who Needs to Know This
Security researchers and developers on a team can benefit from JavaVulBench to evaluate and improve their Java vulnerability detection models
Key Insight
💡 JavaVulBench provides a comprehensive benchmark dataset and evaluation harness for Java vulnerability detection, enabling researchers and developers to evaluate and improve their models
Share This
🚨 Improve Java vulnerability detection with JavaVulBench! 🚨
Key Takeaways
Learn about JavaVulBench, a benchmark dataset and evaluation harness for Java vulnerability detection, and how to apply it to improve vulnerability detection models
Full Article
Title: JavaVulBench: A Java Vulnerability Benchmark with Realistic Splits, a Unified Multi-Backend Harness, and a Leakage-Aware Evaluation Mode
Abstract:
arXiv:2607.02825v1 Announce Type: cross Abstract: We release \textsc{JavaVulBench}, a benchmark dataset and evaluation harness for Java vulnerability detection. The dataset contains $\sim$30{,}600 Java methods spanning 1{,}740 CVEs and 700+ projects, labelled at both method and line granularity, with per-CVE publication dates and five realistic split strategies: random, project-disjoint, temporal, deduplicated, and unseen CWE-family. The harness provides a single \texttt{LlmPrediction} schema ac
Abstract:
arXiv:2607.02825v1 Announce Type: cross Abstract: We release \textsc{JavaVulBench}, a benchmark dataset and evaluation harness for Java vulnerability detection. The dataset contains $\sim$30{,}600 Java methods spanning 1{,}740 CVEs and 700+ projects, labelled at both method and line granularity, with per-CVE publication dates and five realistic split strategies: random, project-disjoint, temporal, deduplicated, and unseen CWE-family. The harness provides a single \texttt{LlmPrediction} schema ac
DeepCamp AI