AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?
📰 ArXiv cs.AI
Learn how AOR-Bench evaluates large audio language models' tendency to over-refuse pseudo-harmful queries and why it matters for safety alignment
Action Steps
- Build a test dataset with pseudo-harmful queries to evaluate LALMs' refusal mechanisms
- Run AOR-Bench to assess the over-refusal rate of LALMs
- Configure refusal mechanisms to balance safety and usability
- Test LALMs with diverse audio tasks to identify potential over-refusal scenarios
- Apply AOR-Bench findings to improve LALMs' safety alignment and reduce over-refusal
Who Needs to Know This
NLP engineers and AI safety researchers can benefit from understanding the limitations of large audio language models and how to improve their safety alignment
Key Insight
💡 Large audio language models can over-refuse pseudo-harmful queries, highlighting the need for balanced refusal mechanisms
Share This
🚨 New benchmark AOR-Bench evaluates large audio language models' over-refusal of pseudo-harmful queries 🚨
Key Takeaways
Learn how AOR-Bench evaluates large audio language models' tendency to over-refuse pseudo-harmful queries and why it matters for safety alignment
Full Article
Title: AOR-Bench: Do Large Audio Language Models Over-Refuse Pseudo-Harmful Queries?
Abstract:
arXiv:2606.21147v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) have demonstrated strong performance across a wide range of audio tasks. As they are increasingly deployed in real-world applications, ensuring their safety alignment has become more important. Although refusal mechanisms serve as a key safeguard by preventing LALMs from responding to harmful requests, they can also lead to {\em over-refusal}, where models incorrectly reject benign queries. This issue is especi
Abstract:
arXiv:2606.21147v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) have demonstrated strong performance across a wide range of audio tasks. As they are increasingly deployed in real-world applications, ensuring their safety alignment has become more important. Although refusal mechanisms serve as a key safeguard by preventing LALMs from responding to harmful requests, they can also lead to {\em over-refusal}, where models incorrectly reject benign queries. This issue is especi
DeepCamp AI