DisaBench: A Participatory Evaluation Framework for Disability Harms in Language Models
📰 ArXiv cs.AI
Learn to evaluate disability harms in language models using DisaBench, a participatory framework co-created with people with disabilities
Action Steps
- Identify disability harm categories using DisaBench taxonomy
- Create pairs of benign and adversarial prompts across life domains
- Evaluate language model responses using human-annotated labels
- Analyze results to detect potential disability harms
- Refine language model training data to mitigate identified harms
Who Needs to Know This
AI researchers and developers can use DisaBench to identify and mitigate disability-related harms in their language models, ensuring more inclusive and safe AI systems
Key Insight
💡 Disability harms in language models can be identified and mitigated using a participatory evaluation framework like DisaBench
Share This
🚨 Introducing DisaBench: a framework to evaluate disability harms in language models 🚨
DeepCamp AI