Introducing IndQA
📰 OpenAI News
OpenAI introduces IndQA, a benchmark for evaluating AI systems on Indian culture and languages
Action Steps
- Evaluate AI models using IndQA to assess their understanding of Indian culture and languages
- Use IndQA to identify areas for improvement in AI models' language capabilities
- Collaborate with domain experts to create similar benchmarks for other languages and regions
Who Needs to Know This
NLP researchers and AI engineers on a team can benefit from IndQA to improve their models' language capabilities, particularly for Indian languages and cultural contexts
Key Insight
💡 IndQA is designed to evaluate AI models' ability to understand and reason about culturally nuanced topics in Indian languages
Share This
🚀 OpenAI introduces IndQA, a new benchmark for evaluating AI systems on Indian culture and languages #AI #NLP
Key Takeaways
OpenAI introduces IndQA, a benchmark for evaluating AI systems on Indian culture and languages
Full Article
# Introducing IndQA | OpenAI
[Skip to main content](https://openai.com/index/introducing-indqa#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Log in[Try ChatGPT(opens in a new window)](https://chatgpt.com/)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
[Try ChatGPT(opens in a new window)](https://chatgpt.com/)Login
OpenAI
Table of contents
* [How it works](https://openai.com/index/introducing-indqa#how-it-works)
* [How we built IndQA](https://openai.com/index/introducing-indqa#how-we-built-indqa)
* [Example questions](https://openai.com/index/introducing-indqa#example-questions)
* [Improvements over time](https://openai.com/index/introducing-indqa#improvements-over-time)
* [The experts behind IndQA](https://openai.com/index/introducing-indqa#the-experts-behind-indqa)
* [Next steps](https://openai.com/index/introducing-indqa#next-steps)
November 3, 2025
[Research](https://openai.com/news/research/)[Release](https://openai.com/research/index/release/)
# Introducing IndQA
A new benchmark for evaluating AI systems on Indian culture and languages.
Loading…
Share
Our mission is to make AGI benefit all of humanity. If AI is going to be useful for everyone, it needs to work well across languages and cultures. About 80 percent of people worldwide do not speak English as their primary language, yet most existing benchmarks that measure non-English language capabilities fall short.
Existing multilingual benchmarks like [MMMLU(opens in a new window)](https://huggingface.co/datasets/openai/MMMLU) are now saturated—top models cluster near high scores—which make them less useful for measuring real progress. In addition, current benchmarks mostly focus on translation or multiple-choice tasks. They don’t adequately capture what really matters for evaluating an AI system’s language capabilities—understanding context, culture, history, and the things that matter to people where they live.
That’s why we built **IndQA**, a new benchmark designed to evaluate how well AI models understand and reason about questions that matter in Indian languages, across a wide range of cultural domains. While our aim is to create similar benchmarks for other languages and regions, India is an obvious starting point. India has about a billion people who don’t use English as their primary language, 22 official languages (including at least seven with over 50 million speakers), and is ChatGPT’s second largest market.
This work is part of our ongoing commitment to improve our products and tools for Indian users, and to make our technology more accessible throughout the country.
## How it works
IndQA evaluates knowledge and reasoning about Indian culture and everyday life in Indian languages. It spans 2,278 questions across 12 languages and 10 cultural domains, created in partnership with 261 domain experts from across India. Unlike existing benchmarks like MMMLU and MGSM, it is designed to probe culturally nuanced, reasoning-heavy tasks that existing evaluations struggle to capture.
IndQA covers a broad range of culturally relevant topics, such as **Architecture & Design, Arts & Culture, Everyday Life, Food & Cuisine,
[Skip to main content](https://openai.com/index/introducing-indqa#main)
[](https://openai.com/)
* [Research](https://openai.com/research/index/)
* Products
* [Business](https://openai.com/business/)
* [Developers](https://openai.com/api/)
* [Company](https://openai.com/about/)
* [Foundation(opens in a new window)](https://openaifoundation.org/)
Log in[Try ChatGPT(opens in a new window)](https://chatgpt.com/)
* Research
* Products
* Business
* Developers
* Company
* [Foundation(opens in a new window)](https://openaifoundation.org/)
[Try ChatGPT(opens in a new window)](https://chatgpt.com/)Login
OpenAI
Table of contents
* [How it works](https://openai.com/index/introducing-indqa#how-it-works)
* [How we built IndQA](https://openai.com/index/introducing-indqa#how-we-built-indqa)
* [Example questions](https://openai.com/index/introducing-indqa#example-questions)
* [Improvements over time](https://openai.com/index/introducing-indqa#improvements-over-time)
* [The experts behind IndQA](https://openai.com/index/introducing-indqa#the-experts-behind-indqa)
* [Next steps](https://openai.com/index/introducing-indqa#next-steps)
November 3, 2025
[Research](https://openai.com/news/research/)[Release](https://openai.com/research/index/release/)
# Introducing IndQA
A new benchmark for evaluating AI systems on Indian culture and languages.
Loading…
Share
Our mission is to make AGI benefit all of humanity. If AI is going to be useful for everyone, it needs to work well across languages and cultures. About 80 percent of people worldwide do not speak English as their primary language, yet most existing benchmarks that measure non-English language capabilities fall short.
Existing multilingual benchmarks like [MMMLU(opens in a new window)](https://huggingface.co/datasets/openai/MMMLU) are now saturated—top models cluster near high scores—which make them less useful for measuring real progress. In addition, current benchmarks mostly focus on translation or multiple-choice tasks. They don’t adequately capture what really matters for evaluating an AI system’s language capabilities—understanding context, culture, history, and the things that matter to people where they live.
That’s why we built **IndQA**, a new benchmark designed to evaluate how well AI models understand and reason about questions that matter in Indian languages, across a wide range of cultural domains. While our aim is to create similar benchmarks for other languages and regions, India is an obvious starting point. India has about a billion people who don’t use English as their primary language, 22 official languages (including at least seven with over 50 million speakers), and is ChatGPT’s second largest market.
This work is part of our ongoing commitment to improve our products and tools for Indian users, and to make our technology more accessible throughout the country.
## How it works
IndQA evaluates knowledge and reasoning about Indian culture and everyday life in Indian languages. It spans 2,278 questions across 12 languages and 10 cultural domains, created in partnership with 261 domain experts from across India. Unlike existing benchmarks like MMMLU and MGSM, it is designed to probe culturally nuanced, reasoning-heavy tasks that existing evaluations struggle to capture.
IndQA covers a broad range of culturally relevant topics, such as **Architecture & Design, Arts & Culture, Everyday Life, Food & Cuisine,
DeepCamp AI