When simulations look right but causal effects go wrong: Large language models as behavioral simulators

📰 ArXiv cs.AI

Large language models can simulate behavioral responses but may not accurately predict causal effects of interventions

advanced Published 6 Apr 2026

Action Steps

Evaluate the performance of large language models on simulating behavioral responses to interventions
Assess the ability of large language models to infer causal effects from natural language inputs
Consider the limitations of large language models in predicting causal effects and potential biases in the data
Develop strategies to improve the accuracy of large language models in predicting causal effects, such as using additional data or refining the models

Who Needs to Know This

Researchers and data scientists working with large language models for behavioral simulation can benefit from understanding the limitations of these models in predicting causal effects, and product managers can use this insight to inform the development of more accurate simulation tools

Key Insight

💡 Large language models may not accurately predict causal effects of interventions despite simulating behavioral responses well

Key Takeaways

Large language models can simulate behavioral responses but may not accurately predict causal effects of interventions

Full Article

Title: When simulations look right but causal effects go wrong: Large language models as behavioral simulators

Abstract:
arXiv:2604.02458v1 Announce Type: cross Abstract: Behavioral simulation is increasingly used to anticipate responses to interventions. Large language models (LLMs) enable researchers to specify population characteristics and intervention context in natural language, but it remains unclear to what extent LLMs can use these inputs to infer intervention effects. We evaluated three LLMs on 11 climate-psychology interventions using a dataset of 59,508 participants from 62 countries, and replicated th

Read full paper → ← Back to Reads