Why We Think
📰 Lilian Weng's Blog
Understanding how models think and utilize test-time compute can improve performance
Action Steps
- Review recent developments in test-time compute and CoT
- Understand how thinking in tokens and continuous space can improve model performance
- Explore the use of latent variables and scaling laws for thinking time
Who Needs to Know This
AI engineers and researchers can benefit from this post as it discusses recent developments in using test-time compute and Chain-of-thought (CoT) to improve model performance
Key Insight
💡 Enabling models to think for longer can lead to significant improvements in performance
Share This
💡 Improving model performance with test-time compute and Chain-of-thought (CoT)
Key Takeaways
Understanding how models think and utilize test-time compute can improve performance
Full Article
Published Time: 2025-05-01T00:00:00Z
# Why We Think | Lil'Log
[Lil'Log](https://lilianweng.github.io/ "Lil'Log (Alt + H)")
* |
* [Posts](https://lilianweng.github.io/ "Posts")
* [Archive](https://lilianweng.github.io/archives "Archive")
* [Search](https://lilianweng.github.io/search/ "Search (Alt + /)")
* [Tags](https://lilianweng.github.io/tags/ "Tags")
* [FAQ](https://lilianweng.github.io/faq "FAQ")
# Why We Think
Date: May 1, 2025 | Estimated Reading Time: 40 min | Author: Lilian Weng
Table of Contents
* [Motivation](https://lilianweng.github.io/posts/2025-05-01-thinking/#motivation)
* [Analogy to Psychology](https://lilianweng.github.io/posts/2025-05-01-thinking/#analogy-to-psychology)
* [Computation as a Resource](https://lilianweng.github.io/posts/2025-05-01-thinking/#computation-as-a-resource)
* [Latent Variable Modeling](https://lilianweng.github.io/posts/2025-05-01-thinking/#latent-variable-modeling)
* [Thinking in Tokens](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-in-tokens)
* [Branching and Editing](https://lilianweng.github.io/posts/2025-05-01-thinking/#branching-and-editing)
* [Parallel Sampling](https://lilianweng.github.io/posts/2025-05-01-thinking/#parallel-sampling)
* [Sequential Revision](https://lilianweng.github.io/posts/2025-05-01-thinking/#sequential-revision)
* [RL for Better Reasoning](https://lilianweng.github.io/posts/2025-05-01-thinking/#rl-for-better-reasoning)
* [External Tool Use](https://lilianweng.github.io/posts/2025-05-01-thinking/#external-tool-use)
* [Thinking Faithfully](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-faithfully)
* [Does the Model Tell What it Thinks Faithfully](https://lilianweng.github.io/posts/2025-05-01-thinking/#does-the-model-tell-what-it-thinks-faithfully)
* [Optimization Pressure on CoT: Good or Bad?](https://lilianweng.github.io/posts/2025-05-01-thinking/#optimization-pressure-on-cot-good-or-bad)
* [Thinking in Continuous Space](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-in-continuous-space)
* [Recurrent Architecture](https://lilianweng.github.io/posts/2025-05-01-thinking/#recurrent-architecture)
* [Thinking Tokens](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-tokens)
* [Thinking as Latent Variables](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-as-latent-variables)
* [Expectation-Maximization](https://lilianweng.github.io/posts/2025-05-01-thinking/#expectation-maximization)
* [Iterative Learning](https://lilianweng.github.io/posts/2025-05-01-thinking/#iterative-learning)
* [Scaling Laws for Thinking Time](https://lilianweng.github.io/posts/2025-05-01-thinking/#scaling-laws-for-thinking-time)
* [What’s for Future](https://lilianweng.github.io/posts/2025-05-01-thinking/#whats-for-future)
* [Citation](https://lilianweng.github.io/posts/2025-05-01-thinking/#citation)
* [References](https://lilianweng.github.io/posts/2025-05-01-thinking/#references)
Special thanks to [John Schulman](https://scholar.google.com/citations?user=itSa94cAAAAJ&hl=en) for a lot of super valuable feedback and direct edits on this post.
Test time compute ([Graves et al. 2016](https://arxiv.org/abs/1603.08983), [Ling, et al. 2017](https://arxiv.org/abs/1705.04146), [Cobbe et al. 2021](https://arxiv.org/abs/2110.14168)) and Chain-of-thought (CoT) ([Wei et al. 2022](https://arxiv.org/abs/2201.11903), [Nye et al. 2021](https://arxiv.org/abs/2112.00114)), have led to significant improvements in model performance, while raising many research questions. This post aims to review recent developments in how to effectively use test-time compute (i.e. “thinking time”) and why it helps.
# Motivation[#](https://lilianweng.github.io/posts/2025-05-01-thinking/#motivation)
Enabling models to think for longer can be motivated in a few different ways.
## Analogy to P
# Why We Think | Lil'Log
[Lil'Log](https://lilianweng.github.io/ "Lil'Log (Alt + H)")
* |
* [Posts](https://lilianweng.github.io/ "Posts")
* [Archive](https://lilianweng.github.io/archives "Archive")
* [Search](https://lilianweng.github.io/search/ "Search (Alt + /)")
* [Tags](https://lilianweng.github.io/tags/ "Tags")
* [FAQ](https://lilianweng.github.io/faq "FAQ")
# Why We Think
Date: May 1, 2025 | Estimated Reading Time: 40 min | Author: Lilian Weng
Table of Contents
* [Motivation](https://lilianweng.github.io/posts/2025-05-01-thinking/#motivation)
* [Analogy to Psychology](https://lilianweng.github.io/posts/2025-05-01-thinking/#analogy-to-psychology)
* [Computation as a Resource](https://lilianweng.github.io/posts/2025-05-01-thinking/#computation-as-a-resource)
* [Latent Variable Modeling](https://lilianweng.github.io/posts/2025-05-01-thinking/#latent-variable-modeling)
* [Thinking in Tokens](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-in-tokens)
* [Branching and Editing](https://lilianweng.github.io/posts/2025-05-01-thinking/#branching-and-editing)
* [Parallel Sampling](https://lilianweng.github.io/posts/2025-05-01-thinking/#parallel-sampling)
* [Sequential Revision](https://lilianweng.github.io/posts/2025-05-01-thinking/#sequential-revision)
* [RL for Better Reasoning](https://lilianweng.github.io/posts/2025-05-01-thinking/#rl-for-better-reasoning)
* [External Tool Use](https://lilianweng.github.io/posts/2025-05-01-thinking/#external-tool-use)
* [Thinking Faithfully](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-faithfully)
* [Does the Model Tell What it Thinks Faithfully](https://lilianweng.github.io/posts/2025-05-01-thinking/#does-the-model-tell-what-it-thinks-faithfully)
* [Optimization Pressure on CoT: Good or Bad?](https://lilianweng.github.io/posts/2025-05-01-thinking/#optimization-pressure-on-cot-good-or-bad)
* [Thinking in Continuous Space](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-in-continuous-space)
* [Recurrent Architecture](https://lilianweng.github.io/posts/2025-05-01-thinking/#recurrent-architecture)
* [Thinking Tokens](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-tokens)
* [Thinking as Latent Variables](https://lilianweng.github.io/posts/2025-05-01-thinking/#thinking-as-latent-variables)
* [Expectation-Maximization](https://lilianweng.github.io/posts/2025-05-01-thinking/#expectation-maximization)
* [Iterative Learning](https://lilianweng.github.io/posts/2025-05-01-thinking/#iterative-learning)
* [Scaling Laws for Thinking Time](https://lilianweng.github.io/posts/2025-05-01-thinking/#scaling-laws-for-thinking-time)
* [What’s for Future](https://lilianweng.github.io/posts/2025-05-01-thinking/#whats-for-future)
* [Citation](https://lilianweng.github.io/posts/2025-05-01-thinking/#citation)
* [References](https://lilianweng.github.io/posts/2025-05-01-thinking/#references)
Special thanks to [John Schulman](https://scholar.google.com/citations?user=itSa94cAAAAJ&hl=en) for a lot of super valuable feedback and direct edits on this post.
Test time compute ([Graves et al. 2016](https://arxiv.org/abs/1603.08983), [Ling, et al. 2017](https://arxiv.org/abs/1705.04146), [Cobbe et al. 2021](https://arxiv.org/abs/2110.14168)) and Chain-of-thought (CoT) ([Wei et al. 2022](https://arxiv.org/abs/2201.11903), [Nye et al. 2021](https://arxiv.org/abs/2112.00114)), have led to significant improvements in model performance, while raising many research questions. This post aims to review recent developments in how to effectively use test-time compute (i.e. “thinking time”) and why it helps.
# Motivation[#](https://lilianweng.github.io/posts/2025-05-01-thinking/#motivation)
Enabling models to think for longer can be motivated in a few different ways.
## Analogy to P
DeepCamp AI