Is AI progress slowing down?

📰 AI Snake Oil

Learn why AI progress may be slowing down and what this means for the future of AI development, particularly with model scaling and inference scaling.

intermediate Published 18 Dec 2024

Action Steps

Read recent news reports from The Information, Reuters, and Bloomberg to understand the challenges faced by leading AI developers like OpenAI, Anthropic, and Google Gemini.
Analyze the quotes from industry insiders like Ilya Sutskever to grasp the changing narrative around AI progress.
Explore the concept of inference scaling, also known as test-time compute scaling, and its potential to improve AI capabilities.
Evaluate the implications of slowing AI progress on your current projects and consider alternative approaches.
Research the latest developments in AI research and development to stay up-to-date with the rapidly changing field.

Who Needs to Know This

Data scientists, AI engineers, and product managers can benefit from understanding the current state of AI progress and its implications for their work, including the potential shift from model scaling to inference scaling.

Key Insight

💡 The AI industry is shifting its focus from model scaling to inference scaling due to challenges faced by leading developers, which may impact the future of AI development.

Full Article

Title: Is AI progress slowing down?

URL Source: https://www.normaltech.ai/p/is-ai-progress-slowing-down

Published Time: 2024-12-18T16:47:58+00:00

Markdown Content:
[![Image 1: AI as Normal Technology](https://substackcdn.com/image/fetch/$s_!YzQY!,w_40,h_40,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6780dda8-5879-4789-bf02-b3ea43a9f85e_1000x1000.png)](https://www.normaltech.ai/)

# [![Image 2: AI as Normal Technology](https://substackcdn.com/image/fetch/$s_!u4S7!,e_trim:10:white/e_trim:10:transparent/h_72,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F595a103f-0f5f-421a-a9ed-009cf269b7b4_3024x652.jpeg)](https://www.normaltech.ai/)

Subscribe Sign in

![Image 3: User's avatar](https://substackcdn.com/image/fetch/$s_!fLlB!,w_64,h_64,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F30f87ce8-8dbc-468f-8f8b-9fbf430e323c_976x974.png)

Discover more from AI as Normal Technology

Analyzing AI as transformative but normal technology, not superintelligence.

Over 79,000 subscribers

Subscribe

By subscribing, you agree Substack's [Terms of Use](https://substack.com/tos), and acknowledge its [Information Collection Notice](https://substack.com/ccpa#personal-data-collected) and [Privacy Policy](https://substack.com/privacy).

Already have an account? [Sign in](https://www.normaltech.ai/p/is-ai-progress-slowing-down)

# Is AI progress slowing down?

### Making sense of recent technology trends and claims

[![Image 4: Arvind Narayanan's avatar](https://substackcdn.com/image/fetch/$s_!bVLI!,w_36,h_36,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdd0d6558-256e-46c4-b2c5-7cf7f808a9c9_693x693.jpeg)](https://substack.com/@aisnakeoil)[![Image 5: Sayash Kapoor's avatar](https://substackcdn.com/image/fetch/$s_!fLlB!,w_36,h_36,c_fill,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F30f87ce8-8dbc-468f-8f8b-9fbf430e323c_976x974.png)](https://substack.com/@sayash)

[Arvind Narayanan](https://substack.com/@aisnakeoil) and [Sayash Kapoor](https://substack.com/@sayash)

Dec 18, 2024

171

16

29

Share

_By Arvind Narayanan, Benedikt Ströbl, and Sayash Kapoor_.

After the release of GPT-4 in March 2023, the [dominant narrative](https://x.com/fchollet/status/1848178049105494084) in the tech world was that continued scaling of models would lead to artificial general intelligence and then superintelligence. Those extreme predictions gradually receded, but up until a month ago, the prevailing belief in the AI industry was that model scaling would continue for the foreseeable future.

Then came three back-to-back news reports from [The Information](https://www.theinformation.com/articles/openai-shifts-strategy-as-rate-of-gpt-ai-improvements-slows), [Reuters](https://www.reuters.com/technology/artificial-intelligence/openai-rivals-seek-new-path-smarter-ai-current-methods-hit-limitations-2024-11-11/), and [Bloomberg](https://www.bloomberg.com/news/articles/2024-11-13/openai-google-and-anthropic-are-struggling-to-build-more-advanced-ai) revealing that three leading AI developers — OpenAI, Anthropic, and Google Gemini — had all run into problems with their next-gen models. Many industry insiders, including Ilya Sutskever, probably the most notable proponent of scaling, are now singing a very different tune:

> “The 2010s were the age of scaling, now we're back in the age of wonder and discovery once again. Everyone is looking for the next thing,” Sutskever said. “Scaling the right thing matters more now than ever.” (Reuters)

The new dominant narrative seems to be that model scaling is dead, and “inference scaling”, also known as “test-time compute scaling” is the way forward for improving AI capabilities. The idea

Read full article → ← Back to Reads

Is AI progress slowing down?

Full Article

Related Videos