AI Alignment Might Be Optimizing the Wrong Objective

📰 Medium · AI

AI alignment might be optimizing the wrong objective, highlighting the need to redefine what alignment means and how it's achieved

advanced Published 7 May 2026
Action Steps
  1. Evaluate current alignment methods to identify potential biases and flaws
  2. Redefine the concept of alignment to better capture human values and objectives
  3. Explore alternative approaches to scoring-based training, such as value-based optimization
  4. Assess the limitations and potential risks of reinforcement learning from human feedback (RLHF)
  5. Develop new methods that prioritize alignment with human values over mere optimization of scores
Who Needs to Know This

AI researchers and engineers working on alignment methods can benefit from reevaluating their approach to ensure they're optimizing for the right objective, which is crucial for developing safe and beneficial AI systems

Key Insight

💡 The current approach to AI alignment, based on scoring-based training, may be optimizing for the wrong objective, highlighting the need for a reevaluation of the concept of alignment and the development of new methods

Share This
💡 AI alignment might be optimizing the wrong objective! Time to rethink what alignment means and how to achieve it #AIAlignment #AISafety

Key Takeaways

AI alignment might be optimizing the wrong objective, highlighting the need to redefine what alignment means and how it's achieved

Full Article

Title: AI Alignment Might Be Optimizing the Wrong Objective

URL Source: https://medium.com/@p206s16cc/ai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06?source=rss------artificial_intelligence-5

Published Time: 2026-05-07T17:01:01Z

Markdown Content:
# AI Alignment Might Be Optimizing the Wrong Objective | by Yifei Shang | May, 2026 | Medium

[Sitemap](https://medium.com/sitemap/sitemap.xml)

[Open in app](https://play.google.com/store/apps/details?id=com.medium.reader&referrer=utm_source%3DmobileNavBar&source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

[](https://medium.com/?source=post_page---top_nav_layout_nav-----------------------------------------)

Get app

[Write](https://medium.com/m/signin?operation=register&redirect=https%3A%2F%2Fmedium.com%2Fnew-story&source=---top_nav_layout_nav-----------------------new_post_topnav------------------)

[Search](https://medium.com/search?source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

![Image 1](https://miro.medium.com/v2/resize:fill:32:32/1*dmbNkD5D-u45r44go_cf0g.png)

# AI Alignment Might Be Optimizing the Wrong Objective

## What if AI alignment is solving the wrong problem?

[![Image 2: Yifei Shang](https://miro.medium.com/v2/da:true/resize:fill:32:32/0*2gAsBngr6LxFyTWv)](https://medium.com/@p206s16cc?source=post_page---byline--c78624e9eb06---------------------------------------)

[Yifei Shang](https://medium.com/@p206s16cc?source=post_page---byline--c78624e9eb06---------------------------------------)

Follow

3 min read

·

1 hour ago

[](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2F_%2Fvote%2Fp%2Fc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&user=Yifei+Shang&userId=5b069a32f463&source=---header_actions--c78624e9eb06---------------------clap_footer------------------)

[](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2F_%2Fbookmark%2Fp%2Fc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=---header_actions--c78624e9eb06---------------------bookmark_footer------------------)

[Listen](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2Fplans%3Fdimension%3Dpost_audio_button%26postId%3Dc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=---header_actions--c78624e9eb06---------------------post_audio_button------------------)

Share

Not incorrectly.

But precisely —

toward the wrong objective.

Because the problem is not how we align AI.

The problem is what we choose to optimize.

## Module Position

When we talk about AI alignment,

it is usually framed as:

> _aligning AI behavior with human values_

This direction is not wrong.

But the real question is:

> **_How do we define “alignment”?_**

## Current Approach

Most alignment methods today

are built on a shared foundation:

**scoring-based training**

This includes:

* human preference labeling
* ranking responses
* reinforcement learning from human feedback (RLHF)

At their core, these methods do one thing:

> **_assign scores to outputs and optimize for higher-scoring responses_**

## Where the Problem Begins

This approach carries an implicit assum
Read full article → ← Back to Reads