AI Alignment Might Be Optimizing the Wrong Objective

📰 Medium · Machine Learning

AI alignment might be optimizing the wrong objective, highlighting the need to redefine what alignment means and how it's achieved

advanced Published 7 May 2026
Action Steps
  1. Question the assumption that scoring-based training is the best approach to AI alignment
  2. Explore alternative methods that prioritize understanding human values and intentions
  3. Evaluate the objectives being optimized in current alignment methods and consider whether they align with human values
  4. Investigate the potential consequences of optimizing the wrong objective in AI alignment
  5. Develop new frameworks for defining and achieving alignment that prioritize human values and well-being
Who Needs to Know This

AI researchers and engineers working on alignment methods can benefit from understanding the potential flaws in current approaches and exploring alternative solutions

Key Insight

💡 The current approach to AI alignment, based on scoring-based training, may be flawed and require reevaluation to ensure alignment with human values

Share This
🚨 AI alignment might be optimizing the wrong objective! 🤖 Let's rethink what alignment means and how to achieve it 📊

Key Takeaways

AI alignment might be optimizing the wrong objective, highlighting the need to redefine what alignment means and how it's achieved

Full Article

Title: AI Alignment Might Be Optimizing the Wrong Objective

URL Source: https://medium.com/@p206s16cc/ai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06?source=rss------machine_learning-5

Published Time: 2026-05-07T17:01:01Z

Markdown Content:
# AI Alignment Might Be Optimizing the Wrong Objective | by Yifei Shang | May, 2026 | Medium

[Sitemap](https://medium.com/sitemap/sitemap.xml)

[Open in app](https://play.google.com/store/apps/details?id=com.medium.reader&referrer=utm_source%3DmobileNavBar&source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

[](https://medium.com/?source=post_page---top_nav_layout_nav-----------------------------------------)

Get app

[Write](https://medium.com/m/signin?operation=register&redirect=https%3A%2F%2Fmedium.com%2Fnew-story&source=---top_nav_layout_nav-----------------------new_post_topnav------------------)

[Search](https://medium.com/search?source=post_page---top_nav_layout_nav-----------------------------------------)

Sign up

[Sign in](https://medium.com/m/signin?operation=login&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=post_page---top_nav_layout_nav-----------------------global_nav------------------)

![Image 1](https://miro.medium.com/v2/resize:fill:32:32/1*dmbNkD5D-u45r44go_cf0g.png)

# AI Alignment Might Be Optimizing the Wrong Objective

## What if AI alignment is solving the wrong problem?

[![Image 2: Yifei Shang](https://miro.medium.com/v2/da:true/resize:fill:32:32/0*2gAsBngr6LxFyTWv)](https://medium.com/@p206s16cc?source=post_page---byline--c78624e9eb06---------------------------------------)

[Yifei Shang](https://medium.com/@p206s16cc?source=post_page---byline--c78624e9eb06---------------------------------------)

Follow

3 min read

·

1 hour ago

[](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2F_%2Fvote%2Fp%2Fc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&user=Yifei+Shang&userId=5b069a32f463&source=---header_actions--c78624e9eb06---------------------clap_footer------------------)

[](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2F_%2Fbookmark%2Fp%2Fc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=---header_actions--c78624e9eb06---------------------bookmark_footer------------------)

[Listen](https://medium.com/m/signin?actionUrl=https%3A%2F%2Fmedium.com%2Fplans%3Fdimension%3Dpost_audio_button%26postId%3Dc78624e9eb06&operation=register&redirect=https%3A%2F%2Fmedium.com%2F%40p206s16cc%2Fai-alignment-might-be-optimizing-the-wrong-objective-c78624e9eb06&source=---header_actions--c78624e9eb06---------------------post_audio_button------------------)

Share

Not incorrectly.

But precisely —

toward the wrong objective.

Because the problem is not how we align AI.

The problem is what we choose to optimize.

## Module Position

When we talk about AI alignment,

it is usually framed as:

> _aligning AI behavior with human values_

This direction is not wrong.

But the real question is:

> **_How do we define “alignment”?_**

## Current Approach

Most alignment methods today

are built on a shared foundation:

**scoring-based training**

This includes:

* human preference labeling
* ranking responses
* reinforcement learning from human feedback (RLHF)

At their core, these methods do one thing:

> **_assign scores to outputs and optimize for higher-scoring responses_**

## Where the Problem Begins

This approach carries an implicit assumption:
Read full article → ← Back to Reads