Meta-Aligner: Bidirectional Preference-Policy Optimization for Multi-Objective LLMs Alignment

arXiv:2604.24178v1 Announce Type: cross Abstract: Multi-Objective Alignment aims to align Large Language Models (LLMs) with diverse and often conflicting human values by optimizing multiple objectives simultaneously. Existing methods predominantly rely on static preference weight construction strategies. However, rigidly aligning to fixed targets discards valuable intermediate information, as training responses inherently embody valid preference trade-offs even when deviating from the target. To