How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
📰 ArXiv cs.AI
Learn how to train reasoning models using the Tsallis loss continuum to adapt to new tasks with limited supervision
Action Steps
- Define the Tsallis loss family $J_Q$ using the Tsallis $q$-logarithm
- Interpolate between RLVR and log-marginal-likelihood by adjusting the $q$ value
- Train a reasoning model using the Tsallis loss continuum with output-level supervision
- Evaluate the model's performance on a new task and adjust the $q$ value as needed
- Compare the results with traditional RLVR and density-estimation approaches
Who Needs to Know This
Researchers and engineers working on reasoning models and reinforcement learning can benefit from this approach to improve model adaptation and performance
Key Insight
💡 The Tsallis loss continuum allows for interpolation between exploitation and density-estimation, enabling more effective adaptation to new tasks
Share This
🤖 Train reasoning models with limited supervision using the Tsallis loss continuum! 📊
Key Takeaways
Learn how to train reasoning models using the Tsallis loss continuum to adapt to new tasks with limited supervision
Full Article
Title: How Fast Should a Model Commit to Supervision? Training Reasoning Models on the Tsallis Loss Continuum
Abstract:
arXiv:2604.25907v1 Announce Type: cross Abstract: Adapting reasoning models to new tasks during post-training with only output-level supervision stalls under reinforcement learning from verifiable rewards (RLVR) when the initial success probability $p_0$ is small. Using the Tsallis $q$-logarithm, we define a loss family $J_Q$ that interpolates between RLVR (at $q{=}0$, the exploitation pole) and the log-marginal-likelihood over latent trajectories (at $q{=}1$, the density-estimation pole). All m
Abstract:
arXiv:2604.25907v1 Announce Type: cross Abstract: Adapting reasoning models to new tasks during post-training with only output-level supervision stalls under reinforcement learning from verifiable rewards (RLVR) when the initial success probability $p_0$ is small. Using the Tsallis $q$-logarithm, we define a loss family $J_Q$ that interpolates between RLVR (at $q{=}0$, the exploitation pole) and the log-marginal-likelihood over latent trajectories (at $q{=}1$, the density-estimation pole). All m
DeepCamp AI