Data Mixing Agent: Learning to Re-weight Domains for Continual Pre-training

arXiv:2507.15640v2 Announce Type: replace-cross Abstract: Continual pre-training on small-scale task-specific data is an effective method for improving large language models in new target fields, yet it risks catastrophic forgetting of their original capabilities. A common solution is to re-weight training data mixtures from source and target fields on a domain space to achieve balanced performance. Previous domain reweighting strategies rely on manual designation with certain heuristics based o