Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers
📰 ArXiv cs.AI
arXiv:2605.05697v1 Announce Type: cross Abstract: Transformers usually expose one inference cost per trained model, while deployed systems often need multiple cost-quality operating points. We study Budgeted Attention Allocation, a monotone head-gating mechanism conditioned on a requested attention budget. Dense warm-starting is important for stability: on a robust synthetic sequence task, one budgeted model reaches 99.7% accuracy at 0.303 estimated attention cost and 100.0% accuracy at 0.504 co
Full Article
Title: Budgeted Attention Allocation: Cost-Conditioned Compute Control for Efficient Transformers
Abstract:
arXiv:2605.05697v1 Announce Type: cross Abstract: Transformers usually expose one inference cost per trained model, while deployed systems often need multiple cost-quality operating points. We study Budgeted Attention Allocation, a monotone head-gating mechanism conditioned on a requested attention budget. Dense warm-starting is important for stability: on a robust synthetic sequence task, one budgeted model reaches 99.7% accuracy at 0.303 estimated attention cost and 100.0% accuracy at 0.504 co
Abstract:
arXiv:2605.05697v1 Announce Type: cross Abstract: Transformers usually expose one inference cost per trained model, while deployed systems often need multiple cost-quality operating points. We study Budgeted Attention Allocation, a monotone head-gating mechanism conditioned on a requested attention budget. Dense warm-starting is important for stability: on a robust synthetic sequence task, one budgeted model reaches 99.7% accuracy at 0.303 estimated attention cost and 100.0% accuracy at 0.504 co
DeepCamp AI