GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
📰 ArXiv cs.AI
Learn how to use language models as selective surrogates for kernel runtime optimization on GPUs, reducing the need for costly hardware measurements
Action Steps
- Train a language model on a dataset of kernel performances to predict runtime
- Use the trained model to select the most promising kernel candidates for hardware evaluation
- Evaluate the selected kernels on the target GPU hardware to validate the predictions
- Refine the language model by incorporating the new measurement data
- Apply the optimized kernel to the deep learning workload to achieve improved performance
Who Needs to Know This
ML engineers and researchers working on deep learning optimization can benefit from this technique to improve kernel performance without excessive hardware measurements
Key Insight
💡 Language models can be used to predict kernel performance and reduce the need for costly hardware measurements, leading to faster optimization of deep learning workloads
Share This
🚀 Accelerate deep learning optimization with language models as selective surrogates for kernel runtime optimization on GPUs! 🚀
Full Article
Title: GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
Abstract:
arXiv:2605.31464v1 Announce Type: cross Abstract: GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, because each evaluation of a kernel requires compilation and repeated execution on a GPU. As improvements in LLM inference reduce the cost of writing novel kernels and LLM
Abstract:
arXiv:2605.31464v1 Announce Type: cross Abstract: GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, because each evaluation of a kernel requires compilation and repeated execution on a GPU. As improvements in LLM inference reduce the cost of writing novel kernels and LLM
DeepCamp AI