When to choose CPU vs GPU: Databricks AI Runtime Explained

The conversation around GPUs has shifted this year. It used to be about training models from scratch. Now it is about token economics. Here is the simple mental model: → CPU for the data work. ETL, feature engineering, SQL, classic ML. → GPU for deep learning. Fine-tuning LLMs, computer vision, recommenders, neural networks. Calling a frontier proprietary model on every request adds up fast at production scale. A lot of teams are realizing they can fine-tune a strong open-weights model like Kimi K2 or Qwen on their own data, run it on GPU, and get a system that is cheaper per token and often better at their specific task. That is where on-demand GPUs start to matter. You pick your accelerator, A10 or H100, attach it to your notebook, and fine-tune the open model that fits your workload.