Why Is My pandas Loop So Slow? iterrows vs Vectorize

Why is my pandas loop so slow? Because it's a loop. Pandas was built to operate on whole columns at once, not one row at a time. Iterrows is the anti-pattern. Vectorize is the answer, over a thousand times faster. Pandas data frames look like spreadsheets, so new users reach for what feels natural, a for loop, one row at a time. That works on a hundred rows. On 28,000, the same code takes a quarter of a second. On a million, it takes minutes. Today, compute daily return on a 28k row OHLC table three ways, iterrows, itertuples, and vectorize. Same answer, drastically different speed. Why is my pandas loop so slow? Let's time three ways to do the same thing. prices.parquet is here, 28,000 rows of OHLC, 14 tickers, eight years. Open NVIM, save as pandas_loop.py. Two imports, time for measuring, pandas for the work. Load the parquet, print the row count. Method one, iterrows, the textbook anti-pattern. For each row, build a series, call dot square bracket twice, append to a Python list. 28,000 Python objects, 28,000 list appends. It works. It's also several hundred times slower than the alternative. The cost is hidden. Pandas has to wrap each row in a series, which involves a dict allocation under the hood. Method two, itertuples. Same loop shape, but named tuples instead of series. No dict allocation. About 10 times faster than iterrows on the same data. Better, but still a Python loop. Still one row at a time. Itertuples is the right escape hatch when you genuinely need the row-by-row pattern. But you almost never do. Method three, vectorize. Drop the loop. Operate on whole columns. Close minus open divided by open applied to the entire data frame in one call. Pandas pushes the work down into NumPy, which pushes it down into C. Save and run. Iterrows takes 280 milliseconds. Itertuples, about 25. Vectorize, a quarter of a millisecond. Over a thousand times faster than the loop you would have written. Three things to take away. First, iterrows is almost always the wrong tool. The series wrapping cost is real, and there's almost always a vectorized alternative. Second, itertuples is the right loop when you genuinely need one, a stateful walk, a sequence that can't be expressed column-wise. About 10 times faster than iterrows, and it returns named tuples. Third, vectorize first. Column math, boolean masks, NumPy functions, and Pandas built-ins like {dot} pct_change, all run in C. Hundreds to thousands of times faster than any Python loop. Like, subscribe, comment, and share if this helped. Subscribe to the playlist for more Python answers.