I got rate limited scraping 100 pages. Here's what actually worked

Spent days fighting rate limits. The fix was simpler than I thought.