How I tracked which AI bots actually crawl my site

📰 Dev.to AI

I launched a new domain two weeks ago and wanted to know which AI bots were actually showing up — not theoretically, but in my CloudFront logs. So I built a small tracker that parses access logs from S3 and reports hits per bot per URL. After 5 days, here's what the data shows. The setup The site is easerva.com — static HTML on S3 + CloudFront, zero JavaScript, JSON-LD on every page, sitemap submitted to GSC

Published 25 Apr 2026
Read full article → ← Back to Reads