For webmasters
AIEOBot: what it is and how to block it
AIEOBot is the crawler behind every AIEO scan. This page lists exactly what it fetches, how it identifies itself, and how to opt out — the same courtesy we ask of any bot on our own site.
What AIEOBot fetches
A single scan is a short, bounded read — never an open-ended crawl. For one domain, AIEOBot fetches:
| Target | Why we fetch it |
|---|---|
| Homepage | The primary page — read once per scan. |
| Up to 4 internal pages | Linked from the homepage, to sample how the rest of the site is marked up. |
| /robots.txt | Checked first, and honoured — see the opt-out below. |
| /sitemap.xml | Read for structure and last-modified dates, never crawled beyond the homepage sample. |
| /llms.txt | Read if present; its absence is not scored against you. |
| DoH TXT records | A DNS-over-HTTPS lookup for security-posture signals (SPF, DMARC). No page content is touched. |
How it identifies itself
Every request carries one honest, browser-compatible user-agent string — the same identity on every fetch, with no fallback disguise if a request fails:
Mozilla/5.0 (compatible; AIEOBot/2.0; +https://aieo.fyi/bot)
If a fetch is blocked or errors, AIEOBot reports that back to the person running the scan as "blocked at edge" rather than retrying under a different name.
Rate and scope
Requests per scan
6 max
Homepage, up to 4 internal pages, robots.txt, sitemap.xml, llms.txt and one DoH lookup.
Scans per visitor
12 / hour
A shared rate limit across the scan tool, applied per requester, not per domain.
How to opt out
AIEOBot reads your robots.txt before it fetches anything else, and honours a rule written for it by name. To block AIEOBot entirely, add:
User-agent: AIEOBot
Disallow: /
A scan of a domain that disallows AIEOBot still runs, but reports the site as blocked rather than reading its content. We only honour rules addressed to AIEOBotby name — a wildcard User-agent: * disallow is not treated as covering AIEOBot, the same convention Google and OpenAI's bots use.
Questions or a formal request
If you believe AIEOBot has behaved incorrectly on your site, or you want a request handled outside robots.txt, contact us directly — see the contact page — or read how scan data is handled on the privacy page.