GetMentionedByAILoading checker…
Four files, four different jobs
- 01robots.txt
- Crawler permissions and sitemap discovery.
- 02llms.txt
- A curated map for agents that choose to read it.
- 03ai.txt
- An emerging policy signal, not a universal standard.
- 04sitemap
- Canonical URL discovery for search crawlers.
No composite grade
A missing optional file is not a failed website.
The checklist keeps every observation separate. robots.txt access can affect crawling; llms.txt and ai.txt are optional signals. We show the evidence instead of turning unlike facts into a made-up score.
AI crawlability FAQ
How do I check whether AI crawlers can access my website?
Start with robots.txt permissions, then confirm crawlers can discover canonical URLs through an XML sitemap. llms.txt and ai.txt are optional signals, not universal requirements.
Do I need llms.txt and ai.txt for AI SEO?
No. Neither file is a universal ranking requirement. They can communicate guidance to agents that support them, while robots.txt and indexable page content remain more established controls.
Why does this audit not give an AI visibility score?
The files have different jobs and different levels of adoption. Combining them into one grade would imply evidence that the checks do not support.
llms.txt and SEO guide · All free tools · llms.txt · crawlers