AI files: what crawlers read
Only a few machine-readable files are documented by AI and search providers. robots.txt decides which bots may read your pages, and Organization markup states who you are. llms.txt is a public proposal: optional, and no provider documents reading it.
robots.txt
HighDecides which bots may read your site. Allow the search and user bots of ChatGPT, Claude, Perplexity and Gemini, or they can't fetch and cite your pages.
Organization JSON-LD
HighSchema.org Organization markup (JSON-LD) with your name, URL and description, for the HTML head of your homepage.
llms.txt
OptionalBusiness name, summary, contact, key links. A public proposal (llmstxt.org); no AI provider documents reading it.
Not documented by any provider
ai.txt, brand.txt, ai.json, identity.json, faq-ai.txt, developer-ai.txt, robots-ai.txt, llm.txt and llms.html are community ideas. No AI or search provider documents reading them, so we neither score nor generate them. They do no harm if you already have them.
Generate your files
robots.txt lines for AI crawlers, Organization markup and an llms.txt, tailored to your site. PRO: 10 generations per rolling 30 days; Business: 15.