Technical AI discoverability

Updated: 23.07.2026

ai.txt

ai.txt is not one settled standard. A legacy convention publishes /ai.txt for text-and-data-mining preferences, while a 2026 individual IETF Internet-Draft proposes richer policy files at /.well-known/ai.txt and /.well-known/ai.json. Adoption, syntax, and compliance vary.

In plain terms

Different ai.txt proposals may express training, scraping, indexing, licensing, attribution, or per-agent preferences, but they are not interchangeable.

A policy declaration communicates intent; it does not technically block a crawler and does not replace standard robots.txt access rules or provider-specific controls.

Sites should document which format they publish and avoid claiming universal support.

Why it matters

As content-usage norms formalize, an explicit permission file helps state your intent to the crawlers that honor it.

For most brands seeking visibility, the priority is signaling openness to citation, not restriction.

How it relates to GetCited.me

The current GetCited.me audit checks the legacy root path /ai.txt for genuine plain-text content and generates a root-level ai.txt draft. It does not currently fetch or validate the proposed /.well-known/ai.txt format.

Free tools for this term

Stop reading, start checking. No signup, no credit card.

Related terms

llms.txtllms.txt is a proposed Markdown file that gives compatible AI tools a curated map of a site's important content.robots.txt (for AI crawlers)robots.txt controls which compliant crawlers may fetch specified URL paths; different AI and search uses may rely on different crawler tokens.Content-SignalContent-Signal is a new robots.txt policy directive for expressing preferences about search indexing, real-time AI input, and model training.

See how AI answers about your market

Free scan: market map, AI visibility snapshot, and a GEO score in about a minute. No card, no call.