Teardown

Unsafe llms.txt Teardown

A fictional teardown for teams creating llms.txt guidance without leaking protected reports, API routes, scan URLs, payment links, or private customer data.

Run free domain check See Starter Snapshot

Unsafe Pattern

RelayBase, a fictional SaaS, published an llms.txt file that mixed public assets with operational and tokenized paths.

Unsafe exclude examples: `/api/`, `/r/`, `/scan?domain=`, tokenized report URLs, payment links, internal job pages, and account-specific pages.
The file pointed crawlers to URLs that were not meant to be indexed or treated as public citation sources.
Several listed URLs returned noindex pages or required private access.
The guidance did not match sitemap, canonical URLs, or visible navigation.

Safe Include Model

A safer llms.txt works as a public-page map, not a private data feed.

Include homepage, product, pricing, docs overview, FAQ, comparison, proof, guide, resource, privacy, and methodology pages when public.
Use canonical URLs and short descriptions that match visible page copy.
Keep protected reports, raw provider output, owner access keys, private customer data, payment URLs, and API endpoints out.
Make sure included pages can be crawled, rendered, and understood without login or private context.

Validation Checklist

Validate the file before asking crawlers or answer systems to rely on it.

Fetch `/llms.txt` and confirm it returns plain text with public URLs only.
Compare included URLs against sitemap, robots.txt, canonical tags, and noindex metadata.
Open each listed page and confirm it contains visible product facts, buyer questions, proof, or policy context.
Re-scan after publishing and treat improvements as public guidance readiness until stronger answer evidence exists.

Copyable sections

Paste into a brief, proposal, or ticket

These snippets are intentionally plain so consultants and teams can reuse them without importing private report data.

Copyable Safe llms.txt Rule

Include only public, canonical pages that explain the product, pricing, docs, FAQ, comparison, proof, resources, policies, and methodology. Exclude API routes, protected reports, scan query URLs, tokenized links, payment URLs, and customer-specific pages.

Copyable Developer Ticket

Audit llms.txt for unsafe URLs. Remove private or operational paths, align the remaining public URLs with sitemap and canonical metadata, and re-scan to confirm public agent guidance is safe and readable.

Evidence limits

Keep the claim boundary visible

Evidence limitation: this fictional teardown supports safe public-page guidance and validation. It does not promise recommendation, ranking, citation, or visibility.

Can llms.txt include protected reports?

No. Protected reports, tokenized URLs, owner access paths, downloads, and private report data should stay out of public crawler guidance.

Can llms.txt include API routes?

No. API routes are operational surfaces, not public citation assets. Keep them out of llms.txt and sitemap.

Does a clean llms.txt prove AI visibility?

No. A clean llms.txt can improve public-page guidance and diagnostic readiness, but it is not proof of platform-wide AI visibility.

Turn the checklist into a diagnostic

Run a free public-domain check, then use pricing to decide whether the $99 Starter Snapshot is worth unlocking.

Run free check View pricing