llms.txt and Lighthouse audit for WordPress

The llms.txt conversation changed because two Google-adjacent surfaces now create confusion:

• Google Search guidance says you do not need llms.txt, AI-specific files, Markdown versions, or special markup to appear in generative AI Search features.
• Chrome Lighthouse’s experimental Agentic Browsing category checks for llms.txt as a machine-readable summary at the domain root.

For WordPress teams, the correct conclusion is not “llms.txt ranks”.

The correct conclusion is:

llms.txt is becoming part of the agentic-readiness conversation, but it remains a guidance surface, not a ranking guarantee or enforcement mechanism.

What the Lighthouse check actually means

The Lighthouse check is narrow.

It asks whether a site exposes a machine-readable summary at the expected location. Chrome’s documentation describes llms.txt as an emerging convention for summarizing a website’s content for LLMs and AI agents.

That makes the file useful as a routing and orientation layer.

It does not prove:

• that Google Search uses the file for ranking;
• that AI Overviews or AI Mode consume the file in a special way;
• that ChatGPT, Claude, Perplexity, Gemini, or other systems use the file;
• that an answer changed because the file exists;
• that crawlers obey the file;
• that the site is agent-ready.

The file can be valuable without being magical.

Why WordPress sites should still care

WordPress sites often expose messy public surfaces:

• home page;
• blog posts;
• categories;
• tags;
• author archives;
• date archives;
• attachment pages;
• search pages;
• WooCommerce product pages;
• cart and checkout routes;
• translation paths;
• plugin-generated pages;
• documentation and support pages.

A crawler or agent can eventually explore that structure, but the site may not make its priorities obvious. A well-maintained llms.txt can compress the public intent:

• what the site is;
• what product or service it offers;
• which pages are primary;
• which pages are support or policy surfaces;
• where the machine should find the best context;
• what the file does not authorize or prove.

That is useful even if it is not a ranking factor.

A useful llms.txt structure for WordPress

A good WordPress llms.txt should avoid generic filler. It should route machines toward the best public sources.

Section	Purpose	WordPress example
Site summary	Explain the site in one short paragraph	Product, service, publication, local business, documentation site
Primary pages	Identify canonical pages	Home, features, pricing, service hubs, documentation hubs
Policy surfaces	Clarify usage and governance	robots.txt, AI usage policy, governance files
Source pages	Point to evidence-rich pages	Comparisons, checklists, guides, cases, definitions
Contact/support	Provide escalation and correction routes	Contact page, support email, documentation
Limits	Prevent over-reading	“This file is guidance, not enforcement.”

For Better Robots.txt itself, the file should point to the product page, the plugin download, the audit documentation, governance pages, crawler-specific pages, and the agentic readiness cluster.

The mistake: treating Lighthouse as the goal

A shallow implementation tries to “pass Lighthouse” by adding a minimal file:

# Example.com

This is our website.

That may satisfy a presence check, but it does not help machines understand the site.

A stronger implementation uses llms.txt to express a maintained routing structure:

# Example.com — machine-readable summary

Canonical site: https://example.com/
Primary product: https://example.com/product
AI usage policy: https://example.com/ai-usage-policy
Robots policy: https://example.com/robots.txt
Source pages:
- https://example.com/definition/topic
- https://example.com/compare/product-a-vs-product-b
- https://example.com/case-study/example

Notice: this file provides guidance. It does not override robots.txt, authentication, legal terms, or runtime access controls.

The difference is not formatting. The difference is governance quality.

How Better Robots.txt helps

Better Robots.txt gives WordPress teams a safer workflow for the file because llms.txt should not be treated as an isolated text blob.

The file should be aligned with:

• robots.txt;
• AI crawler rules;
• sitemap declarations;
• AI usage signals;
• governance files;
• public policy pages;
• source-page architecture;
• the site’s real edition or product scope.

When these layers contradict each other, machine readers may over-interpret, under-interpret, or route to the wrong pages.

Better Robots.txt helps by making llms.txt part of the same crawl-governance workflow as robots.txt rather than a separate afterthought.

What Better Robots.txt cannot prove

A WordPress plugin cannot prove that every external AI system will:

• fetch llms.txt;
• respect the file;
• treat it as authoritative;
• change generated answers because of it;
• rank the site higher;
• avoid unsupported claims.

Those are external behaviors. Some require logs, prompt tests, surfaced URL tracking, and direct observations.

The plugin’s role is narrower and stronger:

publish a coherent, inspectable, machine-readable guidance layer for WordPress.

The recommended sequence

1. Audit the current file

Use the llms.txt checker and the free site audit. Check whether the file exists, whether it is reachable, whether it contains real guidance, and whether it contradicts the rest of the crawl policy.

2. Map the source pages

Before writing the file, decide which pages deserve to be listed:

• product hub;
• pricing or plan page;
• documentation hub;
• AI visibility hub;
• crawler-specific guides;
• policy and governance pages;
• proof or case-study pages;
• support page.

3. Remove weak routes

Do not list low-value WordPress archives, utility pages, duplicate pages, thin tags, internal search pages, or pages that are not stable sources.

4. Align with robots.txt

Do not point machines toward pages you also disallow without explaining why. A contradiction between llms.txt and robots.txt is worse than a short file.

5. Keep the file updated

A stale llms.txt can mislead machines. Review it when you change services, pricing, product capabilities, crawler posture, documentation routes, or governance files.

FAQ

Is llms.txt mandatory for WordPress?

No. It is optional. It is useful when it is accurate and maintained.

Does the Lighthouse check make llms.txt an SEO ranking factor?

No. Lighthouse can inspect a browser-facing machine-readability signal. That does not make the file a Google Search ranking factor.

Should I add llms.txt just to avoid an audit warning?

No. Add it when it improves the site’s machine-readable routing. Avoid empty placeholders.

Should llms.txt contain every URL?

No. It should list priority surfaces, not replicate the sitemap.

Does llms.txt replace robots.txt?

No. Read robots.txt vs llms.txt for WordPress and robots.txt, llms.txt, and WebMCP.

Read next

• Agentic readiness for WordPress
• Lighthouse Agentic Browsing for WordPress
• How to add llms.txt on WordPress
• AI governance files checker
• The machine governance file stack

Official references

For source verification, start with:

• Google Search guidance on generative AI optimization: https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
• Chrome Lighthouse llms.txt audit: https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt
• Chrome Lighthouse Agentic Browsing scoring: https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring