You do not need llms.txt for Google Search visibility. Google says it does not use the file for rankings or its AI search features. This optional Markdown index may still be useful for a documentation workflow in which an agent reads it and follows its links.
Before making one, identify the task it should help with. For a Malaysian digital tools company, that might mean helping a coding assistant find the current installation guide instead of instructions for an older release.
What Is llms.txt?
llms.txt is a proposed reading guide for AI agents: software that can retrieve information while carrying out a task. It uses Markdown, a plain-text format with headings and links. An agent can read the overview and follow relevant links for the detail.
The proposal allows a file at the site root, such as https://example.com/llms.txt, or under a specific path. A file at /docs/llms.txt covers the documentation beneath /docs/. Publishing one does not establish that a particular agent will discover or use it.
When checked on October 8, 2026, FastHTML's public documentation file grouped links under Docs, API, Examples and Optional. Its Docs section linked to a Markdown guide.
What It Cannot Do
These files serve different purposes:
- robots.txt communicates crawl rules to cooperating crawlers. It is not password protection.
- A sitemap helps search engines discover URLs. Listing a URL does not guarantee indexing.
- llms.txt provides context and selected links for agents. It does not grant access, control indexing or require an answer to cite your business.
Google may crawl or index a text file without giving it a special role in Search. Chrome's Lighthouse also has an agentic-browsing audit for llms.txt. Its documentation says a missing file returning 404 is marked Not Applicable; a server error is flagged. This is a retrieval check, not a Google ranking recommendation.
When Might It Be Worthwhile?
Use three questions to decide:
- Which task needs help? Name something specific, such as finding the correct installation instructions for the current release.
- Which agent will use it? Check the tool's documented support or test a workflow that explicitly reads the file. Another company's published file is not proof that your chosen tool uses it.
- Who will maintain it? Assign a documentation owner, including when a generator builds the file from published sources.
For the digital tools company, a small test makes sense if current and legacy guides are easy to confuse and the intended assistant can read the index. For a local service website whose prices or coverage are unclear, correcting those pages is more useful than adding another summary. Start with the AI search readiness guide.
Check whether your documentation platform already generates a file. Mintlify, for example, automatically hosts an index and supports a custom override. Review what exists before introducing a second source to maintain.
How to Make a Small, Useful File
In a plain-text editor, save a file named llms.txt. Use # for the project H1, > for a short summary, and ## for groups of descriptive links. The H1 is the proposal's only required section. Keep the index focused on the chosen task.
Here is an example for a fictional product with current version 2 documentation. Replace the names and URLs with material your site actually publishes:
# Example Tools
> Setup and API documentation for Example Tools version 2.
Use version 2 guidance for new installations.
## Start here
- [Set up version 2](https://example.com/docs/v2/setup.md): Installation and first successful request.
## API reference
- [Authentication](https://example.com/docs/v2/authentication.md): Authentication steps and error responses.
## Optional
- [Version 1 migration](https://example.com/docs/v1-migration.md): Moving an existing installation to version 2.The current release is explicit, each link has a job, and the optional group holds secondary reading. Only use .md URLs if those Markdown pages exist and match the current documentation. Appending an extension does not create a page.
Publish the file at the intended public path. Keep private dashboards and credentials out of it. Update descriptions and destinations when releases or URLs change. If the platform generates the file, edit its source settings rather than a generated copy that the next build may replace.
Check It Before Publishing
Validate the file in a test environment, then repeat the response and link checks at its final public URL:
- Check the response. Open the exact file URL. Confirm that it shows the intended Markdown text rather than an HTML homepage, login screen or error. Ask the developer to verify a successful HTTP response, normally 200, and an appropriate text content type.
- Follow the links. Open every destination. Verify the product version, setup steps and public accessibility. Check Markdown pages against their corresponding human-readable guides.
- Try the intended task. Give a web-enabled agent an index URL it can access and the instruction below. Inspect its available record of pages opened to see what it actually fetched. If that evidence is unavailable, record retrieval as unverified.
- Record the outcome. Save the tool and mode, date, task, retrieved sources and any wrong-version result. Compare against the source guide. If the assistant follows an old link, repair that link or its description and repeat the task.
Use this documentation index to find the installation guide for the current release. State the version and source URL. If you cannot open a source, say so.
A correct answer backed by retrieval evidence demonstrates a working route for that task. It does not prove the agent would find the file without being given its URL. To test automatic discovery, run a separate task without supplying the index and inspect the available retrieval evidence.
Prioritize the Basics First
If customers cannot find the right guide, repair navigation and the guide first. The site architecture guide explains how to connect important pages for people and crawlers.
Check access for the product you want to reach. OpenAI's search-crawler documentation recommends allowing OAI-SearchBot through robots rules and the published IP ranges; it does not list llms.txt as a requirement. That guidance concerns ChatGPT search, not every AI product.
Before adding the file, write down the task, intended tool and person responsible for updates. If you cannot name them yet, leave the file for later.
Frequently Asked Questions
Is llms.txt Mandatory?
No. It is an optional proposal, and Google Search does not require or use it for its search visibility systems. A particular tool or documentation workflow may have its own requirements; check that product's instructions.
How Do You Create an llms.txt File?
First check for an existing platform-generated file. If you need a custom one, save plain-text Markdown as llms.txt, add a project H1 and useful public links, then publish it at the correct path. Verify the response and every linked page.
What Is the Purpose of llms.txt?
It gives an agent a concise starting point for finding relevant website material. For example, a documentation index can separate a current setup guide from migration instructions. Its value depends on the agent using it successfully for that task.

