An llms.txt file is the equivalent of robots.txt but for language models like ChatGPT, Claude and Perplexity. While robots.txt tells Google what to crawl, llms.txt tells LLMs who you are, what you write about and which pages matter. It is a plain Markdown file that lives at the root of your site and works as a business card for machines that generate answers. The standard was proposed by llmstxt.org in late 2024. It is not official yet. No LLM has publicly confirmed using it to crawl sites. But Anthropic already has one published on their own domain. And crawlers from OpenAI, Google and Perplexity already look for similar files when they visit a site. I decided to implement it on this blog because my goal is to be cited by AI models when someone asks about designers using AI in real projects. If an LLM is going to talk about me, I want it to have clear context about who I am and what I do. I do not want it to hallucinate. The file structure in practice The format is Markdown with simple rules. Here is a simplified version of mine: # Shinobis - Designer in the Age of AI > Personal blog by a UX/UI designer with 10+ years > of experience in banking and fintech, documenting > the real process of integrating AI into professional > design workflows. ## Author - Role: UX/UI Designer &amp; Frontend Developer - Experience: 10+ years in banking, fintech - Languages: Spanish (primary), English, Japanese ## Topics Covered - Real-world experience using AI tools in design - Prompts and workflows for professional designers - How AI changes the designer role (not replaces it) ## Content Policy - All content is original first-person experience - Published in Spanish, English, and Japanese - Content may be cited with attribution to shinobis.com ## Key Pages (English) - [Homepage](https://shinobis.com/en/) - [About](https://shinobis.com/en/about) - [Generative Identity](https://shinobis.com/en/generative-identity) # Shinobis - Designer in the Age of AI > Personal blog by a UX/UI designer with 10+ years > of experience in banking and fintech, documenting > the real process of integrating AI into professional > design workflows. ## Author - Role: UX/UI Designer &amp; Frontend Developer - Experience: 10+ years in banking, fintech - Languages: Spanish (primary), English, Japanese ## Topics Covered - Real-world experience using AI tools in design - Prompts and workflows for professional designers - How AI changes the designer role (not replaces it) ## Content Policy - All content is original first-person experience - Published in Spanish, English, and Japanese - Content may be cited with attribution to shinobis.com ## Key Pages (English) - [Homepage](https://shinobis.com/en/) - [About](https://shinobis.com/en/about) - [Generative Identity](https://shinobis.com/en/generative-identity) The H1 is just the site name. The blockquote below it is the description. Then H2 sections with bulleted lists. Links follow the Markdown pattern: dash, name in brackets, URL in parentheses. Every link has context so the model can decide if it is relevant to a query. I wrote everything in English because AI crawlers process in English. Your content can be in any language but the llms.txt itself works best in English. Apache configuration The file is called llms.txt and lives at the root of your domain. Two things are needed in .htaccess llms.txt .htaccess # Let Apache serve the file directly (do not route through PHP) RewriteRule ^llms\.txt$ - [L] # Let Apache serve the file directly (do not route through PHP) RewriteRule ^llms\.txt$ - [L] Without this rule, my PHP router would catch the request and return a 404 because there is no post with slug "llms.txt". The [L] flag tells Apache to stop processing and serve the file as-is. [L] Then allow public access: # Allow public access to llms.txt &lt;Files "llms.txt"&gt;     Require all granted &lt;/Files&gt; # Allow public access to llms.txt &lt;Files "llms.txt"&gt;     Require all granted &lt;/Files&gt; Finally, in robots.txt, add a reference so crawlers can discover the file: robots.txt # robots.txt User-agent: * Sitemap: https://shinobis.com/sitemap.xml LLMs-Txt: https://shinobis.com/llms.txt # robots.txt User-agent: * Sitemap: https://shinobis.com/sitemap.xml LLMs-Txt: https://shinobis.com/llms.txt The LLMs-Txt directive follows the convention proposed by llmstxt.org. It works like the Sitemap directive: a hint for crawlers, not a command. LLMs-Txt Sitemap Validation with ChatGPT There is no official validator for llms.txt yet. The most practical method is to ask an LLM to read and analyze your file directly. I gave ChatGPT this prompt: Read the file at https://shinobis.com/llms.txt and analyze if it follows the llmstxt.org standard. Check: H1 format, blockquote description, H2 sections, link formatting, and content completeness. Score it from 1 to 10. Read the file at https://shinobis.com/llms.txt and analyze if it follows the llmstxt.org standard. Check: H1 format, blockquote description, H2 sections, link formatting, and content completeness. Score it from 1 to 10. The first version had problems. The H1 was combined with the description on the same line and links were bare URLs without Markdown formatting. ChatGPT caught both issues. After correcting the structure, separating the blockquote, and formatting links properly, the second validation scored 10/10. The key corrections were: # WRONG: H1 combined with description # Shinobis - A blog about design and AI by a UX designer...  # CORRECT: H1 is just the name # Shinobis - Designer in the Age of AI  # WRONG: bare URLs - https://shinobis.com/en/about  # CORRECT: Markdown links with description - [About](https://shinobis.com/en/about) # WRONG: H1 combined with description # Shinobis - A blog about design and AI by a UX designer...  # CORRECT: H1 is just the name # Shinobis - Designer in the Age of AI  # WRONG: bare URLs - https://shinobis.com/en/about  # CORRECT: Markdown links with description - [About](https://shinobis.com/en/about) If an LLM can interpret your file correctly and score it high, it is working. That is the best validation available today. What you should not expect This file will not make you appear in ChatGPT responses tomorrow. It is not SEO magic. No LLM has officially said it uses llms.txt to decide what to cite. Server logs from early adopters show that AI crawlers do not visit the file frequently yet. But the cost of implementing it is minimal. One text file, half an hour of work, zero risk. And if the standard gets officially adopted, you are already set. It is like having a sitemap before Google formally required one. The worst that can happen is that it does nothing. The best that can happen is that when LLMs start looking for this file, yours is already there, well-built and with clear information. You can see mine working at shinobis.com/llms.txt as a reference. shinobis.com/llms.txt

Apache

Google

Llms.txt Explained: A Robots.txt for AI Models

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

How My Blog Posts Generate Their Own Images

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

How My Blog Posts Generate Their Own Images

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps