← Back to context Comment by rickette 12 hours ago LLMs.txt is also nonsense since it isn't adopted by any of the major AI players. 12 comments rickette Reply networked 11 hours ago Google has recently added `llms.txt` to Chrome's Lighthouse check for agentic browsing (https://searchengineland.com/google-llms-txt-chrome-lighthou...), so adoption may be coming. Admittedly, I put more faith in <link rel="alternate" type="text/markdown" href="https://example.com/foo.md" title="Markdown version of the <Foo> page"> that I copied from Gwern.net. This convention is discoverable (just read the HTML) and naturally adapts to any website size and structure.I have created an `llms.txt` for my website anyhow. I use a fixed LLM prompt to generate it from the internal links in `index.md`. iamacyborg 11 hours ago Giving a markdown version of a page seems like an interesting choice instead of just embedding a schema marked up one vidarh 10 hours ago Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:https://code.claude.com/docs/en/overview andhttps://code.claude.com/docs/en/overview.md 4 replies → chrisweekly 6 hours ago It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md? dspillett 11 hours ago The same could be said of robots.txtAnd anything else that might tell them not to access something. reddalo 8 hours ago robots.txt predates the modern web though dspillett 4 hours ago My point was that llms.txt not working is no different from them ignoring everything else that came before and probably everything that is yet to come.If they want it, they will take it, polite directives in text files will have no effect. pfannl 8 hours ago To be fair, "not adopted by any major AI player" is probably the most web-standard-compliant phase of a new web standard.
networked 11 hours ago Google has recently added `llms.txt` to Chrome's Lighthouse check for agentic browsing (https://searchengineland.com/google-llms-txt-chrome-lighthou...), so adoption may be coming. Admittedly, I put more faith in <link rel="alternate" type="text/markdown" href="https://example.com/foo.md" title="Markdown version of the <Foo> page"> that I copied from Gwern.net. This convention is discoverable (just read the HTML) and naturally adapts to any website size and structure.I have created an `llms.txt` for my website anyhow. I use a fixed LLM prompt to generate it from the internal links in `index.md`. iamacyborg 11 hours ago Giving a markdown version of a page seems like an interesting choice instead of just embedding a schema marked up one vidarh 10 hours ago Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:https://code.claude.com/docs/en/overview andhttps://code.claude.com/docs/en/overview.md 4 replies → chrisweekly 6 hours ago It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md?
iamacyborg 11 hours ago Giving a markdown version of a page seems like an interesting choice instead of just embedding a schema marked up one vidarh 10 hours ago Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:https://code.claude.com/docs/en/overview andhttps://code.claude.com/docs/en/overview.md 4 replies → chrisweekly 6 hours ago It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md?
vidarh 10 hours ago Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:https://code.claude.com/docs/en/overview andhttps://code.claude.com/docs/en/overview.md 4 replies →
chrisweekly 6 hours ago It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md?
dspillett 11 hours ago The same could be said of robots.txtAnd anything else that might tell them not to access something. reddalo 8 hours ago robots.txt predates the modern web though dspillett 4 hours ago My point was that llms.txt not working is no different from them ignoring everything else that came before and probably everything that is yet to come.If they want it, they will take it, polite directives in text files will have no effect.
reddalo 8 hours ago robots.txt predates the modern web though dspillett 4 hours ago My point was that llms.txt not working is no different from them ignoring everything else that came before and probably everything that is yet to come.If they want it, they will take it, polite directives in text files will have no effect.
dspillett 4 hours ago My point was that llms.txt not working is no different from them ignoring everything else that came before and probably everything that is yet to come.If they want it, they will take it, polite directives in text files will have no effect.
pfannl 8 hours ago To be fair, "not adopted by any major AI player" is probably the most web-standard-compliant phase of a new web standard.
Google has recently added `llms.txt` to Chrome's Lighthouse check for agentic browsing (https://searchengineland.com/google-llms-txt-chrome-lighthou...), so adoption may be coming. Admittedly, I put more faith in
that I copied from Gwern.net. This convention is discoverable (just read the HTML) and naturally adapts to any website size and structure.
I have created an `llms.txt` for my website anyhow. I use a fixed LLM prompt to generate it from the internal links in `index.md`.
Giving a markdown version of a page seems like an interesting choice instead of just embedding a schema marked up one
Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:
https://code.claude.com/docs/en/overview and
https://code.claude.com/docs/en/overview.md
4 replies →
It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md?
The same could be said of robots.txt
And anything else that might tell them not to access something.
robots.txt predates the modern web though
My point was that llms.txt not working is no different from them ignoring everything else that came before and probably everything that is yet to come.
If they want it, they will take it, polite directives in text files will have no effect.
To be fair, "not adopted by any major AI player" is probably the most web-standard-compliant phase of a new web standard.