Comment by networked
13 hours ago
Google has recently added `llms.txt` to Chrome's Lighthouse check for agentic browsing (https://searchengineland.com/google-llms-txt-chrome-lighthou...), so adoption may be coming. Admittedly, I put more faith in
<link rel="alternate" type="text/markdown" href="https://example.com/foo.md" title="Markdown version of the <Foo> page">
that I copied from Gwern.net. This convention is discoverable (just read the HTML) and naturally adapts to any website size and structure.
I have created an `llms.txt` for my website anyhow. I use a fixed LLM prompt to generate it from the internal links in `index.md`.
Giving a markdown version of a page seems like an interesting choice instead of just embedding a schema marked up one
Every page on code.claude.com has a markdown version available by just appending ".md", and Claude Code knows about it. E.g:
https://code.claude.com/docs/en/overview and
https://code.claude.com/docs/en/overview.md
After some consideration, I also applied this convention to every site I build - including content negotiation: Clients can either send an Accept header with their preference, or append an explicit extension (.md|.markdown for Markdown, .json for JSON API responses, or .html for the human HTML page). Together with the content negotiation part, it feels very much like HTTP was intended to work - especially the fact that API clients, AI agents, and humans all use the same URLs, but get the content in the shape they need.
1 reply →
Here's how to do it with more recent versions of Hugo:
https://photostructure.com/coding/hugo-markdown-output/
(It includes the grandparent's head link suggestion, but it's not just "change .html to .md" because I'm old skool and as a wee nerd was told that URLs ending in .html or .php or whatever we're frowned on, so the above link's markdown is available by appending /index.md )
It gets even more "interesting" for markdown-based systems like Astro or Obsidian Publish: author in md -> ship html && optionally serve md?