← Back to context

Comment by rickette

3 days ago

Does any of the LLM providers actually use llms.txt?

If I remember correctly this "standard" was setup by someone but without involvement of any of the major AI players.

16 comments

rickette

Reply
HermanMartinus  3 days ago

I can definitively say llms.txt is not used by any AI players. I run a blogging platform with around 80k blogs and /llms.txt is not requested by anything (other than humans checking to see if there's an llms.txt path).

All regular pages are aggressively scraped to the extent it's a problem I have to consistently manage, but not llms.txt.

  • sunshine-o  3 days ago

    Amazing, I didn't know.

    So it get even stranger, I am the only one reading those /llms.txt ...

  • nickserv  3 days ago

    I'm seeing quite a bit of request for these on my work's GitBook documentation site.

    But perhaps these are developers specifically targeting these pages to feed whatever LLM they are using.

  • 0123456789ABCDE  3 days ago

    > I can definitively say llms.txt is not used by any AI players.

      https://developers.openai.com/llms.txt
  https://docs.anthropic.com/llms.txt
  https://geminicli.com/llms.txt
  https://github.com/llms.txt
  https://docs.aws.amazon.com/llms.txt
  https://openrouter.ai/docs/llms.txt

solumos  3 days ago

No, requesting "Accept: text/markdown" in the headers and returning markdown is the more agreed upon standard at this point.[0]

[0] - https://acceptmarkdown.com/