Comment by jonatron

2 months ago

They're in a unique position where many people allow googlebot but try to block most other bots

3 comments

jonatron

Reply
tenuousemphasis  2 months ago

Allow for the purpose of indexing, not training models.

Like if you give a friend a key to your house so they can check on your plants when you're out of town but they throw a rager and trash the place.

  • VBprogrammer  2 months ago

    > throw a rager

    That was not a phrase I expected to read on Hacker News! Haven't heard it since I was about 13. I always assumed it was a Scottish phrase.