Comment by 1vuio0pswjnm7
2 days ago
https://x.com/sitemap.xml (listed in robots.txt but doesn't work for me)
https://www.facebook.com/robots.txt (186 sitemap files listed, none are accessible to me)
Perhaps these require an "acceptable browser" and/or an account
Why sitemap URLs. Because I'm a sitemaps user; I use sitemap files every day
The /dev/null ones seem to be easily accessible. Unlike some other "social networks"
No comments yet
Contribute on Hacker News ↗