← Back to context

Comment by 1vuio0pswjnm7

2 days ago

https://x.com/sitemap.xml (listed in robots.txt but doesn't work for me)

https://www.facebook.com/robots.txt (186 sitemap files listed, none are accessible to me)

Perhaps these require an "acceptable browser" and/or an account

Why sitemap URLs. Because I'm a sitemaps user; I use sitemap files every day

The /dev/null ones seem to be easily accessible. Unlike some other "social networks"