The server itself seems to work fine. It only seems to be this specific post that's being 429'd.
I'm guessing it's some kind of anti-DDoS setup kicking in.
Mastodon is also quite heavy to host, my single user instance will easily gobble up several gigabytes of memory if you let it. There are more efficient ActivityPub servers but specifically Mastodon seems to be written for running efficiently on huge servers.
It will definitely "never" (barring fairly significant changes) run efficiently, you're right. It's extremely unnecessarily heavy in all kinds of ways that are made in ways that makes it run overall better on a large setup, though. All the instructions are also there to front it by proper caching, but setting it up in a proper resilient way is more effort.
unfortunately this is exactly why mastodon won't work
I already don't trust mastodon links because 9 times out of 10 they simply don't work. Everyone's tiny hobby server falls over when one post gets big, and obviously not everyone is going to scale their servers to support the load of a viral post that might happen once every 6 months and will be 100x their base load
That is the benefit of centralization, the experience for the end user can be controlled completely. Maybe a Mastodon friendly web cache that anyone running a semi-serious instance could easily opt into (for a fee) is needed. As a hedge to keep your Raspberry Pie instance online if something goes viral.
As a community effort where no one is expecting to get rich it might work.
One of my blog posts was submitted to HN that had 194 points and 149 comments[1]. All dates are in UTC.
1 - Unique visitors per day - Including spiders
Hits h% Vis. v% Tx. Amount Data
------ ------ ----- ------ ----------- ----
14439 1.49% 1148 1.19% 106.42 MiB 21/Jan/2023
17043 1.75% 1754 1.81% 184.69 MiB 20/Jan/2023
33560 3.45% 3267 3.37% 491.32 MiB 19/Jan/2023
46568 4.79% 5816 6.01% 637.54 MiB 18/Jan/2023
323797 33.32% 28928 29.88% 4.06 GiB 17/Jan/2023 <- Resubmitted on HN and websites started copy-pasting the article from the big website with the same mistakes, never checking my post which had a note about these mistakes :)
24330 2.50% 3341 3.45% 360.48 MiB 16/Jan/2023 <- Put in a second-chance pool by a moderator and an article with a lot of mistakes published by some big website
17074 1.76% 3348 3.46% 243.44 MiB 15/Jan/2023 <- Published on HN
1041 0.11% 120 0.12% 3.70 MiB 14/Jan/2023
1666 0.17% 171 0.18% 8.40 MiB 13/Jan/2023 <- Post published
991 0.10% 123 0.13% 374.78 KiB 12/Jan/2023
2 - Requested Files (URLs)
Hits h% Vis. v% Tx. Amount Mtd Proto Data
----- ------ ----- ------ ----------- -------- -------- ----
57604 5.93% 31427 32.46% 260.97 MiB GET HTTP/2 /en/2023/01/13/msi-insecure-boot/
31179 3.21% 11263 11.63% 245.20 MiB GET HTTP/1.1 /en/2023/01/13/msi-insecure-boot/
11 - Referring Sites (depends on Referer header, not very accurate for reasons)
Hits h% Vis. v% Tx. Amount Data
------ ------ ----- ------ ---------- ----
446781 45.97% 29686 30.66% 5.95 GiB dawidpotocki.com
14834 1.53% 9485 9.80% 79.85 MiB news.ycombinator.com
(news sites with very low hundreds or even under, nobody checks sources)
HN has millions of page views per day (maybe @dang can give a more accurate and updated number), and things frequently gets reposted elsewhere. Happens many times that things on the frontpage gets brought down to its knees, this wouldn't be the first nor the last.
Just look at it failing the HN hug of death. If this one can't survive techies on a orange site rushing to the site then it cannot possibly survive a lotus of users from Twitter or TikTok rushing into any post on Mastodon, bringing it flat on to the floor.
I can only see Mastodon centralizing to cope with the load. But a server going down on this load from HN tells us it is no where near ready to handle an insurmountable amount of users or even begin to challenge Twitter which hosts 220M+ users every single day.
Mastodon user count has mostly been a steady growth. So far it hasn't really failed at a high level. We aren't in Bitcoin territory, the network isn't really slower than it was a year ago even if number of user is much higher. It's mostly distributed among many instances.
This is mind-blowing. Last I checked, the front page of HN sends tens of requests per second to each link. There are humans who can pack envelopes faster than the typical mastodon server can answer GETs. I'd love to see someone benchmark the top servers for a few seconds to see what it takes to break a reasonable latency SLA.
Found a similar situation, but I think the key is mostly if you're on one of those servers and seek out the content or if you're logged into one of those servers, it won't forward you (even if you click it from here, assuming same browser/container).
yah that's fine, but the replication to other domains does no good for the typical user on the web who only has a link to the original URL, clicks it, sees an error page, and then clicks away
Sites like HN and that other one should track this. It could work like flagging - if enough people mark it as hugged to death (HTD), it would say so next to the link. Maybe it could even redirect to an archive if it’s currently HTD.
From the instance admins:
> We just started serving a http 429 error on the exact url of the post. So everything should go back to normal now.
So to answer GP's question, yes.
If you go to the user's profile and then to the post, it seems to be okay. So perhaps also looking at Referrer.
Doing such a thing never requests to the page that is linked, so it makes sense that nothing is blocking it
for the time being, archive.org has a snapshot of it: https://web.archive.org/web/20230504185520/https://chaos.soc...
The server itself seems to work fine. It only seems to be this specific post that's being 429'd. I'm guessing it's some kind of anti-DDoS setup kicking in.
Mastodon is also quite heavy to host, my single user instance will easily gobble up several gigabytes of memory if you let it. There are more efficient ActivityPub servers but specifically Mastodon seems to be written for running efficiently on huge servers.
Or running efficiently never maybe?
It will definitely "never" (barring fairly significant changes) run efficiently, you're right. It's extremely unnecessarily heavy in all kinds of ways that are made in ways that makes it run overall better on a large setup, though. All the instructions are also there to front it by proper caching, but setting it up in a proper resilient way is more effort.
Not really: hhttps://leah.is/posts/scaling-the-mastodon/ They just got 6 times the normal requests: https://chaos.social/@ordnung/110312089838674624
unfortunately this is exactly why mastodon won't work
I already don't trust mastodon links because 9 times out of 10 they simply don't work. Everyone's tiny hobby server falls over when one post gets big, and obviously not everyone is going to scale their servers to support the load of a viral post that might happen once every 6 months and will be 100x their base load
That is the benefit of centralization, the experience for the end user can be controlled completely. Maybe a Mastodon friendly web cache that anyone running a semi-serious instance could easily opt into (for a fee) is needed. As a hedge to keep your Raspberry Pie instance online if something goes viral.
As a community effort where no one is expecting to get rich it might work.
When someone finds an annoyance, often even anecdotal, that is no evidence of why "Mastodon (or the fediverse) won't work".
It's an annoyance, often anecdotal at most. Not the foundation of why a platform cannot ever "work".
Mastodon is by design about small niche communities rather than centralised twitter alternative.
5 replies →
You can also see the same "toot" as a "tweet": https://twitter.com/jonty/status/1653915932677271552
And the original tooter is apparently Google Translate?
This post is on the front of HN. Many a larger website have succumbed to HN's warm embrace.
Isn't HN pretty small? This post has <400 upvotes over 3 hours. There can't be 1000x that amount of lurkers can there?
I can provide some statistics myself.
One of my blog posts was submitted to HN that had 194 points and 149 comments[1]. All dates are in UTC.
[1]: https://news.ycombinator.com/item?id=34388533
HN has millions of page views per day (maybe @dang can give a more accurate and updated number), and things frequently gets reposted elsewhere. Happens many times that things on the frontpage gets brought down to its knees, this wouldn't be the first nor the last.
5 replies →
I’ve seen other people that posted about the HN embrace talk about 50k extra visitors. I guess this is a single page, so 50k pageviews?
Indeed there are. Tens of us!
Maybe you underestimate how many people want to keep up on things but not interact?
1 reply →
Just look at it failing the HN hug of death. If this one can't survive techies on a orange site rushing to the site then it cannot possibly survive a lotus of users from Twitter or TikTok rushing into any post on Mastodon, bringing it flat on to the floor.
I can only see Mastodon centralizing to cope with the load. But a server going down on this load from HN tells us it is no where near ready to handle an insurmountable amount of users or even begin to challenge Twitter which hosts 220M+ users every single day.
Mastodon user count has mostly been a steady growth. So far it hasn't really failed at a high level. We aren't in Bitcoin territory, the network isn't really slower than it was a year ago even if number of user is much higher. It's mostly distributed among many instances.
Twitter is pretty sluggish, to be fair. 7 seconds to load and render a single tweet on mobile.
https://pagespeed.web.dev/analysis/https-twitter-com-realDon...
Mastodon.social is actually much faster on this particular benchmark. So maybe there is hope.
No trouble viewing it from another Mastodon server:
https://hachyderm.io/@jonty@chaos.social/110307532115312279
EDIT: Ah I guess if you're not logged into a hachyderm.io account, you get forwarded. So probably don't use the above link.
That just redirected me to too many requests.
Maybe, but the admin commented it was intentional for that specific post, it was slowing down the entire site.
> slowing down the entire site
This is mind-blowing. Last I checked, the front page of HN sends tens of requests per second to each link. There are humans who can pack envelopes faster than the typical mastodon server can answer GETs. I'd love to see someone benchmark the top servers for a few seconds to see what it takes to break a reasonable latency SLA.
It has been 20+ years since slashdotting with the requisite hardware and connection upgrades and still things fall over.
1 reply →
Clearly they're not microservicing hard enough.
2 replies →
one alt-social crumbling from throwing rocks at another alt-social
This is peak Web 5.0 right here.
Aren’t federated services grand?
[flagged]
> It looks like these links auto-redirect when I access them from here, but when you access them from the homeserver they are served without redirect
"Works on my machine" isn't going to cut it for running a popular social network
Nobodies going to go around searching for mirrors, they'll just leave and go back to twitter
6 replies →
Found a similar situation, but I think the key is mostly if you're on one of those servers and seek out the content or if you're logged into one of those servers, it won't forward you (even if you click it from here, assuming same browser/container).
1 reply →
yah that's fine, but the replication to other domains does no good for the typical user on the web who only has a link to the original URL, clicks it, sees an error page, and then clicks away
3 replies →
Okay, I guess I'm not getting it either, but how is it federated/decentralized if they all redirect to the original server which throws a 429?
5 replies →
All of these redirect to web.archive.org. Are they using it to offload traffic? That doesn't seem very nice.
1 reply →
None of these links work for me. 429's on all of them right now.
1 reply →
To be fair, each of those is returning 429 at the time of this post.
2 replies →
Ironically, I've seen pages make it to the front of HN that were run on RPis or weaker.
There was a guy running a site off of a single core 32bit ARM SoC that was able to handle the HN frontpage.
chaos.social is run by the chaos computer club, you can assume that they configured it that way on purpose.
my profile, on the same server, loads fine.
This was both funny and ironic at the same time.
Upon clicking reply fastidiously, i got the hn 429
"Sorry, we're not able to serve your requests this quickly."
Wow.
Looks like we hugged it to death
Sites like HN and that other one should track this. It could work like flagging - if enough people mark it as hugged to death (HTD), it would say so next to the link. Maybe it could even redirect to an archive if it’s currently HTD.
lightly blew in its general direction to death
chaos.social is run on four dedicated servers
https://leah.is/posts/scaling-the-mastodon/
Because it is webscale.