Comment by lamontcg
4 years ago
I setup an apache webserver one time with a single index.html with the word "FOO" in it.
It was a placeholder until the software dev team responsible for the server could deploy software onto it (they asked it be setup with something in the index.html so they could confirm it was running, they didn't care what)
The networking team in advance of that wired it up to some load balancers.
They took an IP which had formerly been a decommissioned cluster of webservers serving the main website for the entire company (an internet retailer named after a river in some other country, you've probably never heard of them).
The DNS loadbalancers found the old IP (it had never been deleted from their configs which was the root cause) was now live and it was REALLY fast, so they shunted most of the traffic over to it.
Created a sev1 outage for "users reporting 'foo' on the website"
I'm happy I kept it professional that day.
future alien FOO technology
> Let's say 99 of your 100 machines are taking 750 msec to handle a request (and actually do work), but this single "bad boy" [machine] is taking merely 15 msec to grab it and kill it. Is it any surprise that it's going to wind up getting the majority of incoming requests? Every time the load balancers check their list of servers, they'll see this one machine with nothing on the queue and a wonderfully low load value.
> It's like this machine has some future alien technology which lets it run 50 times faster than its buddies... but of course, it doesn't. It's just punting on all of the work.
-- https://rachelbythebay.com/w/2015/02/16/capture/
they were all returning 200s though for "GET /", so they were perfectly healthy and happy...
optimized as shit for that one request...
This made me VERY happy