Comment by CalRobert
10 hours ago
"For something so core to the business, I'm baffled that they let it get to the point where it was costing $300K per year."
You build something that's a dirty hack but it works, then your company grows, and nobody ever gets around to building it.
I was at a place spending over $4 million a year on redshift basically because someone had slapped together some bad (but effective!) queries when the company was new, and then they grew, and so many things had been built on top they were terrified to touch anything underneath.
This was amazingly common in the 2010s during the Big Data craze. I know, because I was the one slapping the bad queries together.
Most startups didn’t care (to a point) because at that point in their lifecycle, the information they needed to get from those queries (and actions they could take based on it, like which customers were likely to convert and worth spending sales time on, etc) was more important than the money spent on the insane redshift clusters.
The mantra was almost always some version of, “just do it now, as fast as possible, and if we’re still alive in a year we’ll optimize then.”