Comment by cornel_io
4 years ago
That sounds like you've never seen performance of a heavily worked-on subsystem increase by 10x because one guy who was good at that kind of stuff spent a day or two focused on the problem.
I've seen that happen at least 10 times over my career, including at very big companies with very bright people writing the code. I've been that guy. There are always these sorts of opportunities in all but the most heavily optimized codebases, most teams either a) just don't have the right people to find them, or b) have too much other shit that's on fire to let anyone except a newbie look at stuff like performance.
More generally, in my experience performance isn't looked at because it's "good enough" from a product point of view.
"Yes it's kinda slow, but not enough so customers leave so who cares." Performance only becomes a priority when it's so bad customers complain loudly about it and churn because of it.
There’s a bit of incentive misalignment when commercial software performance is concerned. If we presume customers with tighter budgets tend to be more vocal and require more support, and customers on slower machines are often customers on tighter budgets, the business as a whole might actually not mind those customers leaving as it’d require less support resources spent on customers who are more difficult to upsell to.
Meanwhile, the majority of customers with faster machines are not sensitive enough to feel or bother about the avoidable lag.
Though we're now in a situation where lots of software (see: Word and Excel) is painfully slow even on high-end desktop hardware.
That is probably why they have this law called wirths law about the wintel ecosystem. What Andy giveth, Bill taketh away.
Or Gates's law "The speed of software halves every 18 months"
There's also sometimes the incentive to slow things down because if it is too fast, the client will perceive that he paid too much money for an operation that takes no time, i.e. it doesn't exists seems unimportant.
It would be a shame for a truly artistically designed busy-wheel if it didn't get turn once or twice, regardless of the time it was actually needed!
I've made such optimizations and others made them in my code, so:
c) slow code happens to everyone, sometimes you need fresh pair of eyes.
Absolutely: what I should have said is that I've been the one to cause performance problems, I've been the one to solve them, I've been the manager who refused to allocate time to them because they were not important enough, and I've been the product owner who made the call to spend eng hours on them because they were. There are many systemic reasons why this stuff does not get fixed and it's not always "they code like crap", though sometimes that is a correct assessment.
But show me a codebase that doesn't have at least a factor of 2 improvement somewhere and does not serve at least 100 million users (at which point any perf gain is worthwhile), and I'll show you a team that is being mismanaged by someone who cares more about tech than user experience.
"Need" especially, because often it's just that those fresh eyes don't have any of the political history, so doesn't have any fallout from cutting swaths through other people's code that would be a problem for someone experienced in the organisation.
I’ve seen it at least as many times, too. Most of the time, the optimization is pretty obvious. Adding an index to a query, or using basic dynamic programming techniques, or changing data structures to optimize a loop’s lookups.
I can’t think of a counter example, actually (where a brutally slow system I was working on wasn’t fairly easily optimized into adequate performance).
It is nice when a program can be significantly sped up by a local change like that but this is not always the case.
To go truly fast, you need to unleash the full potential of the hardware and doing it can require re-architecting the system from the ground up. For example, both postgres and clickhouse can do `select sum(field1) from table group by field2`, but clickhouse will be 100x faster and no amount of microoptimizations in postgres will change that.
No argument from me. I’m just pointing out that it’s wrong to assert that programmers are general incorrect when they say something can be optimized. Many times in my career, optimizations were trivial to implement, and the impact was massive. There have been a few times when the optimization was impossible without a major rearchitecture, but those times were rare.
Yah, was going to say something like this. I've fixed these problems a few times, and I don't really think any of them were particularly hard. That is because if your the first person to look at something with an eye to performance there is almost always some low hanging fruit that will gain a lot of perf. Being the 10th person to look at it, or attack something that is widely viewed as algorithmically at its limit OTOH is a different problem.
I'm not even sure it takes an "experienced" engineer, one of my first linux patches was simply to remove a goto, which dropped an exponential factor from something, and changed it from "slow" to imperceptible.