Comment by CM30

I suspect part of this (at least where software devs are concerned) is resume driven development. It's a lot easier to get hired at a top company if you've worked on similarly difficult problems before, while working with 'boring', simple technology makes your CV look uninteresting to a potential recruiter.

It also gives you a lot more free time if you can persuade your employer to let you use cutting edge technology rather than needing to save it for your side projects after hours.

Plus, given how many startups fail, you do kinda need a backup plan...

So, I suspect a lot of companies are... encouraged to aim for scale and encouraged to tackle problems in the most overcomplicated way possible so that the people working there can take their experience to Alphabet/Meta/Apple/whatever if things go south.