Comment by fourside
15 hours ago
It was the article that used metrics related to engineering velocity as evidence of success. It makes sense to ask how this is impacting their business metrics. The company is not in the business of shipping more code. I don’t see how asking how this ties back to core business results is thinking about it at the wrong level.
> It was the article that used metrics related to engineering velocity as evidence of success
Success for what? The article opened with: "Last week Will Larson (@Lethain ) asked how leaders are evaluating engineering managers for AI skills. My reply , a list of questions to think about, got a lot of bookmarks. More broadly, I've been thinking about how to build an AI-native engineering org for a while. Sharing some lessons from the last nine months of building at FurtherAI."
The opener is "how to build an AI-native engineering org" and the context is a company of roughly 22 engineers (from a reply.) Nothing here says "evidence of success"
> I don’t see how asking how this ties back to core business results is thinking about it at the wrong level.
In an company of 22 engineers, how would you make these measurements? Would you split your company into a velocity focused group and a quality focused group and make them compete? How? What would a customer motion look like? What would these groups share? This makes no sense. This is like asking a small company to A/B test teams that used a computer vs didn't use a computer.
A larger, more mature organization has the luxury of splitting business units into AI-native vs not-AI-native workflows and comparing COGS/business objectives against each other. I know of very large tech consulting services who did exactly this around the beginning of the cloud era to figure out which aspects of their business they wanted to place on clouds and which aspects they kept on-prem, and even then that was messy because part of the value of on-prem is the scale benefit you get by owning and operating your own DC. But no 22 person engineering team I saw A/B tested this out. Companies were founded either with a cloud-first philosophy or an on-prem philosophy.
If you can’t tell value at a 25 (or 10 or 5) person org, you will not be able to at a larger one.
If you weren’t making intentional decisions about cloud vs on prem… why? I worked with many small businesses that did.
This feels like a value-divorced outsourced engineering mindset. You have the power to understand if things are good ideas or working!
Idk, seems like it’s important to A/B test being “AI native” because it could go badly.
This is a 25 eng startup. Anything can go wrong here. A hike in interest rates can lead to a down round causing the company to go into cost cutting mode, leading to a decrease in product velocity and prod incidents as infra is cut to the bone, in order to cope. Your top engineer can get sick and quit and your next in line isn't strong enough to keep the ship going.
I'm curious if you've ever worked at a startup, tried to invest in a startup, or considered purchasing a startup's product to know what it's like? The whole point is you're taking a risk trying something new and different. If you're just doing what a much bigger, established company is doing, you're fighting against an incumbent who already has much lower COGS than you.
Taking a bet on being AI-native as a startup is taking a bet that use of AI will result in lower COGS than the incumbent. As the old saying goes, "Nobody got fired for buying IBM", and yet Amazon was able to crack into that space by doing something novel. If Amazon was just IBM, why not buy IBM? The ultimate test of AI-native business models will be if these companies become the Amazon of today, or whether we'll write about them the way we wrote about "fifth generation computing" in Japan.
2 replies →