Comment by layer8
11 hours ago
> I'm very much a "builder" type dev that has more fun going from 0-v1 than maintaining and expanding scalable, large systems.
Maintaining and expanding is more challenging, which is why I’ve grown to prefer that. Greenfield and then leaving is too easy, you don’t learn the actually valuable lessons. As experience shows that projects won’t stay in the nice greenfield world, building them can feel like doing illusory work — you know the real challenges are yet to come.
Not sure what type of "greenfield" startup experience you've had, but most of the work I'm talking about involves solving problems that most people simply don't have the combined skill set to solve. Typically problems that involve a substantial amount of quantitative skills combined with the ability to bring those solutions to prod.
Nearly all of the teams I've joined had problems they didn't know how to solve and often had no previously established solution. My last gig involved exploring some niche research problems in LLM space and leveraging the results to get our first round of funding closed, this involved learning new code bases, understanding the research papers, and communicating the findings publicly in an engaging way (all to typical startup style deadlines).
I agree with your remarks around "greenfield" if it just involves setting up a CRUD webapp, but there is a wide space of genuinely tricky problems to solve out there. I recall a maintainer style coworker of mine, who describe himself similar to what you are describing, telling me he was terrified of the type of work I had to do because when you started you didn't even know if there was a solution.
I have equal respect for people such as myself and for people that you describe, but I wouldn't say it is more challenging, just a different kind of challenge. And I do find the claim "you don't learn the actually valuable lessons" to be wildly against my experience. I would say most of my deep mathematical knowledge comes from having to really learn it to solve these problems, and more often than not I've had to pick up on an adjacent, but entirely different field to get things done.
FWIW this is the best kind of job ever for me as well:
"when you started you didn't even know if there was a solution."
Regardless what the problem is - as long as I know _nobody knows if there is a solution_ it's an instant sugar rush.
You are free to bang your head against a stone wall for months trying to crack the damn thing.
OFC you need to deliver in the end. And this requires then ruthless "worse is better" mentality - ship something - anything. Preferably a the smallest package that can act as mvp that you know you can extend _if this is the thing_ what people want.
Because that's the other side of the coin - if the solution is not known - people are also not aware if the solution has true value or if it is a guess.
So in any case you have to rush to the mvp.
Such joy!
Of course the mvp must land, and must be extensible.
But these type of MVP:s are not a slam dunk.
The combined requirement of a) must ship within limited time b) nobody knows what _does_ require a certain mindset.
Interesting, this is a trap that I've seen multiple senior hires fall into. In my experience "we have a problem but no solution" often means (a) there actually is a solution but it's too expensive to implement, (b) there are organizational reasons why this problem exists and a new hire doesn't have the experience or credibility to navigate it or (c ) there is no solution to the problem or the solution is very complex, and by the time the new hire onboards, digs into the problem, and figures it out their credibility is shot because everyone was expecting the senior hire to figure it out in 90 days.
I've found new hires to be more successful when they join, get some easy wins, and then find their own problems to solve. But maybe it's just an artifact of working at large companies where most of the day-to-day stuff is figured out.
2 replies →
Yup. You learn the most valuable information from watching how things break and then fixing them.
It's kind of like when the FAA does crash investigation -- a stunning amount of engineering and process insights have been generated by such work to the benefit of all of us.
> You learn the most valuable information from watching how things break and then fixing them.
Trust me, you get plenty of experience in this as a founding engineer in a startup.
Many of these comments make me wonder how many people here have actually worked at an early stage startup in a lead role. You learn a lot about what's maintainable and scalable, what breaks and what doesn't, in the process rapidly iterating on a product to find your market.
I don't think HN has been frequented by startup engineers with leadership responsibilities in any density in a long time. It's very obvious to me reading a lot of the comments here that most folks are ICs somewhere in a large, bureaucratic software organization. That's why there's so much BOFH style commentary here these days.
(For readers, I don't think there's anything wrong with that but it just means that certain perspectives are overrepresented here that may not be more reflective of the broader industry.)
4 replies →
> Trust me, you get plenty of experience in this as a founding engineer in a startup.
Now be part of the team of folks that keeps that application running for 10, 20, 30 years. Now be part of the transition team to the new app with the old data. Those tasks will also teach you a lot about system stability, longevity, and portability... lessons that can only be learned with more time than a startup has.
I'm a founding engineer in a startup right now, and I was a founding engineer in a startup acquired by a large company. So then I became a part of a large company.
The technical challenges are _very_ different between these environments. In a small company you have to deal with technical breakages all the time, but you don't really have systems-level problems.
Valuable in what metric? I'm very much in the brownfield-has-the-lessons camp, but one of the lessons is that this experience has a very low market value. In fact it's so impossible to downgrade from "senior in $outdated" to "junior in $whateverisconsideredhotrightnow" that any brownfield experience could easily be considered to have negative market value.
I don't think they're mutually exclusive. You could just as easily describe someone with bootstrapping experience as being like an FAA crash investigator who investigates take offs. You get to know exactly what works when moving fast and looking for quick results, and what dooms a short timeline to failure.
> You could just as easily describe someone with bootstrapping experience as being like an FAA crash investigator who investigates take offs.
Takeoff systems aren't analogous to prototype development. I don't know you'd build a prototype plane that's feasible to take to market, without having deep knowledge about how planes are built.
Early design decisions matter. And you don't get to that realisation without dealing with legacy systems where some upstart made terrible decisions that you're now responsible for.
I was just thinking yesterday that knowing all the ways something breaks and behaves is the key to understanding systems.
and people that understand data it critical with storage supply getting tighter
Yep. I have loved fixing problems in systems and the processes that the people working on them use for years.
“What kind of role are you looking for?”
“Technologist flavor of NTSB investigator.”
There was a two year period around 2011-2013 where I experimented with my dev team. We were being "forced" to migrate a legacy enterprise system from .Net/MSSQL to Java/PostgreSQL with replace the front end with a modern, reactive web interface. Only two of the existing developers had Java experience, and both were conveniently senior engineers in two different offices and running their own discrete sub-teams.
One of the guys had a very strong opinion that the ideal architecture was something as abstracted and object oriented as possible with single function classes, etc. I let him run with that. The other guy got frustrated with his sub-team's inability to write code to spec in a language they'd never used before and where they were trying to build some new features they didn't clearly understand. He developed a strong feeling that TDD was the most efficient path forward: he owns the PRD and design, so he just created test stubs and told the remote team to "just write code that passes the test" even if they didn't understand the function of the block.
So, after a few months where did we end up:
1. The "abstract everything" architect's team had an extremely fragile and impossible to maintain codebase because it was impossible for any outsider to tell what was going on.
2. The "just pass the damn tests" guy had a team that had quickly ramped on a new language and they had a codebase that was incomplete (because they were building it like a Lego project) but that everyone could understand because the code blocks generally stood on their own.
What was the next step: to shut down the guy who abstracted everything and force him to drive a quick & dirty rewrite that would be more maintainable, and to also start a major refactoring of the "Lego" team's code because it was so fragmented that it also was fragile and unsuited for production.
I saw this as a terrific learning experience for all involved and I was able to get away with it because the stakes were pretty low and we had time to experiment (part of the ultimate objective was upskilling the team), but the more important lessons were these:
1. Docs matter. Take the time to write clear & detailed specs first because you'll be forced to think of edge cases and functionality that you didn't originally, and it provides a basis for system design, too.
2. Architecture & design matter. Adhering too close to any single paradigm is probably a mistake, but it takes experience on the team to understand where the compromises are and make the best decision for that system.
That second point will not stop being true with the advent of agentic assisted software development. Like others have said, my expectation in the job market is that pay will continue to be depressed for junior hires as employers reset expectations and generally just want folks who can instruct fleets of agents to do the actual coding. Senior staff will become increasingly critical and their jobs will be painful and difficult, because it'll be assumed they can (and will be willing to) do design & code reviews of artifacts originated by agents.
What I am going to be most interested in is what happens in the SRE/Sysadmin world over the next few years as more AI-generated code hits prod in organizations that don't have adequate review & oversight functions.
2 replies →
Sounds amazing.
Everyone can do 0 to 1. Because delivery drives dopamine. The it is the boring thing, but there comes experience to find interest in that part
Oh boy you're wrong. Many people (including myself) are experts at doing 0 to 0.8. The rest is much more difficult.
This is news to me having watched many many many people fail to do 0 to 1.
Maybe you're just a really really good engineer and product thinking hybrid!
It is more challenging, but I feel like it also has fewer people looking for that. That whole "move fast and break things" phrase messed with too many people's heads. I don't think people appreciate this segment of a product's life cycle as much as they should. They're always looking for the quick solutions.
> Greenfield and then leaving is too easy, you don’t learn the actually valuable lessons.
You learn a ton of valuable lessons going from 0 to v1. And a ton of value is created. I guess I'm unclear how you're defining "actually valuable" here.
I suspect the issue is the parent has never worked in an early stage role at a growing startup still iterating on finding product-market-fit. If they had they would realize you learn a lot about "maintaining and expanding", especially when your prototype now has a bunch of users.
This is evident in my personal experience by the fact that I am often the one that sees scaling and maintenance issues long before they happen. But of course parent would claim this is impossible.
Having worked at both greenfield startups and unicorns, I've found that virtually every problem I've encountered at the unicorn startups was caused by folks being incompetent at the greenfield level. Maybe when you get to the scale of Google things are different, but it's certainly possible to build a business big enough to retire off that doesn't require any more technical knowledge than what you'd learn at a two-person pre-PMF startup.
Architecting not knowing how to maintain it.
Edit: a legacy vibe coder
Sure, but that's in many ways the easy part.
If v1 is successful and attracts a lot of users, it will have to have features added and maintained.
Doing that in ways that does not produce "legacy code" that will have to be thrown away and rewritten in a few years is a very different skill than getting v1 up and running, and can easily be what decides if you have a successful business or not.
Taking this to the extreme, I think most lessons represent sunset or dead projects. There's no sweet illusions anymore. No assumptions. No ego. No account for infinite flexibility. No shine. No excitement of a new thing. No holy wars. No astronaut architects. Only you, the ruins and the truth.
Soooo agree. I've had to clean up the messes of people that did the 0-1 in my field and going from 1-unconditionally stable was a lot more work than the 0-1 part.
Having been in both roles, I believe it is important that each side of the “1” give the other a little grace.
When you are going from “1” to stable, there is some breathing room because you have a 1 that works, mostly. Sort of. Dealing with it may be a slow slog of sordid substitutions, but the pressure is different.
Going from 0 to 1 may involve working 80+ hour weeks with little sleep and enormous stress. It may mean meeting deadlines that make or break the product in a mad rush to fulfill a contract that saves or dooms the company. It may mean taking minutes to decide designs when days or months of consideration would have been more appropriate. And it may mean getting a lot of things wrong, but hopefully not so wrong that a version 2 is impossible.
As a final note: often v1 has substantial problems, that’s true. But sometimes it’s actually not that bad, and v2 fails because it was trying to shove the tech de jure (k8s cough cough) where it wasn’t needed so someone could get that shiny architect promotion.
This is unironically my favorite kind of HN comment: to say something incredibly rude and/or condescending but wrap it in the right kind of thoughtful language to qualify as HN nice
The original punchline ("you don’t learn the actually valuable lessons.") was just a bit too sharp, so you even edited in a psuedo-clarification which actually just repeats that punchline but in a softer way, masterful!
The soft sensitive people today have no idea how hard a condesending asshole has to work to live up to his own standards. When they do, one should still find something to troll them with. If you cant find it you complaint about the excessive border radius creating a child friendly fisherprice kind of environment. If that is the worse you can find they should agree and confess they have a fear of sharp edges.
How times have changed
My intention was actually to inspire others to maybe also start preferring long-term/maintenance work, because I feel there’s a lack of enthusiasm for that.
Almost invariably after submitting, I see how I could clarify and/or expand on my thoughts, so I often do end up editing.
This seems wrong? Like if you look at a collection of open SWENG positions, most of them are maintenance roles at large companies. Greenfield software doesn't have the revenue needed to justify much headcount.
In my experience separating the roles out is silly if you're an engineer yourself. We do this a lot and that leads to silly mentalities. Greenfield developer vs maintenance engineer, MVP engineer vs Big Tech dev, FOSS hacker vs FOSS maintainer. Each of those dichotomies speaks to cultural differences that we humans amplify for no reason.
In truth the profession needs both and an engineer that can do both is the most effective. The sharpest engineers I've worked with over the years can hack new, greenfield stuff and then go on to maintaining huge projects. Hell Linus Torvalds started out by creating Linux from scratch and now he's a steward of the kernel rather than an author!
1 reply →
> Almost invariably after submitting, I see how I could clarify and/or expand on my thoughts, so I often do end up editing.
One of the tricks of HN is the 'delay' setting. https://news.ycombinator.com/item?id=231024
> There's a new field in your profile called delay. It's the time delay in minutes between when you create a comment and when it becomes visible to other people. I added this so that when there are rss feeds for comments, users can, if they want, have some time to edit them before they go out in the feed. Many users edit comments after posting them, so it would be bad if the first draft always got shipped.
I've got mine set to 2. It gives me a little bit of time for the "oh no, I need to fix things" or "I didn't mean to say that" and when everyone else can see it.
1 reply →
Business bros will not pay high salaries to maintain software. Software maintenance will always end in India with developers making $20/hr. Or less.
AI makes it look like these developers can do the same job the Americans did building the product to begin with. Even if things fall apart in the end, it won’t stop the attempt to order of magnitude reduce the cost for maintenance.
It's correct tho. If your entire career is nothing but greenfield development, you'll never know the result of your decisions or the impact of tech chosen.
Staff or principals that have a tenure of majority greenfield development are extremely dangerous to companies IMO. Especially if they get hired in a nontraditional tech company, like utilities, banking, or insurance.
Your list is places that treat development as a cost center, but greenfield-only devs don't want to touch that work with a 10-foot pole.
And if your entire career is nothing maintenance and sustaining projects, you'll never know what decisions it takes to build a greenfield application that lives long enough to become a graybeard.
You'll think you do because you see all the mistakes they made, but you'll only have cynical reasons for why those mistakes get made like "they don't care, they just make a mess and move on to the next job" or "they don't bother learning the tools/craft deeply enough moving, it's all speed for them".
-
To indulge myself in the niceness a bit: I don't think you write comments like the one above if you've done both, yet having done both feels like an obvious requirement to be a well-rounded Staff/Principal.
Most maintenance work suffers because of decisions made at the 0 to 1 stage. And most greenfield work fails entirely, never maturing to the maintenance stage.
So both sides have to do something right in the face of challenges unique to their side. And having familiarity with both is extremely valuable for technical leadership.
1 reply →
And yet it is correct. The most valuable engineers today are those who have maintained and expanded the 0..v1 crap from others, and are now driven and ambitious enough to go build the next generation of 0..v1. Armed with that experience, the crap is minimal and value maximized.
Oof ima be the one to say it depends. This is personality based and the truth is a successful product has both. Even late on u want that person willing to break convention to find a new way of doing something. Early u need some seasoning in there too.
Passive aggressiveness isn't the opposite of kindness, and worse than directness, but you'll get away with it here because this is just reddit but more pretentious and all the same biases are intact, just more walls of text instead of getting to the point.
They're 100% correct, though.
They're incorrect, and my reply to a sibling covers why in detail.
But to reword it: if you think the reason 0 to 1 work is typically a duct-taped mess is because of a lack of experience or understanding from greenfield devs, you'll probably fail at 0 to 1 work yourself.
Not that a noob developer great at selling has never landed 0 to 1 work, crapped out a half working mess and left with a newly padded resume... but maintenance work is missing out on by far the most volatile and unpredictable stage of a software project, with its own hard lessons.
The duct-taped nature of 0 to 1 work is usually a result of the intersection of fickle humans and software engineering, not a lack of knowledge.
-
People in maintenance can do things like write new tests against the production system to ensure behavior stays the same... what happens when 1 quarter into a 2 quarter project it turns out some "stakeholder" wasn't informed and wants to make changes to the core business logic that break half the invariants you designed the system around. And then after that it turns out you can't do that, legal pushed back. And then a few weeks later they came to an agreement so now we want a bit of A and B?
Or you're in consumer and there's a new "must have" feature for the space? Maybe you'd like to dismiss as "trend chasing", but that'll just doom your project in the market because it turns out following trends is a requirement for people to look at everything else you've built
Or worst of all, you know that quality engineering of the system will take 8 weeks, and there's a hard deadline on someone else's budget of 4 weeks, and you can of course decline to ship it, but then you'll need a new job. (and I know, you'll say "Gladly, I take pride in my engineering!", but again, you're probably going to end up maintaining a project that only survived by doing exactly what you quit over)
tl;dr it's Yin and Yang: you can't have one without the other, and you need to have a bit of the other side in you whenever you're working in the capacity of either to be a good technical leader.
2 replies →