← Back to context

Comment by a_shovel

21 hours ago

I've heard this is part of why major infrastructure projects in America can be so expensive. A city builds one subway line, and everyone working on the project has no experience, so it takes a long time and is expensive. The expense convinces people to oppose any more projects, so the city doesn't build any public transit for a decade(s). By the time they're ready to build another line, all the experience has evaporated, so the new line takes a long time and is expensive. Repeat forever.

There's an example of this in railway electrification: if you scroll down to the graph in https://publications.parliament.uk/pa/cm5801/cmselect/cmtran... it shows that the UK tends to do electrification as occasional big projects, whereas Germany has consistently done about the same mileage every year for decades, presumably with the same institutions maintaining their expertise and just moving on to the next bit of track. Their costs are a quarter of the UK's...

  • Yeah, but whatever Germany is doing is obviously wrong because 62.5% of their stops have trains arriving within 6 minutes of target time https://www.dw.com/en/over-a-third-of-deutsche-bahn-long-dis... while the UK has 85.9% of their stops having trains arriving within 3 minutes of target time https://dataportal.orr.gov.uk/statistics/performance/passeng...

    So whatever the Germans are doing with their rail, thank god the UK isn't.

    • I don't think you can conclude from overall performance of a complex system that a particular aspect of it (the way of electrifying tracks) is done wrong.

      From what I understand, the problems in the German railway network stem predominantly from growing demand meeting the results of chronic underinvestment in infrastructure. That does not automatically mean that all of the work should be done in big chunks instead of continuously. They still could be doing some things right, and I think it is worth pointing that out. I find it more interesting and productive than blanket dismissal.

    • Iirc that is mostly an issue of the scheduling and not the construction itself. The operator is also known to have reliability problems with many train types (not necessarily the stops or tracks themselves).

Oh, this is an area I know something about. I work on railway software in my day job.

Broadly speaking you are correct. Expertise like mine is rare and fleeting mostly because you can only really build it long term by working at a company which can convince international clients to take them on. Even large countries tend not to have more than a handful of trains being built on any given day of the week.

This is one place where having a business located in a nation long known for its relative neutrality, calmness and international trustworthiness can pay off. All of the Nordics are good at this, really.

Not Just Bikes did a YouTube on Seoul South Korea that brought this point up. They’ve got costs down because they’re working on it continuously.

As a tech writer people have a lot of experience but they never turn it into institutional knowledge because it’s never written down. Ay best it’s tribal knowledge passed by word of mouth.

I know some people refuse to document things because they are hoping for job security but that never happens. Or sometimes for revenge for getting rid of them. But many companies survive despite those efforts.

  • I'm not good at writing documentation, and you can't pay me enough to care about it, sorry. I've tried enough times, and nobody reads it, or it becomes out of date by the time anyone reads it, and I don't see the personal ROI. I'll write notes for future me, and put them somewhere others can read it, if you don't make that onerous. Otherwise, if you want documentation from me, you need to have someone else drag the information out of me and write it down. But, I've only rarely been in organizations that care enough about documentation to do that, so there you go.

    There's always a lot of talk about how documentation is important, but there's never budget for a tech writer (well, you must have found some, as you've taken tech writer as a title, but it's not often available) or a documentation maintainer.

    • Writing / English were my worst subjects in school. Yet I have written internal, dealer, and customer facing documentation because I was the only one knowledgeable on the subject and there was no tech writer.

      Now I work where there is a tech writer and still create internal, dealer, and customer facing documentation, because I am the only one with the knowledge on the subject matter. Some things are filtered to the tech writer so tech writing has been reduced.

      Simply, don't call me or contact me for simple questions. Give me a real problem that others cannot solve. Some people like customer service or being able to be the one that helps. Documentation allows me to not be that person and focus on other things.

      There is only two ways to communicate to a person on how to use that tool only you created. 1) Showing them how to do it. 2) Giving them documentation so they only contact when needed. Option #2 takes time to save time in the long run that can be diverted else where.

      Documentation is part of any product design and software based solution. That new feature time is design, implementation, QA, documentation, and release.

    • Yeah. Being a tech writer is tough because few people appreciate the work.

      But the things I really need from devs is what is the feature supposed to do and why did you do it that way?

      I can read the code to know what it does but often that’s not what it’s supposed to do.

      The why can be simple too. We had a dev write an archive delete function that failed but the way was because the CEO pressured him.

      I’d love to know what you think documentation means.

    • It's not a binary thing... even just a few scattered "why we did it this way" comments in the code base is a lot better than no documentation at all.

    • My day job is product developer and I have written hundreds of pages of documentation. The key is to write it as you go along. Not to wait until the release is ready to go!

This is also the reason Olkiluoto Nuclear Plant Unit 3[0] cost so much.

Nobody had built a nuclear reactor in ages. The last ones were from the 80's and this was a completely new technology (EPR). There was no institutional knowledge.

It didn't help that the French attitude to building was to "just slap it up real fast up north and use it as a reference to get REAL customers". They didn't figure in the fact that STUK (the Finnish radiation authority) is _really_ fucking good at what they do and don't cut any corners for any reason resulting in the French having to build many parts twice because the first attempt was subpar.

[0] https://en.wikipedia.org/wiki/Olkiluoto_Nuclear_Power_Plant [1] https://en.wikipedia.org/wiki/Radiation_and_Nuclear_Safety_A...

There’s strategic bidding as well. Specifications cannot cover every conceivable occurrence over the course of a 4 year construction project, so contractors can structure their bid to be low upfront with big pick ups later for change orders when issues arise.

  • Such tricks, however, are known. The further trick is that those looking at bids can flag gaps or not depending on their connections to the bidders.

    • Only a if the government people in charge care and know. My friends in the federal contracting space are either starving or backing the truck up and looting the place.

    • You’d be surprised how that game plays out… or maybe you wouldn’t if you’ve seen how far over budget public construction projects tend to go.

Thank you for bringing this up. This is profoundly true for big projects (toll roads/transport) and small infra projects (e.g. community solar). The length of time that it takes to develop things like this, combined with the turnover and the sheer amount of context that single developer has to have to be successful with it, is one of the driving forces in why development is such a difficult/risky business.

It's one of the most consequential problems imaginable to solve, particularly as the US begins to realize that we need to compete with decades of China's subsidized energy and industrialization/manufacturing capacity.

Taking it a level deeper, what most don't realize is that infrastructure is an asset class: before someone funds the construction of $100M of solar technology, a developer will spend 2-5 years developing 15 or so major commercial agreements that enable a lender/financier to take comfort that when they deploy such a large amount of cash, they'll achieve a target yield over 20+ years. Orchestrating these negotiations (with multiple "adversaries") into a single, successfully bankable project is remarkably difficult and compared to the talent needed, very few have the slightest clue how to do this successfully.

Our bet at Phosphor is that this is actually solvable by combining natural language interfaces with really sophisticated version control and programming languages that read like english for financial models and legal agreements, which enables program verification. This is a really hard technical challenge because version control like Git really doesn't work: you need to be able to synchronize multiple lenses of change sets where each lens/branch is a dynamic document that gets negotiated. Dynamically composable change sets all the way down.

We are definitely solving this at Phosphor (phosphor.co) and we're actively hiring for whoever is interested in working at the intersection of HCI, program verification, natural language interfaces and distributed systems.

That’s going to be my new business - Subways, et cetera.

We just do subways and get good at it.

That makes sense. It seems like during the continuous "building up America" period of the late 40s through mid 70s there was no problem of getting shit done at reasonable cost, because of continuously available institutional knowledge.

Once large infrastructure projects become sporadic in nature, you begin to run into issues.

The solution has to be continuous stimulus, but that also runs into problems of corruption and capture by special interests (the longer the stimulus, the more incentive there is for 3rd parties to appropriate funds).

  • Somehow, other nations have managed to figure this out. Of the developed world, seemingly only Americans are resigned to the belief that such things are sadly impossible.

    • That's because we're richer and can object. The Europeans get bulldozed by their governments. It's why they're always protesting some online ID law or some "show your photo ID to browse Wikipedia" shit but no one listens to them.

      8 replies →

  • Robert Moses did a lot of bad that we don't want to repeat. We have gone too far the other way but those big projects often did come at high cost - but the cost wasn't dollars

    • Robert Moses was a governance problem. The biggest issue with him is that he was the smartest guy in most rooms, but nobody could control him. 1915 attitude in 1960 was an issue.

Not just that, it can be worse when different companies worked on different parts of the subway. It results in no overall vision and desire to see subway network as a whole to make it efficient and convenient to get from any part of the city to another.

Not just that, it can be worse when different companies worked on different parts of the subway in the city. It results in no overall vision and desire to see subway network as a whole.