Comment by Xenoamorphous
6 hours ago
> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.
From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.
They used the quote in the movie, slightly tweaked.
I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).
Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.
This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.
Engineers versus "engineers".
I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)
1 reply →
In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.
The most famous example may be the perpetual war between architects and engineers/builders.
I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.
Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.
With luck, one out of 100 inventions will show promise on those points.
There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.
OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.
It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.
Perhaps popularity alone is not always the best measure of progress.
I worry that companies like Anthropic, which are moving toward RSI, may prioritize speed over the timely identification of irreversible risks.
Every time I prioritize speed over risks, I, too, end up with repetitive strain injuries.
Did you mean AGI? I'd think whatever else they are doing, LLMs are reducing RSI...
[dead]