Comment by jazzprogramming

> nobody really understands why a medication works.

But they work hard to try to understand it because the more they do the better the results for peoplea health. Also it goes without saying that if someone didn't understand many of the things that we do understand then things would be worse for us all.

I feel like we just don't understand a lot about the human body still. Only a century and a half ago we had a US president die because a doctor rummaged around and damaged internal organs (with unwashed hands, because germ theory was still not uniformly accepted) to try to find a bullet on the wrong side of his body. We've come a long way since then, but it's not like we've been doing the field of medicine rigorously for all that long. The best strategy we have seems like pretty much the same as any other scientific field; come up with ideas based on what we know, try them out, and see if they work. That doesn't necessarily give us any actual insight into why it works though.

Yet it takes rigorous years long tests to understand effects of a medication.

That cant be true, theres a paper trail of sign offs of the most likely people to understand the probabilities.

Please define proof of "really understands why a medication works"

It reminds me of fynemans why do magnets work. Yeah sure does anyone really understand anything? Its metaphors all the way down

its a good thing that our lack of understanding of medicines has never killed anybody or have had any intended consequences

oh wait

That's what makes libertarian AI doomers so strange. "Letting unconstrained machines with human components loose on the world will be okay, but letting unconstrained machines with nonhuman components loose on the world will produce terminators." How can it be both one and the other?

But we can generally assume that the people who built and understand some technology will have the same desires and worries as us, and are bound by the same laws. For instance, if it was a team of strangers who designed the terawatt fusion plant, we can still assume with some confidence that they care about safe plant shutdown, failure recovery, ensuring there are no toxic side effects of operation, etc. Can we say the same when it is machines doing everything?

Imo the counter-argument is that in principle you can research and understand the mechanics of climate change or medicine. You just don't have the free time and/or the motivation. With AI that changes - you could spend your entire life trying in vain to understand its reasoning.

It reminds me of how some religious people say that science is effectively no different to religion because we all take expert opinions on faith. But the difference is that there is a well-defined pathway to understanding, if you wish to do so.

This seems to me like asserting everything in the universe is explicable by physics. It may be technically true, but still not relevant to understanding earthquakes.

(Don’t bother to argue this not true unless you disagree with the essence of the argument.)

Anyway, post-hoc explicability isn’t a counter-argument to the assertion that almost everyone takes almost all technology as magic, from medicine to computers.

I’m still trying to understand your argument. Are you saying that after the fact we understand AlphaGo move 37? But somehow we are never going to understand an LLM’s decision afterwards? Seems like a disconnected take to me.

This is a very Western take though.

The problem with "ignorance" in Western countries (particularly the US right now) is that it's very common for people who don't know to believe they know and form ignorant opinions that they often want to be applied society-wide in some way. You can see this with everything from climate change to vaccines.

In much of the world, even in middle income countries, people are comparatively poor and, in my experience living abroad in such countries for many years, much less concerned with "understanding" and forming opinions about everything under the sun. It doesn't mean they don't value education and are opposed to development/progress, but it does mean that they don't question whether the vaccine they're taking is the product of a conspiracy, think too deeply about why the river is flooding more often, etc.

They just deal with life the best they can and are more focused on supporting their families, enjoying what they can, etc.

Culture and religion play into this. The way secular and Judeo-Christian people look at the world is very different than, say, Buddhists, Muslims, Fulani tribespeople, and so on.

> but more importantly. it’s not my problem.

But it absolutely is! Those people can and do vote.

Some of it has to do with the fact that many people pushed forward training themselves in a way they understood would better themselves for the future. They put in time, and now their sense of value is in question. They're forced to think about things like value (of life, themselves, and hopefully others), and the things they want to remain valuable in the future.

It's interesting to me that you only mentioned the people using 'AI' in the 'short-term' ways, and not the ones that use it to better themselves in the 'long-term' ways. You can spend your own time focusing on either usage, it's really up to you and your concerns. Either group's sense of value is what determines their behavior. Where they spend their time and thinking must be elsewhere, and you disagree with it. Who judges the quality of time spent? You do.

Is it more useful to think about self-improvement, and how to navigate the future in ways that might help you re-establish value of yourself, life, and others? Acquiring knowledge is a struggle, the author mentioned this. There is also Plato's Allegory of the Cave, which highlights some of that struggle, a resistance to change. And we're all limited by time, our genes, our station in life.

The only way to help anyone out of the cave, is to help them believe something different about themselves. To help them believe there is good reason to spend time going deeper into knowledge, or at the very least, allow others with the passion and station for it to do so.

If not for the very least reason that it keeps us 'in the loop' of some central idea behind intelligence (prediction?). Or because we feel it keeps us safer, as a fallback measure because we acknowledge we have to trust other's knowledge to exist.

It's not fun, it's a physiological addiction, because we learned enough about our brains to hack dopamine and reward cycles for profit. It's literal abuse.

yeah, a lot of people are just watching 1 video after another.

Well there is this one framework, CSS, that is quite alive and well :)

But point taken.

Love it! I am now a Software Hermanuticist