Comment by keiferski

This sounds very educated but I don't really see what it has to do with the comment you're responding to (or with AI).

> While it's a fair criticism, just because someone doesn't believe in a god doesn't mean the religious hardware in their brain has been turned off.

Max Stirner said that after the Enlightenment and the growth of liberalism, which is still very much in vogue to this day, all we’ve done is replace the idea of God with the idea of Man.

The object might be different, but it is still the unshakable belief in an idealised and subjective truth, with its own rituals and ministers i.e a religion.

I guess the Silicon Valley hyper-technological optimism of the past years is yet another shift from Man to religious belief in the Machine.

I agree that modern hyper-online moralist progressivism and QAnonism are just fresh coats of paint on religion, but that isn't similar to AI.

AI isn't a worldview; it's an extremely powerful tool which some people happen to be stronger at using than others, like computers or fighter jets. For people who empirically observe that they've been successful at extracting massive amounts of value from the tool, it's easy to predict a future in which aggregate economic output in their field by those who are similarly successful will dwarf that of those who aren't. For others, it's understandable that their mismatched experience would lead to skepticism of the former group, if not outright comfort in the idea that such productivity claims are dishonest or delusional. And then of course there are certainly those who are actually lying or deluded about fitting in the former group.

Every major technology or other popular thing has some subset of its fandom which goes too far in promotion of the thing to a degree that borders on evangelical (operating systems, text editors, video game consoles, TV shows, diets, companies, etc.), but that really has nothing to do with the thing itself.

Speaking for myself, anecdotally, I've recently been able to deliver a product end-to-end on a timeline and level of quality/completeness/maturity that would have been totally impossible just a few years ago. The fact that something has been brought into existence in substantially less time and at orders of magnitude lower cost than would have been required a few years ago is an undeniable observation of the reality in front of me, not theological dogma.

It is, however, a much more cognitively intense way to build a product — with AI performing all the menial labor parts of development, you're boxed into focusing on the complex parts in a far more concentrated time period than would otherwise be required. In other words, you no longer get the "break" of manually coding out all the things you've decided need to be done and making every single granular decision involved. You're working at a higher level of abstraction and your written output for prompting is far more information-dense than code. The skills required are also a superset of those required for manual development; you could be the strongest pre-LLM programmer in the world, but if you're lacking in areas like human language/communication, project/product management, the ability to build an intuition for "AI psychology", or thinking outside the box in how you use your tools, adapting to AI is going to be a struggle.

It's like an industry full of mechanics building artisan vehicles by hand suddenly finding themselves foisted with budgets to design and implement assembly lines; they still need to know how to build cars, but the nature of the job has now fundamentally changed, so it's unsurprising that many or even most who'd signed up for the original job would fail to excel in the new job and rationalize that by deciding the old ways are the best. It's not fair, and it's not anyone's fault, but it's important for us all to be honest and clear-eyed about what's really happening here. Society as a whole will ultimately enjoy some degree of greater abundance of resources, but in the process a lot of people are going to lose income and find hard-won skills devalued. The next generation's version of coal miners being told to "learn to code" will be coders being told to "learn to pilot AI".

Which then leads you to the question "who installed the hardware"?

They are literally publishing a book called "If you build this, everybody dies" and trying to stop humanity from doing that. I feel like that's an important detail: they're not the ones trying to create the god, they're the ones worried about someone else doing it.

What makes an artificial mind greater than ours?

Do you assume that someone will stumble into creating a person, but with unlimited memory and computational power?

Otherwise, if we are able to create this person using our knowledge, we will most certainly be able to augment humans with those capabilities.

I’m sorry, but are you arguing that an LLM is anywhere near a human mind? Or are you arguing about some other AI?

Before that quoted sentence you drew a line from the reformation to people believing that AI is inevitable, then went on to imply these people may even believe such a thing will happen without the involvement of people. These are generalizations which don't fit a lot of the literature and make their best ideas look a bit sillier than they are. It is situations like these that make me think that analogies are better suited as a debate tactic than a method of study.

To the contrary. I sped through my compsci capstone coursework first year of college and spent most of the rest of my time in philosophy, psychology, and sociology classrooms. The "hey if you squint this thing it looks like religion for the non-religious" perspective is just one I've heard countless times. It is perfectly valid to have a fact based discussion on whether there is a biological desire for religiosity, but drawing a long line from that to broadly critique someone's well-articulated ideas is pretty sloppy.

> I pointed out how the thought patterns are often the same across time, and the inability and unwillingness to recognize this is psychologically lazy, to over simplify.

It's not lazy to follow thought patterns that yield correct predictions. And that's the bedrock on which "AI hype" grows and persists - because these tools are actually useful, right now, today, across wide variety of work and life tasks, and we are barely even trying.

> And there are indeed examples of technological acceleration or dispersal which was deliberately curtailed – especially with weapons.

Name three.

(I do expect you to be able to name three, but that should also highlight how unusual that is, and how questionable the effectiveness of that is in practice when you dig into details.)

Also I challenge you to find but one restriction that actually denies countries useful capabilities that they cannot reproduce through other means.

> And there are indeed examples of technological acceleration or dispersal which was deliberately curtailed – especially with weapons

Which examples? Despite curtailment, new countries have acquired nuclear weapons over time.

Efforts to squash technology exist, such as cloning bans, and so on, but they will only work for so long. You might think I'm making a "predestination" argument here, but I'm not. I'm observing the powerful incentives at play (first past the post advantage), noting that historically technology has always advanced, and making a bet that technology will continue to advance. I am supremely confident in that bet. I could of course go out and protest, but there is also a part that doesn't seem present in the original post and your argument, many (maybe most) of us don't want to stop technological progress.

Communism used to be thought of in this way. It enabled societies to cast off old limitations and make remarkable progress. Until it didn't and Communists found themselves and their modernized society stuck well behind the rest of the world. Perhaps LLMs are a similar trap that will generate many lines of code and imagined images but leave us all stupid and with impaired executive function.

To use John Adams' separation of republics into the categories of "the many, the few, and the one," the few in our current day are unusually conflict-adverse both among each other and with respect to the people.

When faced with the current crisis, they look at the options for investment and they see some that will involve a lot of conflict with the many (changing the industrial employment arrangement, rearranging state entitlements), and they see see some that avoid conflict or change. Our few as they are got that way by outsourcing anything physical and material as much as possible and making everything "into computer." So they promote a self serving spiritual belief that because overinvesting in computers got them to their elevated positions, that even more computer is what the world needs more than anything else.

This approach also mollifies the many in a way that would be easily recognizable in any century to any classically educated person. Our few do not really know what the many are there for, but they figure that they might as well extract from the many through e.g. sports gambling apps and LLM girlfriends.

It's a bit like pattern matching the Cold War fears of a nuclear exchange and nuclear winter to the flood myths or apocalyptic narratives across the ages, and hence dismissing it as "ah, seen this kind of talk before", totally ignoring that Hiroshima and Nagasaki actually happened, later tests actually happened, etc.

It's indeed a symptom of working in an environment where everything is just discourse about discourse, and prestige is given to some surprising novel packaging or merger of narratives, and all that is produced is words that argue with other words, and it's all about criticizing how one author undermines some other author too much or not enough and so on.

From that point of view, sure, nothing new under the sun.

It's all too well to complain about the boy crying wolf, but when you see the pack of wolves entering the village, it's no longer just about words.

Now, anyone is of course free to dispute the empirical arguments, but I see many very self-satisfied prestigious thinkers who think they don't have to stoop so low as to actually look at models and how people use them in reality, it can all just be dismissed based on ick factors and name calling like "slop".

Few are saying that these things are eschatological inevitabilities. They are saying that there are incentive gradients that point in a certain direction and it cannot be moved out from that groove without massive and fragile coordination, due to game theoretical reasonings, given a certain material state of the world right now out there, outside the page of the "text".

Nobody serious is claiming theological predesination is based on "nothing", either. Talk about poor pattern matching.

> I think if we actually cared about making a better world, you'd take steps where each successive step is a positive one.

Yeah, but lots of people don't care about that, they care about acheiving their visions of power, and they need an excuse to justify other people suffering for them. They aren’t seeking long term improvements at the cost of short term suffering, they are using a mirage of utopia over the hill to sell people a deal which is only suffering, now and for however long they can be kept in line.

In other words, "if we cared about the world, we would only do things that line up with my personal political beliefs and my political beliefs are obviously correct"