Comment by gjskngnf
17 hours ago
> The output of an LLM is useless without a human mind to comprehend it.
I don’t understand mRNA vaccines, but they are useful to me. The output of an LLM could at least in theory be the design to a major technological advancement, and its implementation with automatic tooling. We don’t have to understand that for it to be useful.
I find this line of argument unconvincing because it ignores the alignment problem. We only administered mRNA vaccines after a large and diverse group of experts with different backgrounds and experiences came together and said that we should go ahead. We only administered them after several rounds of careful testing to ensure that they wouldn’t have damaging effects. The decision to administer mRNA vaccines for to large numbers of people for the first time was arguable risky and certainly controversial, and yet that’s what happened.
I have no issue with LLMs aiding human scientists, but imo we absolutely would need to understand your hypothetical advancements completely and thoroughly before surrendering any agency to them - the LLM’s risk analysis will not reliably reach the same conclusions as a democratic human society
An LLM could also in theory fake alignment and create technology that seems really useful but secretly harms humanity in the long run. If humans don't understand what it does, that possibility becomes more likely.
mRNA vaccines were created by humans who understood them. You exist within a society (a networked distributed information system) where you accept the judgement of the humans who understand and create this things, which is how you end up benefiting without personally understanding.
If we imagine Super Intelligence, where NO human is capable of understanding, then how would it ever be possible for any human to identify what is actually beneficial or not?
This resolves in a paradox, common to all magical thinking. You can certainly wish that some all powerful benevolent entity will solve all of your problems for you, but it is not likely to work out well.
None of this is new. It is the same delusions as alchemy and the same thing that tales about genies warn of.
> mRNA vaccines were created by humans who understood them
If authors understood them, why most of their predictions were wrong? Time to decompose spike proteins, side effects, distribution through brrast milk....
Event the most basic promise 100% protection against infection was wrong! They lied and we allowed infected people into hospitals!
It's incorrect to assume that the information you heard is 100% an accurate representation of the scientific understanding and not information filtered through "public health" which balances that with other interests. I'm not aware specifically what was known at the time about the initial vaccine formulation, but there are several examples where there was a disconnect:
- ~Feb 2020: COVID is known to spread through and linger in the air, not just spread via droplets or fomites; public health messaging fixated on simultaneous 6ft distancing and surface cleaning rather than air filtration and masking
- Spring 2020: it's known that masks reduce the risk of acquiring and spreading infection; in the US, public health messaging stated the opposite, in order to keep as many masks in supply to healthcare workers despite a major shortage of masks
- Jan 2022: it's know that an infected person can remain contagious for over two weeks; in the US, public health isolation guidelines are reduced to 5 days due to noncompliance and in response to pressure from the airline industry, which was experiencing a significant shortage of flight staff due to COVID infection
3 replies →
> why most of their predictions were wrong?
Citation needed.