Comment by yapyap
3 months ago
> If you believe that AI development is a prime national security advantage, then you absolutely should want even more money poured into AI development, to make it go even faster.
This, this is the problem for me with people deep in AI. They think it’s the end all be all for everything. They have the vision of the ‘AI’ they’ve seen in movies in mind, see the current ‘AI’ being used and to them it’s basically almost the same, their brain is mental bridging the concepts and saying it’s only a matter of time.
To me, that’s stupid. I observe the more populist and socially appealing CEOs of these VC startups (Sam Altman being the biggest, of course.) just straight up lying to the masses, for financial gain, of course.
Real AI, artificial intelligence, is a fever dream. This is machine learning except the machines are bigger than ever before. There is no intellect.
and the enthusiasm of these people that are into it feeds into those who aren’t aware of it in the slightest, they see you can chat with a ‘robot’, they hear all this hype from their peers and they buy into it. We are social creatures after all.
I think using any of this in a national security setting is stupid, wasteful and very, very insecure.
Hell, if you really care about being ahead, pour 500 billion dollars into quantum computing so u can try to break current encryption. That’ll get you so much further than this nonsensical bs.
You can choose to be somewhat ignorant of the current state in AI, about which I could also agree that at certain moments it appears totally overhyped, but the reality is that there hasn't been a bigger technology breakthrough probably in the last ~30 years.
This is not "just" machine learning because we have never been able to do things which we are today and this is not only the result of better hardware. Better hardware is actually a byproduct. Why build a PFLOPS GPU when there is nothing that can utilize it?
If you spare yourself some time and read through the actual (scientific) papers of multiple generations of LLM models, the first one being from Google ~~not DeepMind~~ in 2017, you might get to understand that this is no fluff.
And I'm speaking this from a position of a software engineer, without bias.
The reason why all this really took off with so much hi-speed is because of the not quite expected results - early LLM experiments have shown that "knowledge" with current transformers architecture can linearly scale with regards to the amount of compute and training time etc. That was very unexpected and to this day scientists do not have an answer why this even works.
So, after reading bunch of material I am inclined to think that this is something different. The future of loading the codebase into the model and asking the model to explain me the code or fix bugs has never been so close and realistic. For the better or worse.
This line of thinking doesn't really correspond to the reason Transformers were developed in the first place, which was to better utilize how GPUs do computation. RNNs were too slow to train at scale because you had to sequentially compute the time steps, Transformers (with masking) can run the input through in a single pass.
It is worth noting that the first "LLM" you referring to was only 300M parameters, but even then the amount of training required (at the time) was such that training a model like that outside of a big tech company was infeasible. Obviously now we have models that are in the hundreds of billions / trillions of parameters. The ability to train these models is directly a result of better / more hardware being applied to the problem as well as the Transformer architecture specifically designed to better conform with parallel computation at scale.
The first GPT model came out ~ 8 years ago. I recall when GPT-2 came out they initially didn't want to release the weights out of concern for what the model could be used for, looking back now that's kind of amusing. However, fundamentally, all these models are the same setup as what was used then, decoder based Transformers. They are just substantially larger, trained on substantially more data, trained with substantially more hardware.
What line of thinking you're referring to?
Transformers were aimed to solve the "context" problem and authors, being aware that RNNs don't scale at all neither do they solve that particular problem, had to come up with the algorithm that overcomes both of those issues. It turned out that the self-attention compute-scale was the crucial ingredient to solve the problem, something that RNNs were totally incapable of.
They modeled the algorithm to run on the hardware they had at that time available but hardware developed afterwards was a direct consequence, or how I called it a byproduct, of transformers proving themselves to be able to continuously scale. Had that not be true, we wouldn't have all those iterations of NVidia chips.
So, although one could say that the NVidia chip design is what enabled the transformers success, one could also say that we wouldn't have those chips if transformers didn't prove themselves to be so damn efficient. And I'm inclined to think the latter.
2 replies →
> the first one being from DeepMind in 2017
? what paper are you talking about
https://arxiv.org/abs/1706.03762
2 replies →
That sounds to me like dismissing the idea that a Russian SSBN might cross the Pacific and nuke Los Angeles because "submarines can't swim".
Even if the machine learning isn't really intelligent, it is still capable of performing IF..THEN..ELSE operations, which could have detrimental effects for [some subset of] humans.
And even if you argue that such a machine _shouldn't_ be used for whatever doomsday scenario would harm us, rest assured that someone, somewhere, who either does not understand what the machines are designed to do or just pretends that they work like magic, will put the machines in a position to make such a decision.
One could hope...
Even at the height of the Cold War there was always a human between <leader presses button> and <nukes go aflyin'>.
--edit--
...which has me wondering if a president even has the constitutional authority to destroy the entire planet and if one could interpret their command as a 'lawful order'. Makes one think.
On the topic of fail-deadly nukes:
https://en.wikipedia.org/wiki/Dead_Hand
> They think it’s the end all be all for everything.
Is (human-based) general intelligence not one of the fundamental enabling elements of literally every human activity throughout history, regardless of how many layers of automation and technology one has to peel back to get to it?
Can you maybe imagine how the ability to create arbitrary amounts of general intelligence, completely divorced from the normal lengthy biological process, could upend that foundation of human activity?
> They have the vision of the ‘AI’ they’ve seen in movies in mind, see the current ‘AI’ being used and to them it’s basically almost the same, their brain is mental bridging the concepts and saying it’s only a matter of time.
I've found that most AI-related movies exclusively focus on "quality ASI" scenarios, which are mostly irrelevant to our current state of the world, as an immense amount of danger/value/disruption will arrive with AGI. People who are seriously reasoning about the impacts of AGI are not using movies as references. "Those stupid movie watching idiots" is just a crutch you are using to avoid thinking about something that you disagree with.
> Real AI, artificial intelligence, is a fever dream. This is machine learning except the machines are bigger than ever before. There is no intellect.
Do you have any evidence to support this conclusion? And does it even matter? If "fake intellect" can replace a human, that human still has to deal with the very real issue or not having a job anymore. If "fake intellect" is used to conduct mass surveillance, and direct suppression activities towards divergent individuals, those individuals are still going to have a bad time.
>> Real AI, artificial intelligence, is a fever dream. This is machine learning except the machines are bigger than ever before. There is no intellect.
> Do you have any evidence to support this conclusion? And does it even matter? If "fake intellect" can replace a human, that human still has to deal with the very real issue or not having a job anymore. If "fake intellect" is used to conduct mass surveillance, and direct suppression activities towards divergent individuals, those individuals are still going to have a bad time.
I think the "fake intelligence can replace a human" needs more support in general. We know how human intellect works practically (not theoretically) and we know how to apply it in different scenarios. We're still far from knowing how "fake intelligence" works and how to apply it to different scenarios.
I couldn't agree more.
If we're not talking about cyber war exclusively, such as finding and exploiting vulnerabilities, for the time being national security will still be based on traditional army.
Just a few weeks ago, italy announced a 16bln€ plan to buy >1000 rheinmetall ifv vehicles. That alone would make italy's army one of the most equipped in Europe. I can't imagine what would happen with a 500$bln investment in defense,lol. I don't agree with what Meloni's government is doing, but one of the ministers I agree more with is the defense minister Crosetto
Furthermore, what is being shown, at least for the time being, is that open source can be and is crucial in aiding developing better models. This collides with the idea of big, single "one winner takes it all" VC mentality (because let's be honest, these defense pitches are still made by startup/VC bros)
>italy announced a 16bln€ plan to buy >1000 rheinmetall ifv vehicles. That alone would make italy's army one of the most equipped in Europe.
So target practice for a beyond-the-horizon missile system launched ground-to-ground or air-to-ground? As an attacking force, conventional ground forces and tactics are a non-runner in a modern theatre of operations when faced against air and drone support. This is why no single EU country is incentivised into dumping money into any single area - as the only probable defense would be against USA/Russia/China to begin with.
The US proved it beyond doubt in Afghanistan - partisans simply haven't a chance against a gunship with IR or NV optics; the last time they levelled the playing field against air interdictors was in Charlie Wilson's Afghanistan when the Mujahideen took on that era of Soviet gunships with hand-held AA systems.
Italy had a joke of tanks in the last decade(s), that weren't clearly on par with the other European alternatives and the defense budget went to other places, namely navy and air force. Since some of the Italian tanks are being dismissed, there was the need for a new tank, but not having invested much in tanks had the effect of not having cutting edge technology to develop an alternative. So the decision was taken to produce tanks from rheinmetall (Leonardo was available to another German producer, but they didn't want to transfer technology and produce in italy).
Not having tanks is a big no no due to latest events, I do not see the grudge you hold against them when it's clear they wouldn't be used in the same flat scenario of Ukraine or Poland. Given that Italy is highly mountainous, it made sense to prioritize air force and navy first. I think they're also compatible for anti drone guns
Piaggio air space was acquired a few weeks ago by baykar, hopefully it will produce drones.
I do not get why you're saying that a change in the approach of war makes tank not useful anymore, when it's clear they will eventually be adapted to counter drones and similar threats
> I can't imagine what would happen with a 500$bln investment in defense,lol.
The $90,000 bag of bushings becomes a $300,000 bag?
No, at least in italy we would create some gimmick consultant position for some ex politician and repay him/her for past favours /s
It's not one or the other, though. AI-controlled drones are already a thing in Ukraine, today.
Been saying this for years, it's been fucking baffling. Generating images, video and text that sort-of resembles what a human would come up with is genuinely quite impressive. It is not "let's claim it'll fix our country" (looking at you, Keir) impressive though, and I cannot believe so much money has been pumped into it.
But you have to over promise and under deliver, otherwise you won't receive those sweet sweet money
Yes, I'd like to see some examples where our current AI can actually extrapolate rather than interpolate. Let it invent new things, new drawing styles, new story plots, etc. Maybe _then_ it will impress me.
Here you go: https://www.biorxiv.org/content/10.1101/2024.11.11.623004v1
I'm not convinced. This is using the tooling and paradigms invented by humans.
4 replies →
Can you? Most people can't do anything that isn't 99% derivative.
I can only say that exponential curves grow nominally sublinearly before they take off. AI is not quite at the obvious take off point, but owners of the biggest clusters have seen the extrapolations and it isn't pretty - once your competitor achieves take off and you aren't anywhere close, you're done for. The risk of not participating in that are too great.
> This is machine learning except the machines are bigger than ever before. There is no intellect.
Define "intellect".
What is even the possible usage of AI for national security? Generating pictures of kittens riding nuclear weapons to the very end like in Dr Strangelove?
> What is even the possible usage of AI for national security? Generating pictures of kittens riding nuclear weapons to the very end like in Dr Strangelove?
For all that critics of AI dismiss them as lacking imagination, your reaction suggests a lack of imagination.
Off the top of my head: facial recognition and identification to make "smart" guns that hit specific targets with reduced collateral damage (as found on most digital cameras even before smartphones); creating and A/B testing propaganda campaigns; using modified wifi signals as wall-penetrating radar capable of post estimation, heart rate and breathing monitoring[0]; take any self-driving car's AI and conditionally invert the part that says "don't hit pedestrians" when a certain target is spotted; ANPR to track specific vehicles with known owners over long distances; alternative targeting system for cruise missiles in the absence or jamming of GPS systems; using them as red teams in war-game exercises; using them to automate intrusion detection by monitoring for changes to background distributions of basically every measurable event; person-tracking by watching CCTV in secure areas; control systems for security robots (think Boston Dynamics' Spot) that are currently in deployment.
There's likely a lot more, too.
[0] https://openaccess.thecvf.com/content_cvpr_2018/papers/Zhao_...
Gentlemen you can't use AI here, this is the deep learning lab!
Lol: Where I live (Memphis) both “one” and “two” are considered two syllable words. Seriously. Our kids were taught this in the best public elementary school.
How does that work, one-uh, two-uh?
1 reply →
Autonomous weapons.
Agreed. I was working on some haiku things with ChatGPT and it kept telling me that busy has only one syllable. This is a trivially searchable fact.
link a chat please
It wasn't just busy that it failed on. I was feeding it haikus and wanted them broken into a list of 17 words/fragments. Certain 2 syllable words weren't split and certain 1 syllable words were split into two.
Also the narrative that we are currently on the brink of Ai explosion and this random paper shows it has been the same tired old story handed out by ai hawks for years now. Like yes, I agree with the general idea that more compute means more progress for humans and perhaps having a more responsive user interface through some kind of ai type technology would be good. But I don’t see why that will turn into Data from Star Trek. But I also think all these ai hawks kind of narcissistically over value their own being. Like blink and their lives are over in the grand scheme of things. Maybe our “awareness” of the world around us is an illusion provided by evolution because we needed it to value self preservation whereas other animals don’t. There is an inherent belief in the specialness of humans that I suppose I mistrust.
> But I don’t see why that will turn into Data from Star Trek.
"Is Data genuinely sentient or is he just a machine with this impression" was a repeated plot point in TNG.
https://en.wikipedia.org/wiki/The_Measure_of_a_Man_(Star_Tre...
https://en.wikipedia.org/wiki/The_Offspring_(Star_Trek:_The_...
https://en.wikipedia.org/wiki/The_Ensigns_of_Command
https://en.wikipedia.org/wiki/The_Schizoid_Man_(Star_Trek:_T...
Similar with The Doctor on VOY.
Even then, what we have with LLMs is basically already at the level of the ship's main computer as it was written in TNG/DS9/VOY.
But what counts as personhood? No idea. We're as ignorant as of that as the writers of the TV show; humanity's best attempts at philosophy of mind are still making the same kinds of arguments today as we were a century ago, the only real shift is to be slightly more inclusive of other species besides Homo Sapiens, and the only real improvement over two centuries ago is to include all of Homo Sapiens.
The main computer does not make choices stochastically and always understands what people ask it. I do not think that resembles the current crop of LLMs. On voyager the ships computer is some kind of biological computing entity that they eventually give up on as a story topic but there is an episode where the bio computing gel packs get sick.
I believe data and the doctor both would be people to me. But is minuet? The woman created by the binars to distract riker so they could steal the ship? Star Trek goes back and forth as to who counts as a person or not. This is one of the points that is made in the episode where the doctor learns his hologram is cloned many times to work as miners on an asteroid and he says this is slavery. Does minuet get to say she doesn’t want to seduce riker if she doesn’t feel like it? Or is she a program that does whatever the programmer wants?
But moreover linking back to the original technology, does eqtransformer have awareness? Do we treat it as a thinking being that we enslaved to find earthquakes in data because it uses the same technology as ChatGPT? I don’t think anyone thinks we should do this but it seems like that is where people want to take it. That’s strange to me. At what point does data pipelines doing some kind of stochastic transformation and summarization of training data become an individual that presents a desire for autonomy like data or the doctor?
I think there’s lots of questions here to answer and I don’t know the answers to them.
1 reply →
I find the last part of the paragraph offputting and I agree
It used to be much easier to be conservative about AI, especially AGI, after living through three cycles of AI winters. No more. Dismissing it as “merely machine learning” is worse than unfair to the last decade of machine learning ;-)
The hard part now is relatively trivial. Does anyone think that there is a fundamental and profound discovery that evolution made purely by selection in the last 200,000 years? I mean a true qualitative difference?
Sure—-We call it language, which is just another part of a fancy animal’s tool kit.
Does anyone think there is an amazing qualitative difference between the brain of a chimp and the brain of a human?
No, not if they know any biology.
(Although that does not stop some scientist from looking for a “language gene” like FOXP2.)
So what did dumb mutations and 200,000 years of selection do that a group of dedicated AI scientists cannot do with their own genuine general intelligence?
Nothing—-nothing other than putting a compact energy efficient LLM with reinforcement learning on a good robotic body and letting it explore and learn like we did as infants, toddlers and teenagers.
Each one of us has experienced becoming a “general intelligence”. I remember it hit me on the head in 6th grade when I dreamed up a different way of doing long division. I remember thinking: “How did I think that?” And each one of us who has watched an infant turn into a toddler has watched it as an observer or teacher. This is what makes babies so fascinating to “play” with.
We have to give our baby AGI a private memory and a layer of meta-attention like we all gain as we mature, love, and struggle.
I read the linked article and as a neuroscientist I realized the “wait” cycles that improved performance so much is roughly equivalent to the prefrontal cortex: the part of the CNS most responsible for enabling us to check our own reasoning recursively. Delay—as in delayed gratification—-is a key attribute of intelligent systems.
We are finally on the door step to Hofstadter’s Strange Loop and Maturana’s and Valera’s “enactive” systems, but now implemented in silicon, metal, and plastic by us rather than dumb but very patient natural selection.
Karl Friston and Demis Hassabis (two very smart neuroscientist) figured this out years ago. And they were preceded by three other world class neuroscientist: Humberto Maturana, Francisco Valera, and Rich Sutton (honorary neuroscientist). And big credit to Terry Winograd for presaging this path forward long ago too.
> I think using any of this in a national security setting is stupid
What about AI enabled drones and guided missiles/rockets? The case for their effectiveness is relatively simple in terms of jamming resistance.
drone and missile guidance system development has been using ML for decades at this point. That's just as much "AI" as anything currently coming out of the LLM craze.
It's not just target guidance at this point. There are prototypes of drone swarms, for example.
Like a lot of AI boosters, would you like to explain how that works other than magic AI dust? Some forms of optical guidance are already in use, but there's other limitations (lighting! weather!)
Sure thing. The basic idea would be:
1) Have a camera on your drone 2) Run some frames through a locally running version of something like AWS Rekognition's celebrity identification service but for relevant military targets. 3) Navigate towards coordinates of target individuals
It isn't exactly magic, here's a video of a guy doing navigation with openCV on images: https://www.youtube.com/watch?v=Nrzs3dQ9exw
1 reply →
I think jamming resistance is a red herring. AI weapons will have their own failure modes due to jamming. Any sensor modality will have its own particular weakness. Also reasoning model malfunctions as well i.e. hallucinations.
Not to mention false GPS etc...
I would say that they don't require an 500bln$ investment. AFAIK, drone that help lock on target have started being used in Ukraine
I generally agree, piggybacking on innovations in smartphone GPUs / batteries will probably be enough to get locally running AI models in drones.
This somehow reminds me of a certain killer robot from a Black Mirror episode ;)
> This is machine learning
Yeah, I was thinking about this while trying to figure out author affiliations.
There was a Stanford paper a few years ago that dusted off some old intelligence concepts and the authors seemed excited about it.
But given the pace of AI, it's difficult to look in new directions. It will probably take an AI winter and some unbridled enthusiasm immune to burnout to make some real progress outside of feed forward neural networks.
You would solve the ai problem if you correctly defined what intellect even is.
I agree agi wont solve national security but saying this isn’t intelligence is false.
This is ai and trend lines point to an intelligence that matches or barely exceeds human intellect in the future.
You’re part of a trend of people in denial. When LLMs first came out there were hordes of people on HN claiming it was just a stochastic parrot and LLMs displayed zero intellectual ability. It is now abundantly clear that this not true.
We don’t fully understand LLMs. That’s why gains like COT are just black box adjustments that come from changing external configurations. We have no way to read the contents of the black box and make adjustments off of it. Yet idiots like you can make such vast and hard claims when nobody really fully understands these things. You’re delusional.
I agree that LLMs won’t allow us to make some super weapon to give us some edge in national security.
> then you absolutely should want even more money poured into AI development, to make it go even faster.
Indeed. People are welcome to go "all in" on whatever nonsense gambling they want to do with their personal investments, but national security demands actually thinking about things - adversarially. Because the enemy will as well.
It's perfectly possible to lose a war by investing in expensive superweapons that under deliver. The Nazis were particularly bad at this.