Comment by jsharf
3 days ago
Related story, while applying a firmware update to my Kawai CA49 piano, I bricked it due to flashing the wrong file (The process was broken, and I got desperate and tried something stupid, which bricked the piano). Claude walked me through looking for signs of life, and since OTA from the phone app wasn't working for me, it downloaded the Kawai Android APK, decompiled the Java, figured out the hardcoded key used for encrypting the firmware update. Extracted the piano firmware update, decrypted it, and then wrote a flashing script to program the piano from my laptop via bluetooth. My piano was back to working within an hour.
I can't imagine where we are headed. You understand every step of what it did and can appreciate the complexity but it'll only take a few generations for this to become something like magic to the tech priests beseeching the machine spirits for blessings
I think you're overestimating how much the average person knows about how technology operates today, or 30 years ago, or 1000. In some sense, we have been living with magic and tech priests since the Romans built the aqueducts. I wouldn't be surprised if widespread, cheap AI makes it easier for the average person to learn how things around them work, if they are so inclined.
I meet kids today who haven’t heard of Microsoft, who regularly play GTA and hand in assignments made in Powerpoint. 20 years ago I discovered that a friend didn’t know Xbox and Word were both from Microsoft. It’s really hard to understand what is common knowledge in different parts of society.
8 replies →
I'm always shocked by the amount of people that have been looking and using refrigerators their whole life.... And have zero idea how it works.
It looks to me that the far more common use case will be to manipulate technology rather than understand it.
The example with the synth is excellent. Today that kind of work demands somebody knowledgeable operate the AI harness. In short order, the AI may very well come up with the solution of looking online for example programs to decompile without the user even understanding what that means.
The point is, eventually not even experts will understand what it's doing
If religion and human technology are any guide, there will be a lot of this but it will never be the entire sum of human activity. Some of us are just too damn curious. We go straight for the curtain. I refuse to believe that very human pattern won’t continue.
"In the distant future, humans live in a computer-aided society and have forgotten the fundamentals of mathematics, including even the rudimentary skill of counting.
The Terrestrial Federation is at war with Deneb, and the war is conducted by long-range weapons controlled by computers which are expensive and hard to replace. Myron Aub, a low grade Technician, discovers how to reverse-engineer the principles of pencil-and-paper arithmetic by studying the workings of ancient computers which were programmed by human beings, before bootstrapping became the norm—a development which is later dubbed "Graphitics"." [1]
[1] https://en.wikipedia.org/wiki/The_Feeling_of_Power
I’m all for the sci-fi extremes that we might lose valuable skills to cognitive delegation, but the idea that we as a society will forget how to count is… extremely stupid.
1 reply →
I'm not convinced that's where we are heading. LLMs are really good at explaining things ("explain to me like I'm a 5 year old").
A post that lives rent-free in my head points out that a kid who is addicted to chatgpt is going to be more literate - and therefore likely better educated - than a kid who is addicted to tiktok
5 replies →
It's enough to make "explanation" a separate "educational" license to make it less broad used. Or disable it in some countries (this is happening already).
3 replies →
There's a big difference between having something explained to you and developing expertise in it.
I don't see an AI-as-explainer future where expertise isn't sacrificed en masse.
Capitalism rarely supports a currently economically unproductive alternative for future good reasons.
The recent AI tech layoffs are a warning sign that corporate leaders will happily shoot their company's (and the future's) expertise to pad next quarter's financials and trust in 90% correct, but much cheaper, AI.
1 reply →
Imagine someone in a position of power mandating that LLMs should not be good teachers.
5 replies →
I've been writing code since my teens, I've studied assembly... yet the fact that _things_ start happening when I press the power button on my computer are pure magic to me and I like it this way.
I started digging a few times, but, I prefer the "magic".
I prefer at least a superficial understanding.
Hopefully, there will never be a time when at least some folks are not reading books such as:
https://www.goodreads.com/book/show/44882.Code
I'm genuinely puzzled by how you know enough about a system to even understand there is a basic assembly language, but still consider how "switching on" is 'pure magic'.
Doesn't the one explain the other ? It may be turtles all the way down, but at some point there's a fundamental turtle - be it LEA or CMP ?
Keats blamed Newton for taken the magic out of the rainbow with a prism. Personally I think the magic only got greater.
Eh. The only real things you need are:
- On startup processing begins at a known address, and you put the bootloader code over there. Hardware engineers can guarantee this for you.
- Every time you execute an assembly instruction, the program counter either explicitly jumps to a new location or else it just increments by 1. Hardware people can also make this happen as easily as implementing an adder.
Don't get me wrong, there are LOTS of layers between the hardware and most "useful" programs any of us will ever write. But all of them are pretty understandable. They're often not very complicated, just tedious.
1 reply →
turtles all the way down
1 reply →
I think it will be just like Dr. Know in Spielberg's "AI" movie from 2001 — I found it amazing how the oracle, though giving mystic-sounding obfuscated answers, was actually intelligent enough to figure out (a) what the kid was asking for and (2) give the correct answer.
It is amazing how Dr. Know projects where AI is likely to go. And a Kubrick script, no less. Even the commercial overlap, where you pump in coins as the only way to get answers. Did it not also have ads? Truly prescient.
Kids grew up on this man, they are master prompters. You’ll be asking them to fix your holoTV and your crypto phone when you’re too old to read the brainfuck.
Honestly, don't think so. That's certainly the path one might extrapolate if the next generation grows up exactly the same way as the current generation, but that's not how it works.
They will be exposed to this technology throughout childhood as their brains develop and they will develop unique ways to work with it we don't entirely understand just like GenY with cell phones and GenX with home computers. I think you deeply underestimate how adaptable we are as a species, but if you consider that we've been running the same OS and Bios as a species for the past ~40K years, perhaps you might be more optimistic?
As I recall, the Dr. knows were programmed to feed that information to runaway mechas who, who were, in turn, programmed to seek out the blue fairy.
Probably a lesson in there somewhere.
Give it six more months and you'll have a second "oh shit" moment when you peek behind the curtain of LLMs shitting the bed.
I guess tech unsavvy people who are easily amused by LLM tricks will always exist, but they'll be an increasingly smaller minority as time goes on.
This is truly remarkable. Congratulations!
yeah thats mind blowing, ngl
Baller
That's sick.