Comment by tekacs
1 day ago
While everyone has been worried about it becoming too easy to hack into things, something I've been worried about for the last few years is it becoming too hard to hack into things. Right now, all of these things are being broken into to be jailbroken and such, but the companies building these hardware devices have access to the models too.
It seems like a distinctly possible and sad world that they might get so much better at keeping their devices locked down by hardening them with the support of LLMs that future devices become an absolute bitch to jailbreak, even moreso than they were before.
I don’t think it’s symmetrical. As a company, in a lot of cases, you get one shot to secure the thing. And then all owners collectively then get infinite time to jailbreak it.
Tell that to the people trying to break the Xbox One. Took 12 years. As far as I know the latest console still isn't cracked fully. In the past a lot of security holes were due to a supplier not having their security game up to snuff. AI will help B tier security people become A tier so I suspect these Microsoft scenarios will be more common.
Game consoles are basically the top tier of locked-down-home-appliances, because they're sold at a loss with the expectation of making it up later. It's thus very important that the games be difficult to pirate for as long as possible.
Basically nothing else is like that. That does show the top tier of what companies can do if they try hard enough, but I don't think AI will get you there. I expect you still need someone to care _deeply_ to get everything just right to make it as difficult as possible, otherwise there's just so much room for one small mistake.
1 reply →
The immediate question as to why someone is taking so long to break the Xbox One is "Why would anyone want to beak the Xbox One?"
It has very little in the way of exclusives, and it can run emulators on Dev Mode.
It's pointless. That's why no one did for a decade.
Also, there will always be things we can do to a device that no single piece of compiled software can stop. If someone make the ultimate bitchchip, impossible to alter, someone will make the ultimate fukuchip, which works the exact same but can be attached directly around it with minor soldering.
OTA updates?
Only until consumers get fed up with everything from their TV to their coffee maker spying on them and start demanding IOT products that don't actually connect to the Internet and instead interact with a local gateway device. That may never happen, but the barriers to making all your gadgets cooperate and speak the same protocol instead of requiring vendor-specific apps are definitely getting lower.
1 reply →
My 2016 Kindle has been in airplane mode since 2018 and I don't plan on ever changing that.
Mod chips may help there, or UART debugging. You can’t OTA update the PCB inside.
I think this is true, but on the flip side it could become common to buy "blank" hardware where the user provides their own OS or firmware. In the past it was just too burdensome to do this so we expected the hardware vendor to provide the software, but we could enter an era where providing the software is relatively easy.
I think one area we are seeing this is the flood of hand held emulator gaming machines. Companies in China pump these things out and the community has provided numerous custom firmwares to the point a new one seems to come out every week. All vibe coded in Rust but who cares, if it plays the games and looks nice that's all you need.
Even better, there's all sorts of e-waste tier devices that can get a second life now. One of my many projects right now is bringing up my own software stack on the Echo Dot 2nd gen, there's tons of hardware that can be bought for almost nothing (or already sitting in a box :P) just waiting to be liberated.
The next generation of products may be much more aggressively locked down however. Even old stuff like the xbox 360 is still quite difficult to hack.
I used to be kind of indifferent to devices that let you flash your own firmware because I knew there was no way I'd ever bother writing firmware. But now the community is generating decent quality alternative firmwares for everything, devices having an unlocked bootloader could be a requirement going forward. If I buy an ereader in the future it will have to be fully unlocked so I can load whatever the latest community built firmware is.
1 reply →
Why the fuck is it normal for a device bought and owned by you to be inaccessible to you?
Companies say "security" and mean "security of our business model".
So many consumers are convinced that their phone or device got slower “due to age” as though it’s not a literal computer.
So many people think literal computers get slower "due to age" as well, and not mal/bloat ware.
I think Apple was found to slow down old iPhones at some point no?
Edit: https://www.businessinsider.com/apple-paying-113-million-law...
Nah, we just start opening the phones, cutting off some PBC patches, soldering some wires, and intercepting/synthesizing signals live. Keeping phones locked is just a zero sum game.
Not to tempt fate here, but this is why eFuses were adopted at the SOC-level.
Yes, and bus encryption. I used to think "if a human made it, a human can break it" but that may no longer be true with disparate access to frontier models.
5 replies →
Is it just entirely unprofitable to sell hardware devices unsubsidized without locking it all into restrictive ecosystems & services?
Normal profit is not enough any more. It must be 1000x return, infinite growth, monopolize an entire industry style profit.
erm since when was normal profit considered enough?
Normal profit is what an investor demands at the absolute bare minimum. They dont 'feel' better off if its not being beaten.
Nintendo manages to squeak out a margin on hardware unlike the other big players. However, this is only because they trail behind in specs, and their devices are simply cheaper.
My weird take: I wish Nintendo would ship an even less powerful system.
The switch and switch 2 are both just powerful enough that developers can release straight ports of their AAA titles that were developed for other systems, but at the cost of significantly degrading framerate/framepacing, visuals, nothing ever running at native resolution etc.
I would much rather it was more like the Wii and all their previous handhelds where the performance gap between it and the other contemporaneous systems was so big that developers had to develop bespoke games for the system, and ports had to be ground up rewrites.
Everything was so much more playable, and by buying those systems you had access to an entirely different library of games to choose from, rather than the current situation where it's just the first party Nintendo titles that differ from what's available on all the other more powerful systems.
2 replies →
If by "secure" you mean keep owners from using a device in whatever way they wish, then I hope whoever made that decision fails, and I wish them discomfort and inconvenience as karma for the attempt.
I would think the more viable path (in the US) would be the legal system. i.e. Amazon pressuring frontier labs to add "safeguards" to protect their IP.
I'm all for allowing tinkering on a Kindle, but my pacemaker is off limits
I dunno, I have a closed-loop insulin pump / CGM and it pisses me off that the algorithm on which my health and life depend is not available for me to look at, understand and evaluate. Likewise there are all sorts of artificial limits on getting even read-only access to my data, such as blood glucose. It feels wildly disempowering to me.
I imagine I would feel similarly about a pacemaker. How does it work? What algorithm does it use to determine what to do at any given moment? Does it have settings? Are they optimally chosen? Are there bugs? Security flaws?
At least for now, they have to build the hardware with the old shitty AI, and then we get to... let's just say "utilize", the hardware we own using the new and much smarter AIs.
Well, before they do, can someone jailbreak various cars?I am too scared to brick mine.
In Russia car mechanics "unlock" Chinese cars that are locked to be operated in China only (such cars are much cheaper than over-priced international versions having the same hardware).
if so that seems to make 'right to repair' all that more of an imperative.
Basically Apple devices, which are becoming ubiquitous.
There should be no need for hacking. If there is a right to repair, there should be a right to reprogram any computer - and penalty for implementing measures to prevent re-programming. Right to reprogram also means that the consumer gets a choice of firmware instead of being forced to use inferior factory firmware with limited capabilities and creates a possibility for firmwares market. Also reduces electronic waste.
Everybody except capitalists wins with right to reprogram.
What you’re essentially saying is right now we have weak human written code that might have some vulnerabilities, but something written by a frontier LLM may not have those.
More like companies can spend the money to have AI scan everything and pick off all the low hanging fruit. I work on a legacy codebase and these new AI tools are finding mountains of very stupid very serious security issues. After we finally get through them we will have a codebase that has no easily found exploits left.
Not exactly imo, you could spend a crapton of money with white box pen testing with a bunch of frontier llms, and it might become a bunch harder to break into (it generally costs more to attack than it does to defend). Not to mention formal verification might become possible for more devices which might make it nearly impossible to find exploits.
I doubt management will throw away margins to harden every device
1 reply →
It’s also very easy to get esp32 thingies today and run everything open source.
If anything apples closeness in the age of disposable software is going to kill it.
[dead]