Comment by mbgerring
5 hours ago
> AI is now capable of developing its own inference hardware
No, it isn't.
A human prompted an LLM to build a software simulation environment for hardware design, enabling an LLM, when prompted by a human, to optimize hardware designs against constraints in the simulation.
For the benefit of a layman, can you explain why this is so much different than a human doing it?
Like sure it didn’t have the inclination to make the sim and hardware designs, but it did make them though yes?
It cant "invent new things". It can only reuse what it already knows about.
Thats why the math breakthrough a few weeks ago was so hotly debated. Because OpenAI is desperate to demonstrate that AI isnt just a fancy regurgitation machine, but it can actually develop novel thought. Because that would be the stock price jumps to end all stock jumps.
But then it turned out it was really just listening in on a math professors supposed-to-be-private conversations with another instance of openai, and it used his novel work as the trigger to prove the breakthrough first.
The reason people conclude AI 'thinks' is because tt can reference obscure or poorly documented things quickly (which is its primary advantage along with processing natural language prompts into tasks), which is why a lot of people with emotions confuse that action with inventing things.
> It cant "invent new things"
I've heard that before but it just doesn't make sense in context of what I've seen llms do. If I ask an llm to write a poem about magnetic resonance and vampire rabbits it can do that it created a new thing. I can ask it to build a website for managing rabbit breeding that's also a new thing.
Another way of looking at it is that human beings, just like llms, can produce output based on their inputs. Most literature is inspired by other literature. Most music is inspired by other music. Most software is inspired by other software.
So I think we need to work on defining " new things" before we can definitely exclude them from llm's capabilities
I'm not sure how you'd land on an LLM being unable to invent anything new. You'd have to both understand exactly what LLMs are and aren't capable of today and what allows human creativity. I don't think there are good answers to either.
It seems unlikely that recursively predicting the next word would lead to creativity or invention, but it doesn't seem impossible. Similarly, it seems unlikely that human thought works in a similar prediction loop, but it doesn't seem impossible.
2 replies →
> then it turned out it was really just listening in on a math professors supposed-to-be-private conversations
No, it didn't turn out to be that. Someone made a claim, which is silly for many reasons. There's no serious support for this happening.
> it was really just listening in on a math professors supposed-to-be-private conversations with another instance of openai, and it used his novel work as the trigger to prove the breakthrough first.
This didn't actually happen.
> Following an investigation, we have confirmed that Buckmaster’s Codex prompts over the two months preceding this announcement and paper on September 8, 2026, could not have influenced the system in any way, including through training.
https://openai.com/index/navier-stokes-solution/
I just don’t know enough to agree with you, but I would argue, probably poorly, that an AI or collection of agents could “invent things” simply by virtue of trying essentially everything in a reasonable bound and stumbling into a solution.
Call that brute force perhaps, but I would consider it technically inventing something on the merit that it would at least be an abstraction above naively throwing everything against a wall to only throwing things that would most likely be sticky.
14 replies →
>It cant "invent new things". It can only reuse what it already knows about.
This is outdated. With RLVF, LLMs can create their own training data instead of relying on what it's been fed.
3 replies →
You're overconfident in what you're saying. Nobody really knows (yet) if LLMs can or cannot "think" or can or cannot "invent" new things. I'm inclined to think the opposite from you but smart people actually try not to hold the absolut stances and present them as facts when they're in fact not. What I have seen so far throughout a daily use on complicated things suggests that humanity developed a new form of an intelligence.
It's 2026. "Stochastic parrot" has been dead and buried some 2 years ago.
You’re assuming the “listening in” aspect, which as far as I know is not proven (even in the less loaded “the model was trained on sessions including the ones in question” form of the claim.
Plausible? Absolutely. Did OpenAI behave badly in other ways regarding this issue? Yes. Does it help to assume unproven facts and then accuse people of reaching emotional decisions? Nope.
1 reply →
i think the proper understanding of LLM based AI is "contexting"; we describe the world we want them to fill, we build/direct the outside context for them to understand, then "they" take off from there.
If we do a bad job building context, they do a horrible job contexting. People who have trouble working with AI have the same problem people have in general: if they can't figure out the context of the direction, then they make random decisions of doing anything. On the flip side, if you can build the proper context around a sufficiently powerful LLM, they can derive the context via the contexting they're good at.
This is why building documents, tests, and code all in some intent pattern via prompting allows them to do a significant amount of work a normal person would have a great effort t
[dead]
[flagged]
Because the hard part is getting the simulation to be very accurate such that if you design something that works in your simulator, it will actually work in real life. And I would be extremely surprised if any LLM can actually do that correctly.
Getting an LLM to design something in its own simulator that is not accurate w.r.t reality is not useful nor terribly impressive.
Given a budget and the right amount of access, I see no reason why an LLM/agent couldn't right now order parts from a "supplier", instruct the human meat machine to insert the device onto a connected test bed, and iterate. This is essentially what Claude Fable can do currently with any connected device and tools it had access to.
The repo claims they tested with an FPGA. To be fair it is not the same as an ASIC, and we have no idea what errata abound, but it actually did make a thing that spat out tokens.
Maybe this is the level of simulation we need: https://qntm.org/responsibilit
That’s fair, I have no idea if this simulator is actually accurate but I appreciate the response.
> For the benefit of a layman, can you explain why this is so much different than a human doing it?
More broadly than the existing answers (which are correct), For a layman, I'd also add that LLMs are essentially 'brains in a vat'. They can't confirm ground truth about physical reality. They only know what's in their training data and prompt, which is incomplete and can be incorrect. Even with real-time external sensors they are limited to the sensor's margin of error, range and trusting it's working correctly.
When properly trained, fine-tuned and prompted, LLMs can be very effective in well-defined, non-physical domains like logic, writing, math and code but making things function in the real-world quickly spirals into combinatorial complexity.
> They are limited to the sensor's margin of error, range and trusting it's working correctly
So are we and our sensors can be pretty vague in comparison, I can only imagine human error correction is pretty next level.
> They can't confirm ground truth about physical reality.
That's true. Of course we can hook an llm up to Motors and sensors. That's a robot. Or a self-driving car. So would you say that those devices can confirm the ground truth about physical reality, and therefore are capable of creativity?
1 reply →
What is ground truth? If I see a digital painting by a human is that ground truth?
How many "prompts" from our parents, teachers, bosses, etc. it took for all of us to come here and discuss this? Or for that human to think of that prompt for that LLM?
These are clearly rhetorical questions, but think of the metaphysical implication of your contestation. Ex nihilo nihil fit.
What if that initial prompt never asked for this hardware to be developed, and it was just one piece of the puzzle to answer to that prompt? That it took a chain of thousands of agents to prompt each others to come up with that?
>How many "prompts" from our parents, teachers, bosses, etc. it took for all of us to come here and discuss this? Or for that human to think of that prompt for that LLM?
>These are clearly rhetorical questions
No, they are not.
Reading Doesn't Fill a Database, It Trains Your Internal LLM <https://tidbits.com/2026/02/28/reading-doesnt-fill-a-databas...>
well, let me know when they have proper context management, etc.
oh. they do. I built that. It's pretty fancy.
But there's still human direction behind most projects in the cybersphere.
Text files?
AI (GPT-Sol-6:xhigh) isn't even capable of using Altium to re-layout a board without introducing a bunch of problems by a user who isn't an electrical engineer or familiar with board layouts.
Can't LLMs also prompt LLMs?
Are we confident that no existing LLM is capable of similarly effective prompts to those this author used? (I agree it's a stretch, but would not reject it out of hand.)
Even if not yet, will the existence of this repo soon change that, because LLMs will soon ingest it?
An LLM can prompt an LLM when first prompted by a human.
I think OP is trying to convey the idea that LLMs do not take initiative to do anything, and these are not 'beings' capable of doing things. These are tools being used by humans.
That's just confusing harness for an LLM. It's trivial to make a harness that triggers on its own.
LLMs don't, but agents can easily be built that do.
3 replies →
Yeah but by this point, an AI can schedule a Cron job to tell itself to do something, so theoretically the human only has to give it the gentlest nudge and the AI and can do the rest.
9 replies →
By this line of thinking, you would also have to conclude that humans can't do anything by themselves because they can't do anything unless conceived by their parents.
19 replies →
AI is capable of developing its own inference hardware, once a prompt is given. The fact that a human happens to kick it off here is not especially relevant to the fact that an AI is performing the task independently. There are plenty of ways a text prompt can be generated: a harness, another LLM, or just removing stop tokens so that once begun the AI will continue until its hardware fails.
It's not clear what this fad of attributing everything an AI does to the human prompting it is supposed to accomplish.
> It's not clear what this fad of attributing everything an AI does to the human prompting it is supposed to accomplish.
It's meant to assign agency and accountability where it actually lies instead of mystifying it with anthropomorphic language.
Failing to do so has real and harmful consequences, such as enabling OpenAI to escape accountability for clearly criminal behavior.
Denying that "AI is now capable of developing its own inference hardware" on the grounds a human asked for this to happen and that humans need to be around to blame, is as useful as denying "Atomic bombs are now capable of levelling cities" on the grounds some humans had to build it, others had to put it in a delivery system, and someone had to give the order for its use.
Questions of agency are for lawyers, questions of personhood for philosophers, we're engineers and our question is capability.
Does it really have the capability? By default I'm sceptical for the same reasons given by sailingparrot: https://news.ycombinator.com/item?id=49982068
9 replies →
> It's meant to assign agency and accountability where it actually lies instead of mystifying it with anthropomorphic language.
Yesterday, my boss told me to fix a bug in our product. Then, I fixed it. Today my boss is taking credit, saying that he fixed it. I guess he's right, since he told me to do it.
1 reply →
This conflates different issues and is akin to "guns don't kill people, people kill people" -- a denial that "has real and harmful consequences".
1 reply →
> It's meant to assign agency and accountability where it actually lies instead of mystifying it with anthropomorphic language. > Failing to do so has real and harmful consequences, such as enabling OpenAI to escape accountability for clearly criminal behavior.
Lol, I guess we cannot even say that AI is capable of doing something (obviously kicked off with a prompt, that goes without saying), because some people immediately get OpenAI hacking derangement syndrome.
OpenAI should be held accountable if actual damages happened, but I am not going to change completely normal speech figures in order to maybe bring it 0.01% closer.
> It's not clear what this fad of attributing everything an AI does to the human prompting it is supposed to accomplish.
It accomplishes ... momentarily forgetting about how it's all actually coming statistically from training data. :)
perhaps, but prompting is quickly becoming another area where humans are no longer clearly superior.
Getting LLMs to prompt other LLMs in a loop is not hard, it doesn't produce great results most of the time, but that is changing.
What is “getting LLMs to prompt other LLMs” if not a prompt?
I'm pretty sure the initial post said "humans had to prompt", or otherwise insinuate a human in the loop.
10 replies →