← Back to context

Comment by slopinthebag

17 hours ago

I doubt AI will ever become omniscient

Hydrated sugars certainly think they themselves did.

  • if you mean this in a reductionist way for humans, i dont think most of us think we’re omniscient

    my point more broadly is that if you have an incomplete spec, any entity implementing it either needs to be omniscient or make assumptions about the intended behaviour of the system. if constraints or invariants aren't part of the spec it might be difficult or outright impossible to make the right assumptions.

    so while LLMs might be writing all of the code in the future, someone will still need to create the spec + verify it. and often the spec is not really possible to know until implementation begins...

    ive been using astra on a project that isn't even that complex in the grand scheme of things, and it still fails to make the right assumptions. of course i could just give it a full complete spec right?! except i didn't know what the right assumptions were either until implementation anyways. so to some extent they were unknowable to me, and thus unknowable to the llm no matter how powerful.

    I do think in the future humans will probably write very little code themselves, but the work will be all about specification + verification and that means system design, which is the stuff I care about anyways. maybe I won't develop rsi when I get older :)