Comment by jamespwilliams
4 years ago
The fact that it’s able to understand the Python code and come to the correct answer is pretty astonishing in itself, let alone the rest.
The progress in this area is insane. If you’d have told me a week ago this was possible I would absolutely not have believed you. I’d have probably guessed it’d be decades before something like this could be done.
Makes me wonder what they have that they haven't released.
Could a censored AI ever be as powerful as a uncensored one?
https://docs.google.com/document/d/1hfCiyjiotMZvtPlGN_0uqbK9...
Could a military-grade AI be ever more powerful as the public ones?
Once it's created something interesting in Python, ask it to randomly translate it to different languages.
Or have it design a whole SQL schema for you, then ask it to make it work in MongoDB.
And you’re aware that this is, at most, an incremental improvement over the existing GPT3 models?
This is the craziest part, a little clever fine tuning and this model has turned into a completely revolutionary tool
all this capability was already easily available in GPT-3 with some prompt engineering, this mostly just changed it from few-shot to zero-shot.
"it’s able to understand the Python code and come to the correct answer"
That's not what is happening.
Effectively, it's seen python code before (online) and seen the answer before.
There's no understanding.
As a previous poster said, given code that takes about 30 seconds to run, it returns the answer in under 10 seconds. It's not running the code. It's guessing the answer.
It's not running the code, true, but if it's not "understanding", then we (humans) and not "understanding" either. We receive inputs, learn patterns, and apply them; we're just using a different mechanism (that takes years to train and isn't scalable) and getting a slightly different result.
I think the biggest mistake we're making these days is that we somehow believe our intelligence is different from any other form of intelligence we might invent.
I'm so grateful that I got to witness the rise of the Internet & PCs, the Doom to Unreal Engine journey, and the transformer revolution, and that I'll get to see the 100x version of GPT.
I find it hard to believe that it’s seen that exact code online along with the output. It’s quite an unusual expression.
Also, it does not matter that the code takes 30 seconds to execute — I’m not claiming that the model is executing the code. I’m suggesting that the model could be understanding what the code is meant to do, and producing the output from that understanding. The code’s inefficiency is irrelevant in that process.
Exactly! It seems to understand enough about what the commands mean and is able to carry the context forward even if it isn't exactly executing them.
Sure someone must have posted online about ls and rm, but a jumbled combination of those where the model is able to keep track of files listed, created and deleted in the specific context of the chat? Mindblowing!
No that's not right either.
It's looking at the code, and comparing it to what it's seen online, and what code that looked kindof like that output.
That is exactly what you are doing when you glance at code before stepping through it in your mind.
Many of those are code tutorials (w3schools anyone?)
At least one chatbot prompt I’ve seen had it instructed to use the interpreter accessible in the underlying Jupyter instance to run Python code for certain purposes.
Would be surprising if it could actually execute code anywhere...