Comment by psadri
4 years ago
An interesting next step would be to:
1. Give it a goal
2. A bank account + some spending money
3. Access to a web browser and an online identity (Eg gmail account)
Then have it try to achieve that goal.
It could work like:
- You are trying to achieve x. Outline the steps you need to perform
And then:
You are trying to achieve x. Your plan is (outline from initial prompt). You have already done a,b and c with the following outcomes. What’s the next step?
- do the next step
- repeat
I think eventually you'll run into prompt length limitations. I think (but am not even close to qualified in this field) that the attention mechanism is quadratic in complexity so either there's a hardcoded limit or you'll just run out of resources. I think the next major advancement would be to find a way to maintain internal/hidden state that's updated over time like an RNN does instead of having to append to a prompt