Comment by traverseda
5 hours ago
I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.
5 hours ago
I'd be very interested in how a scratch trained model that started training with this neural circuitry would behave. Especially the storage space.
Along these lines I’m curious how a scratch trained model with a few built-in tools (memory, calculator, ?) available during a single generation would behave. Not tool calls as turns.
a single generation would still allow looping in the "CPU" because the instruction set can do it. The model is almost a Turing machine (except for the limit on memory)
As I understand, the problem of continual learning is that if you continue training, it will "overwrite" previous knowledge and this CPU will be lost
There is nothing in theory preventing you from training while keeping a subset of weights locked.
How do you ensure that subset of the weights will continue to be accessible through the others?
1 reply →
[flagged]