Comment by leogao
6 days ago
There's been some work (e.g RASP - https://arxiv.org/abs/2106.06981) on taking logical computations and compiling them into transformer weights.
6 days ago
There's been some work (e.g RASP - https://arxiv.org/abs/2106.06981) on taking logical computations and compiling them into transformer weights.
Sakana AI is also working on merging different transformer models together to combine skills.
https://sakana.ai/evolutionary-model-merge/