Comment by natrys
4 hours ago
At least the SSM layers should have relative PE built in via recurrence, in a hybrid attention model like this.
4 hours ago
At least the SSM layers should have relative PE built in via recurrence, in a hybrid attention model like this.
No comments yet
Contribute on Hacker News ↗