Comment by natrys
5 hours ago
At least the SSM layers should have relative PE built in via recurrence, in a hybrid attention model like this.
5 hours ago
At least the SSM layers should have relative PE built in via recurrence, in a hybrid attention model like this.
No comments yet
Contribute on Hacker News ↗