Comment by natrys

10 hours ago

At least the SSM layers should have relative PE built in via recurrence, in a hybrid attention model like this.

0 comments