Comment by trilogic

It is a nice work, however the domain specific finetuning will always be of higher accuracy prediction. Another thing worth noting is the sequence length used for the training (usually cut to 1024/2048) which is a game changer if left uncut.

I did have a bit of fun myself finetuning esm2 in domain specific bacteria (cause it gives better score) and comparing it to another model (self created) and self created beat it at 25% more accuracy. Then for the 3d structure was coded a 3d protein visualizer hypergraph with the upload file option and visualize instantly the result. 2 days job :)