Comment by dekhn
16 hours ago
Speaking as somebody who has worked within Google Research before: the researchers are under tremendous pressure to publish SOTA and sometimes they juice their results a bit to look competitive when they can't match. This is not uncommon in the field- it's remarkably easy to edit a paper to make yourself look good by omitting information.
One of the most egregious cases of this, in my opinion, is only publishing metrics that cover part of the confusion matrix. “The false-negative rate? That could not possibly matter for a variant effect prediction model; why would we include that in the paper?” Example: AlphaMissense.