Comment by lmeyerov
3 years ago
Both formal and informal peer review are why I like the FOIA, and standards / competition discussion to be open in general. I actually dislike closed peer review, or at least without some sort of time-gated release.
Likely scenarios, and that closed review hides:
- Peer review happened... But was lame. Surprisingly common, and often the typical case.
- If some discussion did come up on a likely attack... What? Was the rebuttal and final discussion satisfactory?
It's interesting if some gov team found additional things... But I'm less worried about that, they're effectively just an 'extra' review committee. Though as djb fears, a no-no if they ask to weaken something... And hence another reason it's good for the history of the alg to be public.
Edit: Now that storage and video are cheap, I can easily imagine a shift to requiring all emails + meetings to be fully published.
Edit: I can't reply some reason, but having been an academic reviewer, including for security, and won awards for best of year/decade academic papers, I can say academic peer review may not be doing what most people think, eg, it is often more about novelty and trends and increments from a 1 hour skim. Or catching only super obvious things outsiders and fresh researchers mess up on. Very diff from say a yearlong $1M dedicated pentest. Which I doubt happened. It's easy to tell which kind of review happened when reading a report... Hence me liking a call for openness here.
You get that the most important "peer review" in the PQC contest took the form of published academic research, right? NIST doesn't even have the technical capability to do the work we're talking about. My understanding is that they refereed; they weren't the peer reviewers.
Replying to your edit I've been an academic peer reviewer too. For all of its weaknesses, that kind of peer review is the premise of the PQC contest --- indeed, it's the premise of pretty much all of modern cryptography.