Comment by westurner

4 years ago

[Scholarly] Code review tools; criteria and implementations?

Does JOSS specify e.g. ReviewBoard, GitHub Pull Request reviews, or Gerrit for code reviews?

2 comments

westurner

The reviews for JOSS happen on github[0] but the journal's not prescriptive about how you develop your package as long as the code is public. The criteria for the JOSS review are very clear[1].

I don't want to oversell the depth of the code review possible; not all of the reviewers will be fully expert in whatever tiny cutting-edge area the package is for (making correctness checks difficult beyond the test suite), and most of us are academics-who-code rather than research software engineers. But the fact it's happening at all is a great step forward.

[0]: https://github.com/openjournals/joss-reviews/issues [1]: https://joss.readthedocs.io/en/latest/review_criteria.html

westurner 4 years ago

Thanks for the citations. Looks like Wikipedia has "software review" and "software peer review":
https://en.wikipedia.org/wiki/Software_review
https://en.wikipedia.org/wiki/Software_peer_review
I'd add "Antipatterns" > "Software" https://en.wikipedia.org/wiki/Anti-pattern#Software_design
and "Code smells" > "Common code smells" https://en.wikipedia.org/wiki/Code_smell#Common_code_smells
and "Design smells" for advanced reviewers: https://en.wikipedia.org/wiki/Design_smell
and the CWE "Common Weakness Enumeration" numbers and thus URLs for Issues from the CWE Top 25 and beyond: https://cwe.mitre.org/top25/
FWIW, many or most scientists are not even trying to be software engineers: they just write slow code without reusing already-tested components and expect someone else to review Pull Requests after their PDF is considered impactful. They know enough coding to push the bar for their domain a bit higher each time.
Are there points for at least in-writing planing for the complete lifecycle and governance of an ongoing thesis defense of open source software for science; after we publish, what becomes of this code?