Embed Notice

Defending science in public we often talk about 'peer reviewed science'. But could this framing contribute to undermining trust in science and holding us back from improving the scientific process? How about instead we talk about the work that has received the most thorough and transparent scrutiny?

Peer review goes a step towards this in having a couple of people scrutinise the work, but there are limits on how thorough it can be and in most journals it's not transparent. Switching the framing to transparent scrutiny allows us to experiment with other models with a path to improvement.

For example, making review open to all, ongoing, and all reviews published improves this. When authors make their raw data and code open, it improves this.

It also gives us a way to criticise problematic organisations that formally do peer review but add little value (e.g. predatory journals). If their reviews are not open and observably of poor quality, then they are less 'thoroughly transparent'.

So with this framing the existence of 'peer reviewed' but clearly poor quality work doesn't undermine trust in science as a whole because we don't pin our meaning and value on an exploitable binary measure of 'peer reviewed'.

It also offers a hopeful way forward because it shows us how we can improve, and every step towards this becomes meaningful. If all we have is binary 'peer reviewed' or not, why spend more effort doing it better?

In summary, I think this new framing would be better for science, both in terms of the public perception of it, and for us as scientists.

#science #metascience #peerreview