The results below are oldish, but interesting around the rate of retractions in the scholarly literature, and there is currently a bit of a debate going on around retractions (e.g. What a massive database of retracted papers reveals about science publishing’s ‘death penalty’ | Science | AAAS Steen RG, Casadevall A, Fang FC (2013) Why Has the Number of Scientific Retractions Increased? PLoS ONE 8(7): e68397. doi:10.1371/journal.pone.0068397 https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0068397 https://journals.plos.org/plosone/article/figure?id=10.1371/journal.pone.0068397.g001 The increase in retracted articles appears to reflect changes in the behaviour of both authors and institutions. ... (more)

eLife is trying another experiment in peer review. When they launched back in 2012 they introduced a form of peer review known as consultative peer review. They are now looking at a new iteration on the peer review idea. Trials in how peer review is done are quite rare, so I think this is going to be interesting to keep track of. The new idea is that once an article has been accepted for full review by one of the editors, the journal is going to publish the article, along with all comments. ... (more)

eLife’s peer review process is really good. One of the key attributes of this is that reviewers are not blind to one another, and they have to consult with one another. This largely removes the third reviewer problem. We also publish the decision letters and the author responses to the decision letter. Reviewers have the option of revealing themselves to authors. As with most review systms our reviewers know who the authors are. ... (more)

The upcoming altmetrics meeting, and a submitted abstract by Kelli Barr prompted me to note down some of my own thoughts on peer review and altmetrics. I would love to make it over to the meeting, but with just a few days now before my first child is born, it ain’t gonna happen. I’ve not read Kelly’s paper, but after reading the abstract my take home message from it would be something along the lines of “don’t replace peer review with altmetrics because you will just replace one bias with another, and at least with peer review the bias is contained within the academic community” ... (more)

Last year I was asked to contribute to a special issue on the evolution of peer review. I got quite excited about doing this, but then realised that I really didn’t have the time to write a paper. I’m not a practicing academic, I build products, and while at Mendeley I really had far too much on my plate to find the time to write up a paper. However the topic does interest me, and I am a strong believer that web scale technologies can help with the scientific communication process though a large number of avenues. ... (more)

The idea of megajournals had not really formalised in my head before, but at the COASP meeting the talk was all about “Megajournals”. [PLoSOne][plosone] is the archetype for this kind of journal, and it had not really struck me before as a huge revolution in the publishing industry, but after listening to a couple of days worth of talks on the topic I’m convincible. Megajournals are so called because they are structured to be able to publish many more articles than has been the normal practice with traditional journals. ... (more)

Session on engaging with peer review This is a very nice panel discussion. For my money there are a number of key points that arose during the discussion: discussions with the public needs to happen where the public is being half assed about engaging the public discourse around papers, and then hiding behind peer review when you run into criticism is really bad, as for example what happened with the arsenic story nasa and science the public needs to be educated that peer review is not binary peer review comments should be made public (not everyone agrees) where we have representations of papers we should look to link to conversations about those papers (trackbacks and so forth) There is a very interesting comment in the Q&A from [Martin Fenner][mf] about peer review in clinical medicine. ... (more)

Axel Boldt posted an interesting short paper discussing how to turn the physics ArXiV into an open peer review system. It’s a short read, about three pages, but if you are familiar with the problems around peer review then you can just jump to part three of the paper which is a little under a page. The solution proposed is to create a new role of editor on the ArXiV, and allow anyone to propose their paper for review. ... (more)

Below follows a long comment that I posted to an article on O'Reilly Radar about patents. The original blog post is here There is an interesting compliment to the patent system in the domain of assigning credit to ideas, which is the academic peer-review system. The inventor of the idea in the case of academia is the author. The equivalent to the patent office is the editorial board of the academic journal that the author submits to. ... (more)