Friday, September 06, 2013

Thoughts on transforming peer review processes: lessons from funding agency initiatives?

Update September 9: the Open Scholar group in the UK is developing something very much along the lines of what I've described below (thanks to Bjoern Brembs). 

The Canadian Institutes of Health Research is in the process of developing a new, streamlined peer review support system. The development process includes testing IT solutions for coordinating peer review. Details can be found on the CIHR website


Could there be lessons from this kind of initiative that would further transition to a completely new form of peer review, for scholarly publications, building an overlay on open access archives? Houghton and colleagues (2009) in a major economic study of scholarly publishing in the UK found that this approach, while more transformative in nature, has the potential to achieve much greater cost savings than a shift to either gold or green open access combined with traditional publishing (both of which were found to lower costs as compared to subscriptions, even for one country moving on its own).

Here is a picture of what I think this might look like:

After a researcher completes a study funded by CIHR, the researcher deposits the preprint into the applicable open access archive (in this case, PubMedCentral Canada). CIHR then would use the technology and processes that they are currently investigating to streamline the peer review process applied at the application stage to coordinate the publication peer review process. When peer review is complete, the final article, with appropriate notice of peer review completion status, is also posted in PMC Canada. Articles can be cross-deposited in the institutional archives of all authors; ideally, this task would be automated.

Why do this? For starters, this ensures that the results of Canadian funded research are preserved in Canada, under the control of Canadian agencies reporting to the Canadian taxpayer. This would be a marked improvement over the current situation where a large portion of publications arising from Canadian taxpayer funded research end up as the "intellectual property"(1) of commercial publishers that report to shareholders, are generally not located in Canada and have no obligations to Canadians.

CIHR and the Canadian universities where the researchers work then have a wealth of published material to assist with demonstrating the value of the work that they do, and to drive website traffic to further demonstrate interest in the work that they do. This in turn can only help funding agencies and universities to receive the support that they deserve.

There are people in the publishing industry whose experience of peer review stems from their own organization's operations who tend to speak as if this was the only, or the most important, form of peer review of scholarly research. Perhaps this is a good time to bring into the discussion the far more complex peer review procedures that go into research before it is even funded - increasingly, research that is not even funded. Currently, for example, the rate of success of applications for Canada's Social Sciences and Humanities Research Council (SSHRC) is about 20%, with some applications being returned approved but not funded (due to lack of funding).

For researchers and universities concerned about prestige, the fact of funding agency success is a mark of prestige. If I had to choose between publishing with Elsevier and SSHRC funding, I know what I'd choose (even if I weren't part of the Elsevier boycott).

Are other research funding agencies conducting similar investigations? Is anyone looking into this?


Houghton, J., Rasmussen, B., Sheehan, P., Oppenheim, C., Morris, A., Creaser, C., et al. (2009). Economics implications of alternative scholarly publishing models: Exploring the costs and benefit: A report to the Joint Information Systems Committee. Loughborough University. Retrieved February 7, 2010 from


1. "Intellectual property" is in scare quotes to indicate that I consider this to be a fictional concept of questionable usefulness at best and considerable harm to society at worst.