The Bohannon "study" published in Science may have consequences beyond what was intended. In brief, Bohannon picked a list of journals largely from the list identified by Beall as predatory and submitted a bogus article that should have not passed peer review. A large number of journals accepted the article for publication. There is some benefit to this study as it confirms a problem first identified by Beall. However, this article reveals more problems than Bohannon or Science thought. First of all, this is a problematic study in itself, published in a prestigious journal but the Bohannon study itself was not peer reviewed, and I argue that no responsible peer reviewer or serious scholarly journal should have published this article in its existing form. Because the journals were cherry-picked from a list of journals known to be problematic, the logical conclusions at most provide supporting evidence for Beall's qualitative work. These results are clearly not generalizable, and the extent of the problem may be quite small (there are close to 10,000 fully open access journals listed in the Directory of Open Access Journals; the 157 journals that accepted this article, most not listed in the vetted DOAJ list, represent about 1.5% of open access journals. This study did not conduct a control group, submitting articles to subscription journals; nor did the author mention the literature on similar hoaxes with subscription journals, a substantive omission that should have been picked up by competent peer review and/or academic editing.
More worrying, however, is that while Bohannon and Science may have meant this as an attack on open access, this study could easily be picked up by those who oppose science and scholarship.
For example, an Economist article on this sting begins with a focus on the Sokal hoax; this was a subscription journal, not OA, so not focusing too strongly on OA is much appreciated by the OA movement. Kudos to the Economist for picking up what Science should have. However, this means that an Economist article is focusing on a critique of scholarly peer review rather than a problem with a small set of journals.
Similarly, a CBC article focuses on the problems with peer review, rather than problems with a few new journals that happen to be OA:
http://www.cbc.ca/thecurrent/episode/2013/10/11/why-a-harvard-scientist-wrote-a-bogus-paper-and-submitted-it-for-publication/
This article illustrates what I consider to be a potential danger to all of scholarship / science, not just open access. Here we have a newspaper article quoting a study as saying that the majority of peer-reviewed journals will accept an article that is obviously fabricated. It is not hard to imagine newspaper articles like this being used as fodder for climate change denial types.
To me, this in itself illustrates the need for careful quality control in scholarly communication. It is unethical for Bohannon and Science to publish an article that could so easily be misinterpreted in this way and used as arguments by opponents of science and scholarship. This is a bigger problem for science and scholarship than all of the predatory journals exposed by the Bohannon sting.
In conclusion: the Bohannon sting provides welcome evidence supporting Beall's qualitative evidence of serious problems with a small sub-set of new publishers who happen to be open access publishers. The study is not generalizable, the article was not peer-reviewed and should not have passed peer review in its current form. At minimum, every effort should have been made to ensure that readers understood the limitations of the study (small percentage of OA journals, non-generalizable study), and referred to literature on similar hoaxes that were accepted by subscription journals in the past.
