Publishers' Association survey on subscriptions: methodological critique

Correction June 6: the 2006 ALPSP report Survey of Librarians on Factors in Journal Cancellation is on the ALPSP website, here: http://www.alpsp.org/Ebusiness/ProductCatalog/Product.aspx?ID=26 Only the press release and the Summary and Conclusions are available to non-members. The Association of Learned, Professional and Society Publishers (ALPSP) and the Publishers' Association just released the results of a survey asking if libraries would continue subscriptions if a majority of the content of research journals was freely available. The results of this study (downloadable from here) appear alarming - predicting catastrophic losses of subscriptions and subsequently journals and publishers! This post is a methodological critique. In summary, the recommendation of this study (against an OA mandate with a 6-month embargo) is not supported by the research presented - and most importantly, by the research omitted, which is in brief all of the considerable evidence that would counter this recommendation, including a 2006 study by ALPSP itself which is not cited and has apparently disappeared from the ALPSP website.

The recommendation from this study is:

It is strongly recommended that no mandate is issued on making all or most journal articles available free of charge after a six month embargo until both libraries and publishers have had time to understand the issues better and have together taken steps to explore alternatives to a fully open access publishing model which could be mutually attractive.

Here is the simply question asked by this ALPSP study:

If the (majority of) content of research journals was freely available within 6 months of publication, would you continue to subscribe? Please give a separate answer for a) Scientific, Technical and Medical journals and b) Humanities, Arts and Social Sciences Journals if your library has holdings in both of these categories.

Methodological critique:

1. The response rate to this study was 26%. With any survey, there is always the possibility of response bias. In the case of a study like this conducted in an environment of contested advocacy for public access policy, the chances of response bias to a survey conducted of librarians by publishers' groups is likely to be above normal.
2. A 2006 ALPSP survey which found that cancellations by librarians would likely be minimal even with IMMEDIATE free access would be minimal appears to be absent from the ALPSP website. Thanks to Mel DeSart for posting conclusions to this study on the SCHOLCOMM list. For example, even if 79% of content were freely available immediately on publication, only 10% of libraries surveyed indicated that they would consider cancelling the journals. That this study, conducted by the same association, which found very different results, was not cited and does not appear on the ALPSP website, strongly supports the idea that the study was biased - only the results supporting ALPSP's preferred conclusion are presented.
3. The recommendation of this study implies that the only possible source of revenue for scholarly journals is subscriptions. Even for traditional journals, this has never been the only source of revenue. For many a society publisher, for example, a key source of revenue has been association memberships rather than subscriptions, and many societies and associations have traditionally subsidized their journals (while in other cases the journals subsidize the association). Advertising is another source of revenue. In many countries (such as Canada), scholarly journal publishing is not seen as a profit-making venture, and so enjoys government subsidies. Article processing fees to provide for open access is a growing potential revenue source. A growing number of libraries provide funds to cover article processing fees for open access. Why did the study not ask libraries if they have such a fund, or would consider developing one if there was a mandate for open access within 6 months? Why not mention that there are now more than 7,000 fully open access journals - including profitable commercial journals, and hundreds of society journals as noted by Suber & Sutton?
4. A key recommendation of this study is that "no mandate is issued on making all or most journal articles available free of charge after a six month embargo until both libraries and publishers have had time to understand the issues better and have together taken steps to explore alternatives to a fully open access publishing model which could be mutually attractive". This is puzzling. If the researchers think that libraries and publishers should work together (I agree), then why was was this a publisher-only study? Wouldn't a combined library / publisher / scholar study - like the PEER project - be a better approach?  Further, this recommendation implies that the idea of libraries and publishers working together to make open access happen is a new one. As I've explained in a bit of detail in my previous post Society publishers: time to quit whining and make the leap to open access, these discussions have been going on for more than decade. Why not cite some of these discussions and related research, such as survey conducted by myself and other researchers across Canada to figure out how to help journals make the leap to open access? This is just one example!