Kudos to Taylor & Francis for releasing early results before they complete analysis. Following are some thoughts which I will update a bit at a time, through a series of posts which I will link to from here, in line with T&F's plans to release analysis through a series of press releases.
Attitudes and values concerning research communication
Attitudes and values regarding dissemination and licenses
Added March 19, 2013: this survey presents an interesting twist on CC-BY. The PDF is licensed CC-BY but when I tried to save it I got this message: "Click "sign" to fill out and sign this form. When you are done, you can save a copy by clicking "done signing".
As an open access advocate, I would like to begin with applause at what I see as a survey with significant methodological problems which nevertheless is an interesting case study in its own right, and in particular because the questions selected suggest both a tilt in bias towards open access and a will to address the practical issues of how to make open access happen. Since I am discussing bias, I should mention my own - I am a librarian and a scholar and I advocate for Freedom for Scholarship in the Internet Age, which includes open access but also scholarly communication led by and for scholars and in the public interest, and I see corporate concentration in scholarly communication as a problem, with open access corporate and corporate-like not-for-profit concentration and power only slightly less problematic than toll access.
While this survey is somewhat useful, this kind of research should be led by academics, not publishers. While publishing companies can and should do research to develop their products and services (this is reflected to some extent in this survey), basic humanities and social sciences research, and particularly research designed to inform public policy developments, should be led by academics. The fact that this survey was conducted by a multinational conglomerate corporation (using the scholar-friendly-sounding brand names Taylor & Francis and Routledge) who obviously control access to this network of tens of thousands of researchers around the world and in effect have the ability to act as gatekeepers or exert monopoly control over the ability to conduct certain kinds of research in a practical manner should ring alarm bells over the extent of power that academics have handed over to publishers. I would encourage Taylor & Francis staff to consider this seriously and establish a committee led by academics to address this issue, to have academic researchers make decisions about surveys that go beyond the practical matters of publishing.
It is unfortunate that the survey questions were developed by the publishing companies' staff, who despite what might be the best of intentions bring a certain perspective and bias to this research. It would have been better to have taken more time to develop a better survey instrument before sending this to tens of thousands of researchers. Aside from the time this has taken up for Taylor & Francis staff and the respondents, with survey research respondent fatigue is a factor to take into account. Researchers who responded to this survey might refuse to participate in another, better designed survey in future.
For example, the following 3 questions reflect a strong bias against open access (p. 6 of the survey). T&F questions are bolded.
"Open access journals are lower quality than subscription journals".
This is a leading question; it's inviting the respondent to think of open access journals as lower quality. Consider for example other possible questions, such as:
"Open access journals and subscription journals are about the same quality".
"Open access journals are the same or better quality than subscription journals".
The same can be said about the next two questions:
Open access journals have lower Production standards (copyediting and typesetting) than subscription journals.
Consider this possible alternative:
"Open access journals have the same or better Production standards (copyediting and typesetting) than subscription journals".
"There are no fundamental benefits to open access publication".
"There are fundamental benefits to open access publication".
Comment: if you're going to ask tens of thousands of scholars to respond to a survey, you should take the time to get the questions right. It would have been appropriate to have the questions developed, or at least reviewed, by academics trained in social bias analysis, and preferably without an attachment either to toll access publishing or to Taylor & Francis per se.
Update March 18 - further comments in response to a comment by Thomas Pfeiffer on Richard Poynder's Google G+ post which alerted me to this study.
Even though I do still agree that such politically sensitive surveys should be conducted by neutral organizations, I don't really agree with your analysis of the survey. To me, the questions on page 6 of the survey merely reflect common prejudices about OA, both positive and negative, and checks how widespread they are. It is usual in social science surveys to use questions framed both positively and negatively for the same construct, which is what they are doing here.
If they reframed the negative items, they'd also have to reframe the positive ones like "Open access offers higher visibility than publication in a
subscription journal" and "Open access journals are cited more heavily than subscription journals". In fact, question 1 has six "pro OA" items and only three "anti-OA" items. I don't see how that is supposed to set an anti-OA frame.
[Disclosure: I am a scholar as well and doing this kind of research has been central to my undergrad studies as well as my current career]
In one sense I agree, Thomas. This is 9 misleading or biased questions, with more biased in favour of open access than against. This is actually useful evidence - moreso, to me, in the sense that "a large publisher is choosing to including more questions in their survey biased towards open access than against it" than in terms of making the results meaningful.
If the aim is to get at researcher's attitudes, then this is the set of questions to ask with respect to quality - strongly agree to disagree:
Open access journals are lower quality than subscription journals.
Open access journals are the same quality as subscription journals
Open access journals are higher quality than subscription journals.
Even here, there is a strong probability of order bias in the questions. If you're doing a survey that based on a 19% response rate must have been sent to a hundred thousand researchers, you should control for this (for example, by changing the order of the responses on different survey forms).
Thanks for raising this, though, I'll add this to my post.