Wednesday, March 27, 2013

Is DOAJ inadvertently promoting publisher power over scholars?

The Directory of Open Access Journals has a new feature, to narrow a search by Creative Commons licenses. This can be very useful for the article level search, as searchers may be looking for re-usable content. I use this type of limitation when searching flickr for photos that I can re-use, for example.

However, when this is used to calculate the percentage of DOAJ journals using a particular CC license, as Peter Suber does in this post this inadvertently makes the assumption that choice of licensing is made (should be made?) by journals and publishers, not by scholars.

Merely framing a research question in this way can limit the way that we think of potential answers. For example, there appears to be no way in this categorization to consider an approach such as that of First Monday, which provides authors with a range of choices. This is the option that I recommend for journals, the one that is most compatible with author choice and Freedom for scholarship in the internet age. The last thing that scholars need in the transition to open access is to replace subscription-publisher overlords with open access-publisher overlords.

This post is part of the Creative Commons and Open Access Critique series.

Wednesday, March 20, 2013

Attitudes and values regarding research communication

Following are my comments on the attitudes and values regarding research communication section from the Taylor and Francis survey (p. 7). For my overall comments, see this post. To avoid creating an extremely long blog post, I'll divide up responses into more readable bits.

Summary

The most telling piece of data on this page is support for the access problem. 38% disagree or strongly disagree with the statement that "Researchers already have access to most of the articles they need". This is strong support for the existence of a problem with access to scholarly articles for researchers, particularly coming from a survey of authors associated with a for-profit traditional publisher. To illustrate the bias inherent in some of the statements concerning money and whether publishers are essential to research communication process, consider one statement about money and research communication that was not asked:  Researcher salaries are more important than excessive publisher profits.

The statement that "publishers are an essential part of the research communication process" likely reflects both social desirability response bias (our responses are shaped by what we think people want to hear) and question bias (there is no way to state that publishers are disposable, or important or valuable rather than essential).

Responses to statements about the money involved in research communication may reflect the fact that researchers are usually not involved in the money aspect. For the commercial publisher, this is a multi-billion dollar a year business, but for the researchers themselves, this still looks and feels like a primarily gift economy. Responses to the statement about re-use of research result contradict results presented on the next page, and may be interpreted as a desire to facilitate re-use combined with a concern to ensure that re-use is appropriate and to the benefit of researchers and their works, supporting arguments that I make in my Creative Commons and Open Access critique series. Finally, the question about the importance of research data is too flawed to be meaningful. For example, there is no way of knowing how many researchers saw free access to research data as less important than free access to articles are working in disciplines or sub-disciplines that do not use research data.

Detailed comments

Three of the questions deal with money in various ways. The prominence of these questions may illustrate the publisher's bias. If researchers were designing questions about their colleague's attitudes and values regarding research communication, would questions about how this gets paid for be as close to top of mind as it would be at a for-profit company? The responses to these questions suggest not. One interpretation of these responses is that the majority of researchers seem to be in agreement with keeping money out of the picture for both authors and readers. This may reflect that this multi-billion dollar highly profitable business for a very few commercial scholarly publishers continues to function as a gift economy for the researchers themselves. They give away their work as authors and peer reviewers for free. Subscriptions and purchases are almost entirely paid for by the institution through the library, not the researchers themselves. They don't see the money trail, and they don't want to see it. This is healthy, from my perspective, but one should keep in mind when asking these questions that the respondents may be largely unaware of how the funding of this system works.

These are the statements I am referring to:
Publication of research should not be limited by ability to pay (86% agree or strongly agree)
All research outputs should be free for everyone to read online (66% agree or strongly agree)

The dissemination of research is a common good and should not be monetized in any way (67% agree or strongly agree).
The statement "Publishers are an essential part of the research communication bias" likely reflects two sources of bias. First, the question bias - respondents are led to consider whether publishers are essential - not whether they are disposable, and not even whether they are valuable or important but perhaps not essential. Second, social desirability response bias. This is a survey sent by a publishing company. We have a tendency to respond in the way people want us to respond, and this tendency might be higher among those who choose to respond to a publisher's survey. The 77% agree or strongly agree response has to be taken with a grain of salt. For a brief description of social desirability response bias and the related effect of acquiescence (yes-saying) bias, see Bowling, A. (2005).

To illustrate the bias inherent in both this statement and the statement concerning money, consider one statement about money and research communication that was not asked:  Research salaries are more important than publisher profits. 

Responses to the statement "There should be no restrictions on reuse of research outputs" contradict responses to the statements about dissemination on the next page. 82% agree or strongly agree that there should be no restrictions, but on the next page only 40% agree that their work should be reused in any way. Only 32% agree or strongly agree that it is acceptable...without my prior knowledge or permission...for "others to use my work for commercial gain". I argue that the fine points of understanding the consequences of various approaches to licensing for re-use are currently not fully known in my series Creative Commons and Open Access Critique. Taking the two contradictory responses together, one might suggest that there is a desire to facilitate re-use but that researchers would like careful thought to go into the terms and conditions, to ensure that there are no negative consequences for researchers. In other words, this contradiction supports my arguments on this topic.

The low positive response rate to the statement "Researchers already have access to most of the articles they need" - only 37% agree or strongly agree - may be the most telling of the questions on this page. This low positive response rate contrasts with what appears to be an overall tendency towards highly positive responses to the other questions on this page. Whether researchers have access to the articles they need or not is clearly within the authority of respondents to answer, unlike the other questions on this page. Responses to this question strongly support the need for open access. 38% of researchers either disagree or strongly disagree that researchers already have access to most of the articles they need.

The final statement "Free access to data matters more to me than free access to research articles" had a low positive response rate - only 23% agree or strongly agree. However, this is a highly problematic question. For example, much research in the humanities and social sciences is not data-driven, and for research that is data-driven, in HSS there are often complex issues around privacy that would need to be addressed before data sharing can happen. Because this question was not linked to a question about research data use in the researcher's discipline, it is highly likely that lack of interest in free access to data is conflated with lack of interest in data, period, and recognition of the problematic nature of data sharing. Then there is question bias. Even a strong free access to data advocate might not agree that this is more important than free access to research articles.

Reference

Bowling, A. (2005).  Mode of questionnaire administration can have serious effects on data quality J Public Health (September 2005) 27(3): 281-291 first published online May 3, 2005 doi:10.1093/pubmed/fdi031http://intl-jpubhealth.oxfordjournals.org/content/27/3/281.full

This post is part of the Taylor and Francis Open Access critique survey series




Monday, March 18, 2013

Taylor & Francis Open Access Survey: Critique

Taylor & Francis have just released early results of a survey of 14,700 of their authors on opinions relating to open access.

Kudos to Taylor & Francis for releasing early results before they complete analysis. Following are some thoughts which I will update a bit at a time, through a series of posts which I will link to from here, in line with T&F's plans to release analysis through a series of press releases.

Attitudes and values concerning research communication
Attitudes and values regarding dissemination and licenses

Overall comments

Added March 19, 2013: this survey presents an interesting twist on CC-BY. The PDF is licensed CC-BY but when I tried to save it I got this message: "Click "sign" to fill out and sign this form. When you are done, you can save a copy by clicking "done signing".


As an open access advocate, I would like to begin with applause at what I see as a survey with significant methodological problems which nevertheless is an interesting case study in its own right, and in particular because the questions selected suggest both a tilt in bias towards open access and a will to address the practical issues of how to make open access happen. Since I am discussing bias, I should mention my own - I am a librarian and a scholar and I advocate for Freedom for Scholarship in the Internet Age, which includes open access but also scholarly communication led by and for scholars and in the public interest, and I see corporate concentration in scholarly communication as a problem, with open access corporate and corporate-like not-for-profit concentration and power only slightly less problematic than toll access.

While this survey is somewhat useful, this kind of research should be led by academics, not publishers. While publishing companies can and should do research to develop their products and services (this is reflected to some extent in this survey), basic humanities and social sciences research, and particularly research designed to inform public policy developments, should be led by academics. The fact that this survey was conducted by a multinational conglomerate corporation (using the scholar-friendly-sounding brand names Taylor & Francis and Routledge) who obviously control access to this network of tens of thousands of researchers around the world and in effect have the ability to act as gatekeepers or exert monopoly control over the ability to conduct certain kinds of research in a practical manner should ring alarm bells over the extent of power that academics have handed over to publishers. I would encourage Taylor & Francis staff to consider this seriously and establish a committee led by academics to address this issue, to have academic researchers make decisions about surveys that go beyond the practical matters of publishing.

It is unfortunate that the survey questions were developed by the publishing companies' staff, who despite what might be the best of intentions bring a certain perspective and bias to this research. It would have been better to have taken more time to develop a better survey instrument before sending this to tens of thousands of researchers. Aside from the time this has taken up for Taylor & Francis staff and the respondents, with survey research respondent fatigue is a factor to take into account. Researchers who responded to this survey might refuse to participate in another, better designed survey in future.

For example, the following 3 questions reflect a strong bias against open access (p. 6 of the survey).  T&F questions are bolded.

"Open access journals are lower quality than subscription journals".

This is a leading question; it's inviting the respondent to think of open access journals as lower quality. Consider for example other possible questions, such as:

"Open access journals and subscription journals are about the same quality".

or

"Open access journals are the same or better quality than subscription journals".

The same can be said about the next two questions:

Open access journals have lower Production standards (copyediting and typesetting) than subscription journals.

Consider this possible alternative:

"Open access journals have the same or better Production standards (copyediting and typesetting) than subscription journals".

and

"There are no fundamental benefits to open access publication".

Possible alternative:

"There are fundamental benefits to open access publication". 

Comment: if you're going to ask tens of thousands of scholars to respond to a survey, you should take the time to get the questions right. It would have been appropriate to have the questions developed, or at least reviewed, by academics trained in social bias analysis, and preferably without an attachment either to toll access publishing or to Taylor & Francis per se.

Update March 18 - further comments in response to a comment by Thomas Pfeiffer on Richard Poynder's Google G+ post which alerted me to this study.

Thomas Pfeiffer:
Even though I do still agree that such politically sensitive surveys should be conducted by neutral organizations, I don't really agree with your analysis of the survey. To me, the questions on page 6 of the survey merely reflect common prejudices about OA, both positive and negative, and checks how widespread they are. It is usual in social science surveys to use questions framed both positively and negatively for the same construct, which is what they are doing here.
If they reframed the negative items, they'd also have to reframe the positive ones like "Open access offers higher visibility than publication in a
subscription journal" and "Open access journals are cited more heavily than subscription journals". In fact, question 1 has six "pro OA" items and only three "anti-OA" items. I don't see how that is supposed to set an anti-OA frame.

[Disclosure: I am a scholar as well and doing this kind of research has been central to my undergrad studies as well as my current career]

My comments

In one sense I agree, Thomas. This is 9 misleading or biased questions, with more biased in favour of open access than against. This is actually useful evidence - moreso, to me, in the sense that "a large publisher is choosing to including more questions in their survey biased towards open access than against it" than in terms of making the results meaningful.

If the aim is to get at researcher's attitudes, then this is the set of questions to ask with respect to quality - strongly agree to disagree:

Open access journals are lower quality than subscription journals.
Open access journals are the same quality as subscription journals
Open access journals are higher quality than subscription journals.

Even here, there is a strong probability of order bias in the questions. If you're doing a survey that based on a 19% response rate must have been sent to a hundred thousand researchers, you should control for this (for example, by changing the order of the responses on different survey forms).

Thanks for raising this, though, I'll add this to my post.

Thursday, March 14, 2013

A problem with CC-BY: permitting downstream use with no strings attached is the toll access model

The Creative Commons - Attribution (CC-BY) only license grants blanket permission rights for commercial use to any third party downstream. Proponents of CC-BY argue that this will open up the possibility for new commercial services to serve scholarship. This may or may not be; this is a speculative argument at this point. However, if this happens, this opens up the possibility that these new services will be made available on a toll access basis, because none of the CC-BY licenses is specific to works that are free of charge.

This is very similar to the current model for dissemination of scholarship. Scholarly research is largely funded by the public, whether through research grants or university salaries. Scholars must make their work public (publish) in order to continue to receive grants, retain their jobs and advance in their careers. They give away their work to publishers with no strings attached, often signing away all copyright. A few publishers have taken advantage of this system to lock up scholarship for their private profit.

One potential outcome of a CC-BY default for scholarship is a next generation of Elsevier-like toll access services. Many scholars and the public whose work was given away through CC-BY could be unable to afford the latest and best services made possible by their contributions. This is just one of the reasons to give serious thought to this matter before recommending a CC-BY default. For more, please see my Creative Commons and open access critique series.

Thanks to Heather Piwowar for posting an opposing view on google g+ that helped me to work through this argument.

Wikipedia, scholarship and CC-BY

-->
The Wellcome Trust's Robert Kiley, a long-time open access advocate, raised a common rationale for a default CC-BY license in a comment on Richard Poynder's interview of Mike Rossner.  Following are my comments. Summary: a scholarly CC-BY is not compatible with the Wikipedia conception of attribution, which involves anonymity. Permitting open re-use in Wikipedia with attribution (assuming the problem of Wikipedia anonymity is overcome) means that any Wikipedia editor can change the words of scholar, a situation that seems highly likely to result in scholars being incorrectly cited due to the edits of others. I am a fan of Wikipedia, have contributed as an editor in the past and may do so in the future, and am in favour of increasing the scholarly content in Wikipedia. However, I argue that what needs to happen is that Wikipedia policy and practices need to be more flexible to accommodate the needs of scholars and their works, rather than all scholars being required to give away all of their work for blanket commercial rights to any third party to suit the preferences of the current Wikipedia team. 

Following is my comment on this topic on Poynder's blog.

Robert Kiley says: Equally, the NC clause prohibits a user from sharing content at resources like Wikipedia. See table at: http://en.wikipedia.org/wiki/Wikipedia:FAQ/Copyright)

Comment: first, I would like to say that I am a fan of Wikipedia. I use Wikipedia, have contributed to Wikipedia in the past, and would like to see more scholarly content in Wikipedia. However, there are some problems with including scholarly content in Wikipedia, and I argue that these are not resolved by CC-BY.

The first problem is the difference in the expectations of attribution. For scholars, the key elements in attribution are things like the scholar, journal, and publisher. In Wikipedia, the norm is anonymity Attribution. This means that Wikipedia and scholarly works are not compatible from the perspective of Attribution.

Another reason for caution in including scholarly works in Wikipedia is the norm that anyone can edit. If the work of a serious scholar is deposited in Wikipedia, it could be "corrected" by someone with far less knowledge. If this is combined with Attribution (assuming the Wikipedia anonymity problem is overcome), then it seems highly likely that the result will be incorrect citations of scholar's works. This would harm rather than benefit scholarship.

Finally, if we want to see more scholarly works in Wikipedia, it is Wikipedia policies and practices that need to change to accomodate this rather than a wholesale transformation of scholarship to accomodate the preferences of the current Wikipedia team.

To return to Robert Kiley's comment: it is not the NC clause that prohibits a user from sharing content at Wikipedia, but rather Wikipedia's policies that do not permit works with NC clauses. Wikipedia is free to develop a more flexible policy at any time.

This post is part of my Creative Commons and Open Access critique series. 

Monday, March 11, 2013

Open Access Scholarly Publishers Association (OASPA) ironically demonstrates limitation of CC-BY

Update March 12: OASPA has added a downloadable spreadsheet with the data - thanks, OASPA! Note that the spreadsheet does not include a license - perhaps it is as obvious to OASPA as it is to me that data mining of the spreadsheet requires no such license?

The OASPA blog today features a chart illustrating Growth in the Use of the CC-BY License

Ironically this post demonstrates that CC-BY is not sufficient for one of its most touted goals, facilitating data and text mining. A picture-based chart is placed on a CC-BY licensed blog, but the data is not posted and there is no explanation provided of the method for collecting the data, which limits both our ability to interpret the data and (were the data available) our ability to mine or re-use the data.

If the data had been posted in downloadable format, then anyone could have mined it. There would have been no need to check the license of the blog.

One reason why some of us might like to download the data is to compare with other metrics. For example, I suspect that the growth curve of the use of the CC-BY license reflects to a large extent the overall Dramatic Growth of Open Access. In contrast with OASPA, my Dramatic Growth series features a full downloadable dataset. Feel free to data mine away!

Thursday, March 07, 2013

Wednesday, March 06, 2013

Predicting increasing costs and reduction in open access: comments on the Research Councils UK revised OA Policy and Guidance

The Research Councils UK (RCUK) has just issued a revised OA Policy and Guidance http://www.rcuk.ac.uk/documents/documents/RCUKOpenAccessPolicyandRevisedguidance.pdf

This is a stellar example of well-intentioned but poorly crafted government policy. I predict that this policy will increase the costs of scholarly publishing by creating an incentive for publishers to develop open access article processing fees with no incentive to keep prices reasonable and actually decrease access, by providing an incentive for journals to increase embargo periods (to force authors to choose the OA via APF).

Relevant sections of the policy:
Expectations of researchers:

Researchers, as the generators of all of the research papers and responsible for much of their peer review, are expected to publish any peer-reviewed research papers... in journals that are compliant with the RCUK policy on Open Access. 
Compliance of journals:
RCUK recognises a journal as being compliant with this policy if: 
The journal provides, via its own website, immediate and unrestricted access to the final published version of the paper, which should be made available using the Creative Commons Attribution (CC BY) licence. This may involve payment of an ‘Article Proces sing Charge’ (APC) to the publisher. 
Or,
The journal consents to deposit of the final Accepted Manuscript in any repository, without restriction on non-commercial re-use and within a defined period. No APC will be payable to the publisher. In this latter case, RCUK will accept a delay of no more than six months between on-line publication and the final Accepted Manuscript becoming Open Access. In the case of papers in the arts, humanities and social sciences (which will mainly be funded by the AHRC and the ES
RC), the maximum embargo period will be twelve months. In some circumstances, where funding for APCs is unavailable during the transition period, longer embargo periods may be allowable (see section 3.5).
Comment: this policy provides journals an incentive to offer an open access option via article processing fees which authors are forced to choose if the journal's embargo period is longer than what is acceptable to RCUK. The UK only produces about 6% of the world's scholarly literature, so OA to this literature will not enable UK libraries to cancel subscriptions. To maximize revenue, a journal can provide an OA via APF option at the price of their choosing and extend the embargo period to avoid having authors choose the self-archiving option. The majority of scholarship is not nationally based, so increased embargo periods are unlikely to be restricted to the UK. This means that the UK is likely to enjoy less access to non-UK scholarship in the coming years than would be the case if this policy had not been adopted.

Thus in spite of the best of intentions this is a poor policy and let's hope funders elsewhere do not look to this as a model. Fortunately in this case the US is getting it right.

Another problem with the policy is the assumption that licensing (CC-BY) can achieve the re-usability that is desired. As I've discussed in detail elsewhere http://poeticeconomics.blogspot.ca/2012/10/critique-of-cc-by-series.html, this just won't work. The result will be a corpus of CC-BY licensed locked-down PDFs or even more open documents with locked-down image-based charts and graphs that are useless for text and data-mining and re-use.