Should altmetrics take a step back and reconsider what the main purpose /
research question is? I should suggest that what we need is an
alternative to the current power of the impact factor in assessing the
work of scholars. This may or may not involve metrics of any kind. My
suggestion for starters is that we need a system that is not as reliant
on metrics of any kind.
Having said that, some metrics studies that might actually be useful:
-
does an emphasis on quantity of publication increase duplication of
content and/or reduce quality? With respect to the latter, this is what I
have heard from senior experts in scholarly publishing and I think both
Brown and Harley touch on this in their reports - at least with respect
to books, pushing scholars to publish two books rather than one to get
tenure means pressure to publish in less time than it takes to write a
good book. So pushing for quantity seems likely to correlate with
reduced quality (a hypothesis worth testing?)
One advantage to
studying the disadvantages of pushing for quantity is that if the
hypothesis (quantity correlates negatively with quality) is correct,
then that is evidence that can reduce the workload of scholars -
something I expect that scholars are likely to support
Other possibilities:
- scholars might want to know about journals:
- average and range of time from submission to decision
- level of "peer" doing the peer review (grad student? senior professor?)
- extent and quality of contents (this has to be qualitative analysis; sampling makes sense)
Shifting
from a print-based scholarly communication system to an open access
knowledge commons, while retaining or increasing quality and reducing
costs, is possible - but it's not easy. It is worth taking the time to
think things through and get at least some stuff right.
