First, there is the question of whether the current push towards quantitative metrics makes any sense at all. I have talked about the problems of this kind of instrumental rationality is my book chapter, The implications of usage statistics as an economic factor in scholarly communication https://circle.ubc.ca/handle/2429/954 (In brief, usage stats are likely to have significant negative impacts, from discouraging use to discouraging important but not necessarily popular entire fields of research); and the second chapter of my draft thesis http://pages.cmns.sfu.ca/heather-morrison/chapter-two-scholarly-communication-in-crisis/ (search for irrational rationalization).
One example, from an interview study of scholarly monograph publishers I did recently, is the impact of pushing scholars to publish more books to obtain tenure. This pressure is not consistent with the time it takes to write books that are really worth publishing and reading; so in this instance, we have a quantitative metric intended to improve quality and productivity (of our academic staff) which appears to be lowering quality (more mediocre books, more book production when the problem for all of us is not enough reading material, but rather not enough time to read).
Research, both quantitative and qualitative, is my recommendation before putting too much stock in Total Impact or other alternative metrics as a replacement for Impact Factor in evaluating the quality of scholarship. Note that I do not recommend retaining Impact Factor, but rather minimizing or eliminating quantitative approaches to evaluating the quality of scholarship.
The use of social networking tools may be particularly problematic. One research area that I recommend is examining the effects of traditional biases. I would hypothesize that social networking tools would be likely to exhibit the following biases prevalent in modern society:
- men would show more impact than women
- minorities would have lower impact
- developing country authors would have lower impact
- introverts would have lower impact than extraverts
- authors adept at social networking tools would have more impact than authors less comfortable with these media
Another area to consider (for all kinds of tools, not just social networking tools), is the impact of funding considerations. If universities are relying more on corporate funders at a time in society such as today when the political will seems to be on the side of the wealthy who wish to cut spending on social justice, it is reasonable to hypothesize that there will be a smaller proportion of funding for social justice issues. What happens to research on poverty issues and its impact if there is less funding for research and academic positions in this area (hence fewer researchers), at the same time that there is less funding for government services and social workers (fewer potential readers and tweeters), and many of those poverty research is meant to help have lost their homes and jobs and may find it difficult to get internet access?