Source: iunewind/iStockphoto
colleagues, argued for a new type of metric to capture the diversity
of the contemporary scholarly system—making manifest both the
heterogeneity of scholarly output and the impact of scholarship on
science and society alike. As stated: “Altmetrics expand our view of
what impact looks like, but also of what’s making the impact.” Priem defines altmetrics
as the “study and use of scholarly impact measures based on activity in
online tools and environments” and lists blogs, microblogs, reference
management systems, and data repositories as potential sources.
There has been a proliferation of activity around altmetrics since this
introduction, spurred in no small part by the growing emphasis of
funding agencies on the demonstration of impact “beyond academia.” Scholarly metrics have never been without criticism; however, the expansion of data and sources and the increased use for evaluation have brought renewed concern around the ethical principles of research metrics.
are some of the particular challenges facing altmetrics, including the
misappropriation of the term impact, the narrow scope of focus, and the
potential goal displacement of scientific activity:
but asserts that publishers can “showcase research impact”,
institutions can have a “richer picture of their online research
impact”, and researchers can monitor “personal research impact.”
Similarly, the tagline of Impactstory—a non-profit focused on aggregating individual-level altmetric data—urges you to “Discover your impact today.” Scientometric studies have
followed suit—regularly employing the term impact when discussing
altmetric measures. This is in keeping with hallowed traditions within
science evaluation—the simultaneously reviled and revered Journal Impact
Factor (JIF) continues to reign over scientific publishing and, in
doing so, makes metonymic the relationship between citation counts and
impact. It is no far stretch, then, to equate new metrics of scholarly
attention with impact.
seems, however, a great distortion of the meaning of impact. Does the
act of tweeting evoke an image of forcible contact? Does a save on
Mendeley represent the strong effect of an article on the user? The term
impact connotes far greater engagement and transformative effect than
is currently justifiable with altmetric data. A more persuasive claim is
that what is captured are metrics of attention of a scholarly
object—the nature of this attention is something much more complex and
far less understood. One can easily find examples of extremely high
Altmetric.com scores which are the result of a viral joke, proofreading error, or scientific hoax. Behind
these outliers are undoubtedly scores of articles whose recognition in
policy documents, popular press, and on social media is a legitimate
sign that the work is relevant and interesting to a broader public. How
to identify the underlying mechanism of altmetric attention remains a
critical challenge.
the mechanisms and motivations of altmetric attention is hampered by
the inability to accurately identify the public upon which altmetrics is
effecting change. The notion of a mythical science-tweeting-lay-public
is persistent in the narrative, yet absent from empirical studies. In a recent study of tweets to journal articles from PLoS ONE, PNAS, Science, and Nature,
we identified more than a third of those tweeting links to scientific
articles as holders of doctoral degrees—far exceeding the proportion of
doctoral degree holders in the general population. Knowledge about who
is generating traces of attention is a necessary factor in establishing
the credibility of altmetrics.
challenge is in the realization of the expansive goals of the altmetric
movement. The promise of altmetrics was a broadening of measures of
impact that took into account all the various ways in which scholarship
is produced and disseminated. However, most altmetric studies have
focuses exclusively on journal article metrics from relatively few
platforms. While it is laudable to demonstrate the attention an article
is receiving from these sources, it is a far cry from the foundational
message—that is, that scholarship is no longer confined to “slow rigid
formal communication systems.” Where are the metrics that capture the
cacophony of scholarly activity? Researchers and practitioners must
think creatively about the types of scholarship which remain hidden,
despite the valiant efforts of the altmetric movement.
but perhaps most importantly, is the concern of goal displacement—an
inevitable byproduct of the promotion of altmetrics. Campbell’s Law
states the issue most precisely: “The more any quantitative social
indicator is used for social decision-making, the more subject it will
be to corruption pressures and the more apt it will be to distort and
corrupt the social processes it is intended to monitor.” Libraries have
begun to incorporate altmetrics into institutional repositories and to provide guidance
to researchers on how to document altmetrics on their CVs for the
purposes of promotion, tenure, and merit evaluation. The underlying
assumption is that an article has more worth if it has a higher
altmetric score and, by extension, a scholar‘s worth increases as more
articles receive mention. One might argue that review of scholars is
more nuanced and that reviewers would not fall prey to such gross
misinterpretation of data. However, as those who have studied citation
analysis or sought publication in a high JIF journal can attest, numbers
are persuasive to evaluators examining dozens of dossiers.
scientific community, administrators, and policy makers should take
care lest we let the tweet become the end in itself: topics should not
be chosen for their potential to go viral, nor should scholars spend
inordinate time managing their reputations online. Altmetrics should be
harnessed not to replace any existing metrics, but rather to expand the
tools available to demonstrate the diffusion of science. Responsible use
of altmetrics requires that we diligently seek to understand the
underlying mechanisms of measures of attention, expand our ability to
capture the diversity of traces of scholarly activity, and realize that
attention is not impact.
Attention is not impact" and other challenges for altmetrics | Wiley
No comments:
Post a Comment