Monday, 11 September 2017

Measuring the impact of your research | NUCATS: NU Clinical and Translational Sciences Institute


Measuring the impact of your research

There are many metrics that can be used to measure the impact of
one’s research. While each metric looks at a different scope of research
impact, there isn’t a single metric that illustrates all the aspects of

In June, Kristi Holmes, PhD, Associate Professor of Preventive Medicine and Director of Galter Health Sciences Library, participated as faculty at the training course on Health Research Impact Assessment at Alberta Innovates Health Solutions. During this course, Holmes provided guidance on measuring research impact through publications.

“It is critical for investigators to be able to understand the impact
of their work and communicate that impact to the broader community,”
Holmes said, “Publication data can be a good place to start this
process, given the importance of communicating our work through the
published literature.”

Choosing metrics

Choosing an appropriate metric is important for accurately portraying
the impact of one’s research. Holmes recommends asking the following
questions to help determine the appropriate metrics:

  • What level assessment will you look at?
  • Publication, individual scholar, a research group, institution, country, etc.
  • What do you want to examine?
  • Productivity, impact, collaboration, subject area, etc.
  • What is your overall purpose?
  • What data do you have available?
  • Who is your audience?
Common Indicators

Generally speaking, there are a few common indicators that can be
used when leveraging publication data. For example, the number of
publications can be used as a measure of knowledge production, citations
can serve to indicate impact and broader visibility of the work, and
co-authorship patterns can be used to demonstrate the level and types of
collaboration.[1] The particular journal where the research is published can even be used as an indirect indicator of quality.

Three commonly discussed metrics include the h index, m index and
Journal Citation Reports impact Factor (IF) Score. Each one is
appropriate in different situations.



Analysis of metric

h index

#papers (h) that have received at least h citations

-Good for more established authors

-Combines publications and citation counts into a single metric

-Difficult to compare between disciplines

-Doesn’t factor in context or source of citations

-Can be applied on the individual or group level

m index

h index/# of years since first paper

-Variant of h index

-Useful for early-career authors, as it accounts for the number of years since the first paper

Journal Impact Factor score

Average number of times articles from the journal published in
the past two years have been cited in the Journal Citation Reports year

-Does not provide metrics about a specific author or paper

-Does not predict scientific research impact

-This score should be used as a comparison tool between journals,
not to compare authors or to measure an individual’s research impact

iCite: a new tool from NIH

A new tool for evaluating bibliometrics was recently created by the NIH, called iCite.
iCite is easy to use: users upload the PubMed IDs of articles of
interest and the tool then displays metrics on these papers such as the
number of articles, articles per year, citations per year, and Relative
Citation Ratio[2] (a field-normalized metric that shows the citation impact of one or more articles relative to the average NIH-funded paper).

Why does measuring research impact through publication data matter? 

Dissemination of research discoveries through the published
literature is an important part of the scholarly process and data about
publications can be a very powerful indicator about research.
Publication data can help investigators build their research careers in
many ways. It can be used to illustrate qualifications to undertake a
new research project, justify a grant renewal, or show accomplishments
for tenure or a promotion. In addition, publication data can also be
used to highlight other aspects of the research or the study team,
including domain impact, willingness to share research findings through
Open Access publishing practices, as well as collaboration with authors
in different areas of research or with those at other institutions.

“Publication data is a powerful tool to understand research impact,
but it is just the tip of the iceberg” Holmes shares. “It is essential
to go beyond the publication to understand and look for opportunities to
communicate meaningful health outcomes to the public, policy makers,
and other key stakeholders.”

For more information, contact Holmes or Galter Library’s Metric and Impact Core.

Posted: July 11, 2016

Measuring the impact of your research | NUCATS: NU Clinical and Translational Sciences Institute

No comments:

Post a Comment