Source: https://libguides.cam.ac.uk/wolfsoncollege/impact
Wolfson College Academic Skills: Measuring the impact of your research
What are metrics?
Research impact can be measured in many ways. Qualitative methods vary; the most important one is various forms of peer-review. Quantitatively you can use publication counts; amount of research income; number of PhD students; size of research group; number of PI projects; views and downloads of online outputs; number of patents and licenses obtained; and metrics - ways of measuring patterns of authorship, publication, and the use of literature.
These are used by researchers to decide what to read and where to publish but also to measure how their own work may have been cited. Universities might use them to judge the quality of research and to decide on strategic priorities. They tend to be more heavily used by researchers in STEM subjects, because of the citation culture and that the key data suppliers are journal article-rich databases.
There are a bewildering array of metrics , all of which have their limitations. These include:
- Research output metrics - to find out how often an output, or a group of outputs, has been cited by others.
- Author metrics - to explore the impact of an author, or a group of authors, based on the citation rates of their outputs.
- Journal metrics - to assess the impact of a journal and to compare journals in the same field.
- Altmetrics - measures based on social media data rather than traditional publication citation counts.
-
Research Impact Metrics - Elsevier
PDF guide to the different types of metrics used to demonstrate the impact of research publications. -
Mastering Metrics - Taylor Francis
PDF infographic explaining journal and article level metrics
Research in 3 minutes: metrics
Watch this video for a quick overview of terms and concepts.
Bibliometrics
This is statistical analysis of written publications. You can analyse journal titles using the well-known Impact Factor or alternatives such as Eigenfactor or SCImago. Alternatively you may wish to analyse an individual paper or researcher (including yourself).
Problems with bibliometrics
Bibliometrics remain highly controversial as proxy indicators of the impact or quality of published researchers. There are many caveats to be aware of:
- citation cultures vary enormously between disciplines; what is low ranking in one subject, may be considered high ranking in another.
- seminal papers can skew results
- review papers may be cited more than original research
- self-citations
- poor quality research may be highly cited for the wrong reasons (see the, now retracted, Andrew Wakefield et al article making a link between the MMR vaccine and colitis and autism spectrum disorders).
This video demonstrates the importance of noting the possible pitfalls when using bibliometrics.
Altmetrics
Alternative metrics track the online attention that research receives. These are a more immediate reflection of impact of work than more traditional metrics, which take many years to generate any meaningful data.. When judging the impact of research online it is helpful to consider:
- Context - Raw data or metrics can be interesting, but they are less useful or meaningful without the story behind them.
- Active engagement - Lightweight engagement (e.g. likes, shares, follows) might be a valuable indicator, but it might be difficult to assert what it actually proves.
- Evidence of engagement or impact - This means having access to screen captures or urls, rather than descriptions.
- Currency - it is easy for this sort of data to be deleted or buried, so record it as it happens, don't try to collect it retrospectively
They are intended to compliment traditional metrics, not replace them.
Tools
You can use in-built feature like Twitter Analytics or the statistics on your blog to see the impact you are making. Sites such as Academia.edu list the number of times a paper has been viewed. Alternatively, there are tools which aggregate impact such as:
You can also set up automated tools to track your impact on social media, such as Impact Story. For a list, along with their pros and cons, see the handout of a session delivered by Jenni Lecky-Thompson (Philosophy), Matthias Ammon (MML), Katie Hughes (OSC), Helen Murphy (English) in November 2017. Please note that you can no longer register with Storify.
Problems with altmetrics
Just as bibliometrics have their flaws, so do altmetrics. Mentions on Twitter and in the news do not necessarily correspond to research gaining a high profile. It may because the research was flawed, has been disproved, contained a funny word, or was involved in some tangential way to a news-worthy story. For example, when Stephen Hawking's PhD thesis was recently made Open Access, the scale of traffic to the server caused major problems on the internet. It was this, rather than the research itself, that was reported in the press and elsewhere online. However, it isn't possible to tell this from the raw data:
Managing your citations
Sign up for an ORCID Identifier: The Open Researcher Community ID is an increasingly recognized persistent digital identifier. See the tab on Publishing your Research for more information
Get a ResearcherID with Web of Science: A ResearcherID can be linked to your ORCID number and facilitates citation metrics and publication tracking using Web of Science tools. With a ResearcherID, you will be included in the Web of Knowledge author index allowing other researchers to learn more about your work and affiliations. Sign up here.
Create a Google Scholar Citations Profile: Google Scholar citations allows authors to track citations to their scholarly works and to calculate numerous citation metrics based on Google Scholar citation data. By setting up a profile, you will be able to disambiguate yourself from authors with the same or similar names.
Increasing your citations
If you are interested in increasing the number of citations your work receives, you may be interested in the 33 ways mentioned in the paper Effective Strategies for Increasing Citation Frequency.
One way is to make your research more visible by making it Open Access. See the tab on Publishing your Research for more information.
Many publishers also offer ways of increasing the attention your articles receive, such as the Kudos service from Wiley.
Using social media is a great way to increase the coverage your work receives. If you aren't sure what you can share with followers and peers, check out How can I Share It?
Further resources
Metrics Toolkit - a resource for researchers and evaluators that provides guidance for demonstrating and evaluating claims of research impact. With the Toolkit you can quickly understand what a metric means, how it is calculated, and if it’s good match for your impact question.
Snowball Metrics - a bottom-up initiative owned by research-intensive universities (including Cambridge) around the globe, to ensure that its outputs are of practical use to them, and are not imposed by funders, agencies, or suppliers of research information.
Collecting Research Impact Evidence - a guide for best practice.
No comments:
Post a Comment