Friday 7 October 2022

Measuring Research Quality and Impact - Research Guide

 Source: https://libguides.murdoch.edu.au/measure_research/researcher

Measuring Research Quality and Impact - Research Guide

Researcher Impact

Demonstrating Impact

Researchers may be asked to demonstrate the quantity, quality and impact of their research publications for a variety of purposes, including:

  • to enhance researcher profiles and promote their research to potential collaborators and investors
  • to benchmark their productivity for performance reviews
  • to support promotion or tenure applications
  • to support grant or other funding applications and progress reports

The output of an individual researcher can be measured using citation metrics, other alternative metrics, measures of esteem, and indices such as an h-index or field weighted citation impact (FWCI).

Calculating Your Field Weighted Citation Impact or h-index

FWCI 

The field-weighted citation impact (FWCI) is an author metric which compares the total citations actually received by a researcher's publications to the average number of citations received by all other similar publications from the same research field. The global mean of the FWCI is 1.0, so an FWCI of 1.50 means 50% more cited than the world average; whereas, an FWCI of .75 means 25% less cited than the world average.

The following tool can be used to calculate your FWCI:

SciVal

SciVal allows you to compare the research performance of individual researchers and research groups to Australian and global benchmarks, and to identify and evaluate collaboration partners. Benchmarking data for individual researchers includes FWCI and h-index.

h-index  

The h-index is an author metric, first proposed by J. E. Hirsch in 2005, that attempts to measure both the productivity and citation impact of the publications of an author. 

The h-index is a measure of an individual's impact on the research community based upon the number of papers published and the number of citations these papers have received.

The method of calculating a h-index is the number of articles in a database that have received the same number or more citations over time. As an example, a researcher with an h-index of 20 has (of their total number of publications) 20 papers which have been cited at least 20 times each:

Source: Wikipedia

An h-index is not skewed by a single highly cited paper nor by a large number of poorly cited papers.

As well as SciVal, the following tools can also be used to calculate your h-index:

InCites

InCites is a customised, citation-based research evaluation tool on the Web that enables you to analyse the institutional productivity and benchmark your output against peers worldwide. Incites also provides the Category Normalized Citation Impact (CNCI) of a document which is calculated by dividing the actual citation count by an expected rate for documents with the same document type, year of publication, and subject area.

Scopus - Scopus is an online citation database that indexes resources in the health and life sciences, chemistry, physics, mathematics, engineering, earth and environmental sciences, social sciences, psychology, and economics, business and management. Scopus allows you to measure your h-index using the Scopus citation data and to create a h-graph.

Web of Science Core Collection - Web of Science Core Collection is an online citation database that covers the sciences, social sciences, arts and humanities. It allows you to measure your h-index using the Web of Science data and to create a h-graph.

Note: As these databases index different sets of journals over different time periods, your h-index may vary significantly between the two tools. You should check all tools and carefully select the measurement that best represents your output. When a h-index is included in documents, you should state which tool you used and the date the data was obtained. A more accurate h-index may be obtained by collating data from both of these tools, and other citations databases, using an ORCID; however if you wish to create a true h-index based on all unique citations to your publications from all sources, you will need to calculate it manually.  The fewer papers you have the more significant each citation becomes in terms of calculating your h-index.

Google Scholar - The My Citations feature requires a Google login and password. The Publish or Perish software analyses academic citations based on information from a variety of data sources, including Google Scholar and Microsoft Academic Search, to obtain the raw citations, and presents a range of metrics including the h-index.

Note: Google Scholar citations have been criticised for duplications and misidentifications. Careful analysis and verification of results is recommended.

Other Metrics and Measures of Esteem

Alternative metrics and measures of esteems may also indicate the quality and impact of a researcher's publications.

Measures of esteem are additional factors which may provide evidence of research quality. These measures may include:

  • awards and prizes
  • rankings in prestigious lists
  • membership of professional or academic organisations
  • positions on national and international committees or strategic advisory bodies
  • influence on government or public policy
  • industrial advisory roles
  • research fellowships
  • patents or other commercial output
  • international collaborations
  • successfully completed research grants or projects
  • editorial or reviewer role for a significant journal or reference work
  • keynote and plenary addresses at conferences, conference organisation or presentations at significant international conferences

Researcher Profiles and Identifiers

Research profiles allow you to:

  • Manage your publications list
  • Avoid misidentification
  • Track citation counts
  • Attribute your research output to Murdoch University
  • Promote your research locally, nationally and globally 
  • Be identified by potential collaborators
  • Enhance your Murdoch staff profile page by including a link to other profiles

There are a number of researcher profile services available and you may find that more than one profile service is required to include all of your research. Each service has different advantages.

Free researcher profiles are available from the following services:

ORCiD (Open Researcher and Contributor iD)
ORCiD is a persistent digital identifier which distinguishes you from other researchers. It provides two basic services:

  1. a registry to obtain a unique identifier and manage a record of activities, and 
  2. interfaces that support system-to-system communication and authentication.

Registering for an ORCiD profile streamlines manuscript submission and grant applications. It enables you to attach your identity to research objects such as articles, citations, datasets, patents, experiments and equipment. Some journals now require you to submit your ORCiD as part of the process when submitting articles for publication.

Find out more information about the role of ORCiD in the Murdoch University research process.

Author Identifier - Scopus (Elsevier)
The Scopus Author Identifier, by Elsevier, assigns a unique number to each author in the Scopus database and groups together all their publications. It accounts for variant versions of names by matching affiliations, addresses, subject areas, co-authors and dates of publication. It automatically calculates citation counts and the h-index.

ResearcherID - Web of Science (Clarivate Analytics)
ResearcherID, by Clarivate Analytics, assigns a specific ID number to each registered researcher in order to reduce identity issues. Researchers can add the details of their publications to their individual profile and these are made available to anyone searching the database. Citation metrics such as number of times cited and h-index are automatically calculated using details from Web of Science.

Citations - Google Scholar
Create a profile in Google Scholar to ensure correct attribution of your publications and citations. Profiles can be private or public. All citations should be checked for accuracy.

Resources

MyRI (Measuring your Research Impact) online tutorial - Module 2 : Track your Research Impact - a cross institutional project from University College Dublin, Dublin City University, Dublin Institute of Technology and Maynooth University.

Research Metrics Guidebook (SciVal) - Elsevier

Scopus Quick Reference Guide  - Elsevier

Web of Science Quick Reference Guide  - Clarivate Analytics

No comments:

Post a Comment