Source: https://hub.wiley.com/community/exchanges/discover/blog/2018/10/23/what-metrics-matter-the-most-to-you
Conferences
are a great opportunity to connect with researchers across the
spectrum, from Editors to early career researchers . This year, I
brought with me a question for delegates: What Metrics Matter The Most
To You?
I wanted to
know what research metrics were important – what is used to judge the
quality of research, to identify worthwhile articles, and how can
researchers best measure the success of their work?
To
get the conversation started, and to get as much feedback as possible, I
used a survey board with the option to leave a comment or to simply
post an Agree/Disagree sticker. The aim was to allow people to engage as
much as they wanted, and many delegates chose to discuss the (de)merits
of the metrics in more detail. Delegates at the European Consortium for
Political Research (ECPR), the Academic Association for Contemporary
European Studies Annual Conference (UACES), and Remote Sensing &
Photogrammetry Society Annual Conference (RSPSoc) participated–
providing a diverse range of interactions, across countries and
disciplines.
Citations
were overwhelmingly identified as the most important metric, yet many
researchers commented on how unhappy they were with this. Citations are
largely important for institutions as a measure of impact, but some
questioned the extent to which citations truly reflect engagement. Is
this a fair measure by which to judge output? There was also a clear
distinction between citations and Impact Factor, with questions raised
at to how relevant Impact Factor is, especially with tools such as
Google Scholar freely available.
Social
media sharing was the most divisive metric proposed. Some wondered how
to effectively track social media use and questioned the level to which
social media sharing represented real engagement. Each conference drew
out slightly different discussions - at ECPR there was a generational
divide, with younger scholars identifying it as a place to find content.
RSPSoc’s attendees included industry professionals and government
representatives, so it is unsurprising that they felt social media was
the place to find useful research. Attendees at both conferences
suggested that social media sharing might lead to citations and
downloads. UACES delegates gave a mixed response, and although one
participant bemoaned the use of Twitter, it was still identified as a
“21st Century Reality”. This sentiment was echoed by most
delegates, who thought it would play a bigger role in research metrics
in the future.
At
both ECPR and RSPSoc, downloads, while acknowledged as important,
received little attention. This was not the case for the delegates of
the UACES conference, who noted the importance of downloads, and the
potential impact they can have on the subject. At both UACES and RSPSoc
delegates highlighted the move away from articles to downloading other
types of content. As the research community moves further towards an
Open Research landscape, more emphasis is placed on the reproducibility
and accessibility of data sets – tracking the downloads, references, and
replications of data sets was flagged as a big concern for the future.
Some
conference attendees went beyond thinking about these three metrics.
One key observation made at UACES was the emphasis placed on metrics by
different disciplines; in this case, Law researchers observed that
neither citations nor Impact Factor were relevant to them, but it raised
questions as to how Political Scientists might apply their metric
criteria to finding and assessing law research. There were also
conversations as to how useful metrics were; is there too much focus on
metrics and are they being used fairly? Many delegates at UACES and
RSPSoc agreed with these sentiments, but still put their Agree sticker
on the citations section of the survey board.
Almost
everyone noted that in the modern world, one metric no longer ruled
them all. A combination was needed to truly understand the impact of
research - however you choose to define it - and to measure engagement.
No comments:
Post a Comment