Source: http://blogs.lse.ac.uk/impactofsocialsciences/2016/02/02/tracking-the-path-from-research-to-public-policy-on-the-social-web/
The process by which research gets put into action is far from clear cut, argues Stacy Konkiel.
Extracting references to research from policy documents is a
step towards illuminating the murky path. But we should be careful not
to disregard other forms of evidence like online and media mentions as
they are closely interrelated and may even lead to quicker impacts upon
public policy.
Forget the Ivory Tower stereotype you might know: the majority of
researchers want to have an impact upon public policy. That’s according
to one study of public health researchers: 86% of those interviewed
agreed that their most important task “is to produce research that can
further public health policy and/or practice” (Haynes 2011).
This attitude can be seen increasingly in online discussions, too,
where many researchers–particularly in the social sciences–are reflecting critically upon what it means
to be an engaged scholar. This attitude may, in part, be due to the
rise of the “impact agenda” at many universities worldwide, whereby
researchers are increasingly expected to produce research that has “real
world” implications. Results-driven initiatives like the 2014 REF and
similar exercises have made it clear: no longer is being a prolific,
highly cited author enough; you must have an influence on the lives of
the public, as well.
So, how can a researcher learn about their impacts upon public
policy, especially in cases where they may not have had a direct
relationship with policymakers? I’d argue that tracking the impact of
research upon policy can and should be done through analyzing citations
in public policy documents. And, backed by the promising results of an exploratory study
I recently presented at the 2015 ASIS&T SIG/MET workshop, I believe
that researchers who wish to have an impact upon public policy can use
social media tools like Twitter to raise their profile and expedite
their policy impacts.
Before we begin, let’s take a step back: what does it mean for research to have an impact upon public policy?
once argued that “most studies appear to come and go without leaving
any discernible mark on the direction or substance of policy”. But the
times, as they say, are changing. According to one recent study,
approximately one in four policymakers use academic research to inform
their work on a daily or weekly basis (Zardo & Collie, 2014), and the ways in which they put that research into action are varied (more on this below).
That said, the process by which research gets put into action is far from clear cut. As Haynes et al (2011)
point out, “Linear, rational depictions of the ‘the policy cycle’ are
increasingly viewed as idealised normative models that poorly describe a
far messier process.” Indeed, Weiss (1979)
describes several processes by which research is enacted into policy,
the most common of which is practically osmosis. In her article, she
describes the Enlightenment Model of knowledge utilization, by which
social science research influences policymakers in a way that the
policymakers themselves may not even realize. Research might introduce
general ideas or ways of thinking about problems that are influential in
such an indirect way that researchers may not be credited–often because
policymakers don’t even know they’ve been influenced at all!
adaptation of Knott and Wildawsky’s “Stages of Knowledge Utilization”
(seen below), has a lot to do with the difficulty of tracking awareness
of research. Much of the battle of getting research put into policy
(Stages 4-6) lies simply in making policymakers aware that such research
even exists (Stage 1). Such awareness often comes via the mainstream
media (Tieberghien, 2014),
making the media a powerful platform for researcher outreach
strategies. Other effective ways of getting research into practice
include fostering relationships with policymakers and the creation of
translational outputs (lay readings, protocols, training materials, etc)
based upon research findings (Newson et al, 2015; Cohen et al 2015).
Thanks to the social web, it’s now possible to track research’s
journey along each of these stages in a way that couldn’t have been
imagined by Knott and Wildawsky back in 1980. We can use text mining to:
Most importantly, we can extract references to research from public
policy documents; this is perhaps the most concrete way of understanding
impact. There are three major themes for citations to research in
public policy documents, according to Weiss in Haynes et al (2011):
that researchers should use to discover the influence they’ve had upon
“the real world” via public policy. But we should be careful not to
disregard other impact evidence like the altmetrics described above,
because they’re closely interrelated with and can even lead to quicker
impacts upon public policy.
has shown that research outputs cited in policy that receive a high
amount of attention online (as measured by being in the top 0.25% of
outputs ranked by Altmetric score)
tend to get cited more quickly. The mean time to first policy citation
for “high attention” research is 1.7 years, compared with 3.4 years for
other research discussed online.
“High attention” research also receives slightly more policy
citations (1.18 citations per article, compared to 1.09 citations per
article). To be clear, this is an exploratory analysis with small sample
sizes (n=116 for both), but the preliminary data looks promising.
What do these findings mean for the average researcher interested in
making an impact upon public policy? Simply put: making a splash online
and in mainstream media can get the attention of policymakers and
potentially put your important discoveries into practice faster. The
latter point–speed–is especially important for time-sensitive policy
issues (e.g. responding to the international refugee crisis, curbing the
spread of the Zika virus, and so on). Tips on how to promote your work
to the right audiences can be found in the Impactstory e-book, The 30-Day Impact Challenge.
Academia’s still got a long way to go to if we’d like for all
relevant research to get the policy impacts it deserves. Many
researchers aren’t recognized by the powers that be for the impacts that
their work has had in non-scholarly contexts. The incentives to do
outreach simply don’t exist. Luckily, many scholars aren’t waiting
around for the system to change. To explore more about your research’s
impact upon public policy–alongside other evidence of discussions and
reuse on the social web–check out the free Altmetric bookmarklet.
Disclosure: The author is an employee of Altmetric.com, an altmetrics service provider. Altmetric is a Digital Science company.
Note: This article gives the views of the author(s), and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
About the Author:
Stacy Konkiel is Research Metrics Consultant with @altmetric.
Impact of Social Sciences – Policy impact and online attention: Tracking the path from research to public policy on the social web.
Policy impact and online attention: Tracking the path from research to public policy on the social web.
The process by which research gets put into action is far from clear cut, argues Stacy Konkiel.Extracting references to research from policy documents is a
step towards illuminating the murky path. But we should be careful not
to disregard other forms of evidence like online and media mentions as
they are closely interrelated and may even lead to quicker impacts upon
public policy.
Forget the Ivory Tower stereotype you might know: the majority of
researchers want to have an impact upon public policy. That’s according
to one study of public health researchers: 86% of those interviewed
agreed that their most important task “is to produce research that can
further public health policy and/or practice” (Haynes 2011).
This attitude can be seen increasingly in online discussions, too,
where many researchers–particularly in the social sciences–are reflecting critically upon what it means
to be an engaged scholar. This attitude may, in part, be due to the
rise of the “impact agenda” at many universities worldwide, whereby
researchers are increasingly expected to produce research that has “real
world” implications. Results-driven initiatives like the 2014 REF and
similar exercises have made it clear: no longer is being a prolific,
highly cited author enough; you must have an influence on the lives of
the public, as well.
Image credit: Cornelis Brizé (circa 1622–before 1670) [Public domain], via Wikimedia Commons
So, how can a researcher learn about their impacts upon public 
policy, especially in cases where they may not have had a direct
relationship with policymakers? I’d argue that tracking the impact of
research upon policy can and should be done through analyzing citations
in public policy documents. And, backed by the promising results of an exploratory study
I recently presented at the 2015 ASIS&T SIG/MET workshop, I believe
that researchers who wish to have an impact upon public policy can use
social media tools like Twitter to raise their profile and expedite
their policy impacts.
Before we begin, let’s take a step back: what does it mean for research to have an impact upon public policy?
How research influences public policy
In the past, it was difficult to know how and when research influenced policy. Weiss (1979)once argued that “most studies appear to come and go without leaving
any discernible mark on the direction or substance of policy”. But the
times, as they say, are changing. According to one recent study,
approximately one in four policymakers use academic research to inform
their work on a daily or weekly basis (Zardo & Collie, 2014), and the ways in which they put that research into action are varied (more on this below).
That said, the process by which research gets put into action is far from clear cut. As Haynes et al (2011)
point out, “Linear, rational depictions of the ‘the policy cycle’ are
increasingly viewed as idealised normative models that poorly describe a
far messier process.” Indeed, Weiss (1979)
describes several processes by which research is enacted into policy,
the most common of which is practically osmosis. In her article, she
describes the Enlightenment Model of knowledge utilization, by which
social science research influences policymakers in a way that the
policymakers themselves may not even realize. Research might introduce
general ideas or ways of thinking about problems that are influential in
such an indirect way that researchers may not be credited–often because
policymakers don’t even know they’ve been influenced at all!
Tracking research impacts on the social web
The murky path from research to policy, summarized in Landry, Lamari & Amara’s 2003adaptation of Knott and Wildawsky’s “Stages of Knowledge Utilization”
(seen below), has a lot to do with the difficulty of tracking awareness
of research. Much of the battle of getting research put into policy
(Stages 4-6) lies simply in making policymakers aware that such research
even exists (Stage 1). Such awareness often comes via the mainstream
media (Tieberghien, 2014),
making the media a powerful platform for researcher outreach
strategies. Other effective ways of getting research into practice
include fostering relationships with policymakers and the creation of
translational outputs (lay readings, protocols, training materials, etc)
based upon research findings (Newson et al, 2015; Cohen et al 2015).
Thanks to the social web, it’s now possible to track research’s
journey along each of these stages in a way that couldn’t have been
imagined by Knott and Wildawsky back in 1980. We can use text mining to:
- discover when research is discussed in news articles;
- see the traces of researchers’ engagement with policymakers in their conversations posted publicly on listservs and Twitter; and
- track downloads and pageviews for non-traditional research outputs
like protocols, white papers, and the like, to understand how widely
read translational outputs are.
Most importantly, we can extract references to research from public
policy documents; this is perhaps the most concrete way of understanding
impact. There are three major themes for citations to research in
public policy documents, according to Weiss in Haynes et al (2011):
- data used instrumentally to inform policy decisions;
- argument used strategically to address values and interests or symbolically to support, justify or refute predetermined positions; or
- ideas that influence the policy climate and agenda-setting by illuminating, supporting and challenging existing paradigms.
that researchers should use to discover the influence they’ve had upon
“the real world” via public policy. But we should be careful not to
disregard other impact evidence like the altmetrics described above,
because they’re closely interrelated with and can even lead to quicker
impacts upon public policy.
“High attention” research gets cited more quickly and often in policy
Initial researchhas shown that research outputs cited in policy that receive a high
amount of attention online (as measured by being in the top 0.25% of
outputs ranked by Altmetric score)
tend to get cited more quickly. The mean time to first policy citation
for “high attention” research is 1.7 years, compared with 3.4 years for
other research discussed online.
“High attention” research also receives slightly more policy
citations (1.18 citations per article, compared to 1.09 citations per
article). To be clear, this is an exploratory analysis with small sample
sizes (n=116 for both), but the preliminary data looks promising.
What do these findings mean for the average researcher interested in
making an impact upon public policy? Simply put: making a splash online
and in mainstream media can get the attention of policymakers and
potentially put your important discoveries into practice faster. The
latter point–speed–is especially important for time-sensitive policy
issues (e.g. responding to the international refugee crisis, curbing the
spread of the Zika virus, and so on). Tips on how to promote your work
to the right audiences can be found in the Impactstory e-book, The 30-Day Impact Challenge.
Academia’s still got a long way to go to if we’d like for all
relevant research to get the policy impacts it deserves. Many
researchers aren’t recognized by the powers that be for the impacts that
their work has had in non-scholarly contexts. The incentives to do
outreach simply don’t exist. Luckily, many scholars aren’t waiting
around for the system to change. To explore more about your research’s
impact upon public policy–alongside other evidence of discussions and
reuse on the social web–check out the free Altmetric bookmarklet.
Disclosure: The author is an employee of Altmetric.com, an altmetrics service provider. Altmetric is a Digital Science company.
Note: This article gives the views of the author(s), and not the position of the LSE Impact blog, nor of the London School of Economics. Please review our Comments Policy if you have any concerns on posting a comment below.
About the Author:
Stacy Konkiel is Research Metrics Consultant with @altmetric.
Impact of Social Sciences – Policy impact and online attention: Tracking the path from research to public policy on the social web.
No comments:
Post a Comment