EU to ‘improve and maximise’ impact of research
Critics fear focus on research impact in next funding programme could lead to a REF-type exercise
November 10, 2016
The
next framework programme for European research funding should have “a
more sophisticated approach” to impact, according to Carlos Moedas, the
European Commissioner for research, science and innovation.
Speaking at a European Research Area conference in Berlin last month,
he said that the commission had an obligation to understand and
communicate the impact of its work to finance ministers and the general
public.
It is not yet clear how he hopes to develop the impact agenda
in Europe. But experts are concerned about the direction it could push
science and question whether the actions of the European Research
Council, which funds blue-skies research solely on the basis of
excellence, will be protected.
Mark Reed, professor of social innovation at Newcastle University,
fears the commission may be considering a post hoc assessment exercise
to evaluate the impact of research grants that have previously been
awarded.
That could raise the prospect of such a project being similar to the
Higher Education Funding Council for England’s controversial research
excellence framework, which uses case studies of the impact generated by
research to help influence how much funding universities secure in the
future.
“To do that at the scale of the EU would be fairly monumental,” Professor Reed told Times Higher Education.
Academics currently applying for European research funding do have to
describe the impact that they expect their work to have. Impact is one
of three criteria, alongside excellence, and quality and efficiency,
used to decide which proposals secure funding.
But Professor Reed said that simply beefing up this part of European
grant applications would not be sufficient to achieve what Mr Moedas
appears to want. “He may have set himself a task that is rather
challenging to achieve. If he is serious about this, as he sounds, then
he will need to think about looking back in time,” he added.
He added that there was a “need to change” the pre-award assessment
of impact at the European level because often EU-funded researchers have
a “huge focus on communicating their outcomes” with the idea that
disseminating their findings will lead to impact.
This results in “very costly events for policymakers…to which very
few people turn up”, he said. Changing this culture so that researchers
start to think about impact in a structured and planned way ahead of
starting their project could be “low-hanging fruit” for Mr Moedas and
would be “very easy” to introduce, he said.
Professor Reed was not the only academic to have concerns about where Mr Moedas’ comments would lead European research.
Peter Strohschneider, president of the German Research Foundation,
said that the speech “implies a rather weak concept of what science and
research is about. It takes research as a means of economic growth and
innovation,” he said.
Introducing a REF-type exercise in Europe would be “very expensive”
and lead to “scientific and intellectual shortcomings”, he said.
“Questions of metrics are questions of power…This is all about political
decisions on what we take as relevant in research. This is not
illegitimate but it is not the wisest way to steer the entire EU
research system,” he added.
Katrien Maes, chief policy officer at the European League of Research
Universities, said that how the commission decides to measure impact
will depend on the structure of the next framework programme.
The commission has done much work to simplify the complicated process
of applying for European research funding, but there is still work to
be done in this area, she said. “This potentially complicates things
again enormously,” she added.
But she urged the commission not to define impact as “a narrow
economic act. Impact can take a very long time to materialise and it is
not just a question of patents, spin-offs and licences”, she said.
Lidia Borrell-Damian, director of research and innovation at the
European Universities Association, said that the ERC should be protected
against the introduction of any measures of impact.
“The ERC is a [red] line. If they do introduce impact there, then I
think they risk the wrath of the sector,” she said, adding that the EUA
and others have sent the commission “very strong signals” about this.
In 2012, Helga Nowotny, then president of the ERC, said that adopting an impact agenda would destroy the organisation.
Dr Borrell-Damian added that the European Innovation Council, a new
proposition to support the commercialisation of technologies, could
create the impact that Mr Moedas is looking for.
Whether he is hoping to improve research impact through the EIC, or
pre- or post-award assessment, remains unclear, but there is other
evidence that he is clearly looking at options. Earlier this year, the
commission put out a call for experts to sit on a high-level group to
advise them on maximising the impact of European research and
innovation.
In a statement for THE, the commission spokeswoman for
research, science and innovation said: “Improving the impacts of
EU-funded research and innovation is a very high priority.”
She added that “clear progress” had been made on impact during
Horizon 2020, the current framework programme. “[W]e will seek to
further improve and maximise the impact of future EU funding for
research and innovation, knowing that not all research will have a
concrete and immediate impact,” she said.
policymakers worldwide grapple with the need to prove the economic value
of research.
The idea of measuring the impact of research in a serious and
systematic way originated in Australia, with the aborted research
quality framework. In the mid-2000s, this looked at the use of case
studies to describe the impact of completed research – a system
eventually adopted in the UK’s 2014 research excellence framework.
Impact is again gaining ground in Australia. In 2012, the Australian
Research Council established a working group with other government
research agencies to look at the possible ways of measuring engagement
and impact for their next research assessment exercise.
Attila Brungs, vice-chancellor and president of the University of Technology Sydney, said: “Having an acknowledgement of impact is how we change the world and make it a better place.”
He added that Carlos Moedas, the European Commissioner for research,
science and innovation, should “stay away” from using a “few simple
quantitative measures around impact” as these tend to measure engagement
not impact. Using the wrong metrics can also “be very distorting and
drive the wrong behaviour” among researchers, he added.
He favours the case study approach to measuring impact but added that
Mr Moedas should not rush into impact assessment before learning the
lessons that other research systems have.
Kathrin Möslein, chair for information systems innovation and value creation at the University of Erlangen-Nuremberg, said that Mr Moedas should set up a comparative project that looks at the most promising practices in impact assessment.
In Germany, researchers are usually asked to produce impact
strategies in the early phase of grant proposals, and previous
achievements of impact are usually assessed alongside the project
proposal. At the end of a project, the evidence of
impact is again considered. “[Impact] is usually not an issue discussed
separately from research, but an integral part of research practice,”
she added.
In the US, a project is under way that looks at the economic impact of research in a very different way.
The UMETRICS project at the Institute for Research on Innovation and Science at the University of Michigan
uses national statistics – such as the number of businesses started and
job creation – to look at the impact individual researchers have
compared with those not working on research projects.
Julia Lane, professor in the Robert F. Wagner Graduate School of Public Service at New York University,
thinks that Mr Moedas should take heed of the US system, adding that
the way research impact was measured in the UK was “a bureaucratic
nightmare” that was “a waste of time and energy and has been pretty
unconvincing”, she said.
“It is not serious, it is not scientific. So for the scientific
community to have such an unscientific approach is an embarrassment,”
she added.
In other parts of the world, the impact agenda is less developed.
In China and Hong Kong, measures of impact are not required for
research funding. Walter Ho, director of the Office of Research and
Knowledge Transfer services at the Chinese University of Hong Kong,
said: “At the moment, more focus is being directed to technology
transfer rather than impact. Some metrics such as [the] number of
patents granted, companies formed and licensing income are evaluated.”
But he is anticipating that this will change in the years ahead. “The
Research Grants Council in Hong Kong will probably require universities
in Hong Kong to report impact cases in the next round of assessment
exercise in 2018,” he added.
holly.else@tesglobal.com
next framework programme for European research funding should have “a
more sophisticated approach” to impact, according to Carlos Moedas, the
European Commissioner for research, science and innovation.
Speaking at a European Research Area conference in Berlin last month,
he said that the commission had an obligation to understand and
communicate the impact of its work to finance ministers and the general
public.
It is not yet clear how he hopes to develop the impact agenda
in Europe. But experts are concerned about the direction it could push
science and question whether the actions of the European Research
Council, which funds blue-skies research solely on the basis of
excellence, will be protected.
Mark Reed, professor of social innovation at Newcastle University,
fears the commission may be considering a post hoc assessment exercise
to evaluate the impact of research grants that have previously been
awarded.
That could raise the prospect of such a project being similar to the
Higher Education Funding Council for England’s controversial research
excellence framework, which uses case studies of the impact generated by
research to help influence how much funding universities secure in the
future.
“To do that at the scale of the EU would be fairly monumental,” Professor Reed told Times Higher Education.
Academics currently applying for European research funding do have to
describe the impact that they expect their work to have. Impact is one
of three criteria, alongside excellence, and quality and efficiency,
used to decide which proposals secure funding.
But Professor Reed said that simply beefing up this part of European
grant applications would not be sufficient to achieve what Mr Moedas
appears to want. “He may have set himself a task that is rather
challenging to achieve. If he is serious about this, as he sounds, then
he will need to think about looking back in time,” he added.
He added that there was a “need to change” the pre-award assessment
of impact at the European level because often EU-funded researchers have
a “huge focus on communicating their outcomes” with the idea that
disseminating their findings will lead to impact.
This results in “very costly events for policymakers…to which very
few people turn up”, he said. Changing this culture so that researchers
start to think about impact in a structured and planned way ahead of
starting their project could be “low-hanging fruit” for Mr Moedas and
would be “very easy” to introduce, he said.
Professor Reed was not the only academic to have concerns about where Mr Moedas’ comments would lead European research.
Peter Strohschneider, president of the German Research Foundation,
said that the speech “implies a rather weak concept of what science and
research is about. It takes research as a means of economic growth and
innovation,” he said.
Introducing a REF-type exercise in Europe would be “very expensive”
and lead to “scientific and intellectual shortcomings”, he said.
“Questions of metrics are questions of power…This is all about political
decisions on what we take as relevant in research. This is not
illegitimate but it is not the wisest way to steer the entire EU
research system,” he added.
Katrien Maes, chief policy officer at the European League of Research
Universities, said that how the commission decides to measure impact
will depend on the structure of the next framework programme.
The commission has done much work to simplify the complicated process
of applying for European research funding, but there is still work to
be done in this area, she said. “This potentially complicates things
again enormously,” she added.
But she urged the commission not to define impact as “a narrow
economic act. Impact can take a very long time to materialise and it is
not just a question of patents, spin-offs and licences”, she said.
Lidia Borrell-Damian, director of research and innovation at the
European Universities Association, said that the ERC should be protected
against the introduction of any measures of impact.
“The ERC is a [red] line. If they do introduce impact there, then I
think they risk the wrath of the sector,” she said, adding that the EUA
and others have sent the commission “very strong signals” about this.
In 2012, Helga Nowotny, then president of the ERC, said that adopting an impact agenda would destroy the organisation.
Dr Borrell-Damian added that the European Innovation Council, a new
proposition to support the commercialisation of technologies, could
create the impact that Mr Moedas is looking for.
Whether he is hoping to improve research impact through the EIC, or
pre- or post-award assessment, remains unclear, but there is other
evidence that he is clearly looking at options. Earlier this year, the
commission put out a call for experts to sit on a high-level group to
advise them on maximising the impact of European research and
innovation.
In a statement for THE, the commission spokeswoman for
research, science and innovation said: “Improving the impacts of
EU-funded research and innovation is a very high priority.”
She added that “clear progress” had been made on impact during
Horizon 2020, the current framework programme. “[W]e will seek to
further improve and maximise the impact of future EU funding for
research and innovation, knowing that not all research will have a
concrete and immediate impact,” she said.
Lessons on impact
The impact agenda has been gaining momentum in recent years aspolicymakers worldwide grapple with the need to prove the economic value
of research.
The idea of measuring the impact of research in a serious and
systematic way originated in Australia, with the aborted research
quality framework. In the mid-2000s, this looked at the use of case
studies to describe the impact of completed research – a system
eventually adopted in the UK’s 2014 research excellence framework.
Impact is again gaining ground in Australia. In 2012, the Australian
Research Council established a working group with other government
research agencies to look at the possible ways of measuring engagement
and impact for their next research assessment exercise.
Attila Brungs, vice-chancellor and president of the University of Technology Sydney, said: “Having an acknowledgement of impact is how we change the world and make it a better place.”
He added that Carlos Moedas, the European Commissioner for research,
science and innovation, should “stay away” from using a “few simple
quantitative measures around impact” as these tend to measure engagement
not impact. Using the wrong metrics can also “be very distorting and
drive the wrong behaviour” among researchers, he added.
He favours the case study approach to measuring impact but added that
Mr Moedas should not rush into impact assessment before learning the
lessons that other research systems have.
Kathrin Möslein, chair for information systems innovation and value creation at the University of Erlangen-Nuremberg, said that Mr Moedas should set up a comparative project that looks at the most promising practices in impact assessment.
In Germany, researchers are usually asked to produce impact
strategies in the early phase of grant proposals, and previous
achievements of impact are usually assessed alongside the project
proposal. At the end of a project, the evidence of
impact is again considered. “[Impact] is usually not an issue discussed
separately from research, but an integral part of research practice,”
she added.
In the US, a project is under way that looks at the economic impact of research in a very different way.
The UMETRICS project at the Institute for Research on Innovation and Science at the University of Michigan
uses national statistics – such as the number of businesses started and
job creation – to look at the impact individual researchers have
compared with those not working on research projects.
Julia Lane, professor in the Robert F. Wagner Graduate School of Public Service at New York University,
thinks that Mr Moedas should take heed of the US system, adding that
the way research impact was measured in the UK was “a bureaucratic
nightmare” that was “a waste of time and energy and has been pretty
unconvincing”, she said.
“It is not serious, it is not scientific. So for the scientific
community to have such an unscientific approach is an embarrassment,”
she added.
In other parts of the world, the impact agenda is less developed.
In China and Hong Kong, measures of impact are not required for
research funding. Walter Ho, director of the Office of Research and
Knowledge Transfer services at the Chinese University of Hong Kong,
said: “At the moment, more focus is being directed to technology
transfer rather than impact. Some metrics such as [the] number of
patents granted, companies formed and licensing income are evaluated.”
But he is anticipating that this will change in the years ahead. “The
Research Grants Council in Hong Kong will probably require universities
in Hong Kong to report impact cases in the next round of assessment
exercise in 2018,” he added.
holly.else@tesglobal.com
EU to ‘improve and maximise’ impact of research | THE News
No comments:
Post a Comment