Research Data
Q&A: How to use bibliometrics to evaluate interdisciplinary research
Eleonora Palmaro, a bioengineer at the Italian Institute of Technology (and an Elsevier trainee) shares her insights
By Sophia Katrenko, PhD Posted on 16 December 2015
Sophia Katrenko, PhD, is a data scientist who leads Elsevier’s Data Science team.Venice brought leading professors and researchers together to discuss
“Advanced Research Management Tools” at the Italian Research Management
Workshop. Delegates from universities in Venice, Bologna
and Belfast met with Elsevier colleagues to discuss global rankings,
research assessment, smart data and algorithms, and the use of software
such as Pure.At the conference, Eleonora Palmaro, a biomedical engineer in charge of publications analysis at the Italian Institute of Technology (IIT),
gave insight into IIT’s best practices for research performance
assessment
in interdisciplinary areas. She also shared her experience as an
intern at Elsevier’s Amsterdam headquarters, where she works with our
Data Science team to learn about the data and methodologies we use to
support the research assessment process.
Recently I interviewed her for Elsevier Connect.
Eleonora, what is bibliometrics and how
does it contribute to advancing science?
EleanoraPalmero analyzes the research performance at the Italian Institute of
Technology (IIT) in comparison with universities worldwide using
bibliometric indicators. As an intern with Elsevier’s Data Science team,
she is learning about the data and methodologies we use to support the
research assessment process.
analysis of science through its products, such as publications and
bibliographies. In the last few years, bibliometric indicators have been
officially brought into national assessment exercises (including
the Italian one), researchers to decide where to publish, to assess
the impact of their research, and to find experts in their research
area. It
also provides reliable instruments to direct funding because it helps
to define where it is most profitable to invest. However, social science
and arts also need, for instance, a qualitative parameter like the peer
review of experts, on which the ranking of journals also depends.As we know, research would not exist
without productive and performative research institutions. What kind of data do
we need to assess the productivity of a research institution?
To
assess the productivity of a research institution, we consider not only
scientific publications but the institute’s ability to attract funding
through international projects as well as researchers’ impact on
industry – for example, their patents or start-ups.
You work for an Italian research center
whose scientific activity started in 2006. How is the IIT doing in terms of
research performance? And what are they doing to further improve the impact of their
scientific production?
The institute has obtained
excellent results, confirming that we are going in the right direction.
Due to its international nature, the IIT puts qualitative and
quantitative evaluation at the center of scientific planning and efforts
to attract successful foreign researchers. An external evaluation
committee verifies that scientific goals have been achieved and makes
sure that the institute adopts international research management
standards. This activity goes hand in hand with a yearly internal annual
assessment of employees’ performance, according to an MBO
(Management by Objective) model and with the evaluation of
research program. Furthermore, the institute voluntarily underwent the
2012 national research assessment, emerging as one of the best research
institutes of its size. To further improve evaluation criteria, we are
also developing customized
ad-hoc assessment
practices.
IIT promotes interaction among research
areas. What are the major difficulties in assessing interdisciplinary research
sectors?
One of the major difficulties we face during
the assessment of interdisciplinary products, including our own, is to
identify core research areas. It is particularly challenging for
disciplines like robotics -- a branch of engineering that overlaps with
electronics, computer science, artificial intelligence,
mechatronics, nanotechnology and bioengineering – to set boundaries
and determine which articles belong to it. If a disciplinary area is not
defined by current classification systems, our approach is to look at
the content of scientific products and search for some key concepts that
we have previously
identified with a dictionary.
Despite
very recent progress, Italian research institutions seem unable to retain their
researchers and to attract foreign “brains.” According to the Italian National
Statistics Agency (Istat), only 6 percent
of PhDs in Italy are non-Italian. This share is slightly higher than a decade
ago (2.2 percent). How does the IIT tackle the “brain drain” and favor “brain
circulation”?
Brain drain is a real problem only if it
is not balanced by “brain gain.” Balancing does not always occur,
especially because Italian research infrastructure and salaries are
usually not up to international standards. Besides that, red tape
contributes to discouraging international researchers.
Thanks to its international character, our institute manages to
attract a significant number of international researchers. Currently, 30
percent of the PhDs doing their research at the IIT have foreign
nationality. We cannot add much more to this question, however, because
PhD titles are not released
by the IIT.
What
did you like the most about you experience in the Netherlands? Is there
anything you are taking away from working for Elsevier? What are the skills and
experiences that you will bring back to IIT?
Of the
Netherlands and in particular Amsterdam, I like the international
environment, the capability of different cultures to live peacefully
together, the possibilities to experience in just one city different
kinds of food, music, sports, exhibitions, events. All these experiences
are combined with
a unique scenario characterized by the canals, the different profiles
of the houses, the countless bikes, the windmills.
From the job’s
point of view, I gained more awareness of the importance of bibliometric
analysis as tool to help to make strategic decisions. I understood the
state-of-the-art tools and practices of this discipline and the
challenges we face. I appreciated for the first time the power of data
mining. I will go back with a broader knowledge of the tools, with
more ideas and with the desire to study more to become a data scientist.
I am very grateful for this internship. With the communication and
exchange of ideas and people between companies and research institutes
and universities can be
born tomorrow’s solutions.
Elsevier Connect Contributor
Dr. Sophia KatrenkoisTeam Lead for Data Science at Elsevier, focusing on information
extraction from various data sources, data modeling and analysis. Sophia
has extensive knowledge of predictive analytics, having worked for over
10 years
in the fields of natural language processing and, more recently, risk
modeling.
Prior to joining Elsevier, she had been building default
prediction models for small and medium enterprises at the Capital Tool
Company, G-20 SME Finance Challenge Winner 2010. Sophia had been
affiliated with the University of Utrecht, University of Amsterdam (the
Netherlands), Tuebingen University
(Germany) and Lviv Polytechnic University (Ukraine). She has been
involved in organizing events on machine learning, served on the program
committee of multiple international conferences and published over 30
papers.
Sophia holds a PhD in Computer Science from the University
of Amsterdam on the topic of information extraction, and an MSc in
Computer Science (with distinction) from Lviv Polytechnic National
University, Ukraine.
Q
No comments:
Post a Comment