Twitter can help with scientific dissemination but its influence on citation impact is less clear
Researchershave long been encouraged to use Twitter. But does researchers’
presence on Twitter influence citations to their papers? José Luis Ortega
explored to what extent the participation of scholars on Twitter can
influence the tweeting of their articles and found that although the
relationship between tweets and citations is poor, actively
participating on Twitter is a powerful way of promoting and
disseminating academic outputs, potentially indirectly influencing the
scholarly impact and improving prospects of increased citations.
Many altmetric studies – meaning
altmetrics in general rather than Altmetric.com specifically – have
wrongly taken the conceptual framework of bibliometrics as a model to
understand the meaning of alternative metrics. For example, it has been
assumed that the mention of a research paper on Twitter is comparable to
a bibliographic citation, taking for granted that a tweet can be an
appreciation index and Twitter a kind of citation index (Eysenbach, 2011; Shuai et al., 2012).
However, the processes that generate a citation are very different to a
tweet. To have an article cited one must first conduct a study, write a
paper and publish it in an indexed journal; whereas the mention of a
paper on Twitter requires one only to write a short message. This
difference could cause the greater intervention of authors in the
mentions of their own papers, at least relative to the case of citations
(i.e. instances of self-citation).
altmetrics in general rather than Altmetric.com specifically – have
wrongly taken the conceptual framework of bibliometrics as a model to
understand the meaning of alternative metrics. For example, it has been
assumed that the mention of a research paper on Twitter is comparable to
a bibliographic citation, taking for granted that a tweet can be an
appreciation index and Twitter a kind of citation index (Eysenbach, 2011; Shuai et al., 2012).
However, the processes that generate a citation are very different to a
tweet. To have an article cited one must first conduct a study, write a
paper and publish it in an indexed journal; whereas the mention of a
paper on Twitter requires one only to write a short message. This
difference could cause the greater intervention of authors in the
mentions of their own papers, at least relative to the case of citations
(i.e. instances of self-citation).
Starting from that hypothesis, my recent research explores to what extent the participation of scholars on Twitter can influence the tweeting of their articles,
and, by extension, the likelihood of those articles being cited. To
that end, 4,166 articles from 76 Twitter users and 124 non-Twitter users
were analysed. These data were crawled using PlumX Analytics, which
counts the number of tweets a document receives, whereas Scopus was used
to extract citation numbers. Finally, a manual search was done to
distinguish authors with a handle on Twitter from others not registered
to that social network.
and, by extension, the likelihood of those articles being cited. To
that end, 4,166 articles from 76 Twitter users and 124 non-Twitter users
were analysed. These data were crawled using PlumX Analytics, which
counts the number of tweets a document receives, whereas Scopus was used
to extract citation numbers. Finally, a manual search was done to
distinguish authors with a handle on Twitter from others not registered
to that social network.
Image credit: twitter-sparrow 092 by Josey. This work is licensed under a CC BY 2.0 license.
Results showed that papers from Twitter
users could be on average 33% more tweeted than documents of non-Twitter
users (Twitter users mean = 2.33 tweets per paper; non-Twitter users
mean = 1.75). Obviously, this result is important not because
participating on Twitter improves the mention of academic papers, but
because the number of mentions could be greatly influenced by the
authors of those papers. This introduces the possibility of manipulation
of tweets metrics and puts in doubt the suitability of Twitter as
source for research evaluation.
users could be on average 33% more tweeted than documents of non-Twitter
users (Twitter users mean = 2.33 tweets per paper; non-Twitter users
mean = 1.75). Obviously, this result is important not because
participating on Twitter improves the mention of academic papers, but
because the number of mentions could be greatly influenced by the
authors of those papers. This introduces the possibility of manipulation
of tweets metrics and puts in doubt the suitability of Twitter as
source for research evaluation.
Logically, merely having a Twitter account
is not enough. So, specifically, what author activity on Twitter most
influences the mention of their papers? To answer that question, number
of tweets, followers and followings of 76 Twitter users were extracted
and time-normalized. Next, a linear regression analysis was performed to
detect the variable that most influences the mention of articles.
Results showed that the number of followers explains 34% (R2=.341)
of the tweets received by their publications, claiming that a 1%
increase in followers would generate 0.31% of new tweets. This result
demonstrates that the number of followers is an important factor for the
spreading of messages on Twitter. However, the effect of this variable
is small because an author needs three times more followers to gain only
one mention more.
is not enough. So, specifically, what author activity on Twitter most
influences the mention of their papers? To answer that question, number
of tweets, followers and followings of 76 Twitter users were extracted
and time-normalized. Next, a linear regression analysis was performed to
detect the variable that most influences the mention of articles.
Results showed that the number of followers explains 34% (R2=.341)
of the tweets received by their publications, claiming that a 1%
increase in followers would generate 0.31% of new tweets. This result
demonstrates that the number of followers is an important factor for the
spreading of messages on Twitter. However, the effect of this variable
is small because an author needs three times more followers to gain only
one mention more.
Nevertheless, the most interesting
question in this study is to clarify the relationship between tweets and
citations. Does authors’ presence on Twitter have some influence on the
next citations to their papers? Results show that there is no
statistical difference (p value = 0.144), and therefore to be
or not to be on Twitter does not affect the number of citations
(non-Twitter users mean = 1.77; Twitter users mean = 2.00). This once
more evidences the poor relationship between tweets and citations and
puts in doubt suggestions that mentions in social networks can be
considered an early proxy of research impact. However, when the
regression was applied to detect if some activity parameters on Twitter
are related to citations, a weak yet significant relationship was found
between followers and citations.
question in this study is to clarify the relationship between tweets and
citations. Does authors’ presence on Twitter have some influence on the
next citations to their papers? Results show that there is no
statistical difference (p value = 0.144), and therefore to be
or not to be on Twitter does not affect the number of citations
(non-Twitter users mean = 1.77; Twitter users mean = 2.00). This once
more evidences the poor relationship between tweets and citations and
puts in doubt suggestions that mentions in social networks can be
considered an early proxy of research impact. However, when the
regression was applied to detect if some activity parameters on Twitter
are related to citations, a weak yet significant relationship was found
between followers and citations.
Thus, followers explain only 17% (R2=.171)
of the number of citations, meaning a 1% increase of followers could
produce 0.24% of citations. Although this result could suggest that
participation on Twitter has some influence on the citation of articles,
it should be interpreted in a different way, introducing the concept of
dissemination. As we have seen, Twitter followers act as information
speakers by retweeting articles. Authors with a large number of
followers can reach a much wider audience, increasing the likelihood
that their papers are cited in future. This new interpretation
highlights the importance of dissemination in the citation of articles
and suggests that part of research impact could be explained by the
intensity in which a paper is spread. Likewise, the more media used to
promote an academic result, the greater the likelihood of it being read
and later cited.
of the number of citations, meaning a 1% increase of followers could
produce 0.24% of citations. Although this result could suggest that
participation on Twitter has some influence on the citation of articles,
it should be interpreted in a different way, introducing the concept of
dissemination. As we have seen, Twitter followers act as information
speakers by retweeting articles. Authors with a large number of
followers can reach a much wider audience, increasing the likelihood
that their papers are cited in future. This new interpretation
highlights the importance of dissemination in the citation of articles
and suggests that part of research impact could be explained by the
intensity in which a paper is spread. Likewise, the more media used to
promote an academic result, the greater the likelihood of it being read
and later cited.
In conclusion, actively participating on
Twitter is a powerful way of promoting and diffusing our academic
outputs. This allows us to maintain a wide network of followers that
amplify our message and reach a larger audience. Indirectly, such broad
dissemination could influence the scholarly impact, slightly improving
the prospect of increased citations. Therefore, Twitter cannot be viewed
as a citation index, but as an information-spreading network; and the
tweets of articles should not be considered an impact indicator, but a
measure of research dissemination. However, these conclusions set out a
disturbing fact about the use of citations for research evaluation. If
dissemination could be a factor for citation success, then to what
extent are citations a reflection of research quality and novelty?
Resolving this doubt would be an appealing challenge for those studying
bibliometrics.
Twitter is a powerful way of promoting and diffusing our academic
outputs. This allows us to maintain a wide network of followers that
amplify our message and reach a larger audience. Indirectly, such broad
dissemination could influence the scholarly impact, slightly improving
the prospect of increased citations. Therefore, Twitter cannot be viewed
as a citation index, but as an information-spreading network; and the
tweets of articles should not be considered an impact indicator, but a
measure of research dissemination. However, these conclusions set out a
disturbing fact about the use of citations for research evaluation. If
dissemination could be a factor for citation success, then to what
extent are citations a reflection of research quality and novelty?
Resolving this doubt would be an appealing challenge for those studying
bibliometrics.
This blog post is based on the authors’ article, ‘To be or not to be on Twitter, and its relationship with the tweeting and citation of research papers’, published in Scientometrics (DOI: 10.1007/s11192-016-2113-0).
Note: This article gives the views of
the author, and not the position of the LSE Impact Blog, nor of the
London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
the author, and not the position of the LSE Impact Blog, nor of the
London School of Economics. Please review our comments policy if you have any concerns on posting a comment below.
About the author
José Luis Ortega is
a web researcher partner of the Cybermetrics Lab at the Spanish
National Research Council (CSIC). He has published more than 40 research
papers about web metrics (link analysis, altmetrics, etc.), information
consumption, web usage mining and academic search engines (Google
Scholar, Microsoft Academic Search). Recently, he has released the
monograph Social Network Sites for Scientists: A Quantitative Survey
where he analyses the most relevant academic social networks
(ResearchGate, Academia.edu, Mendeley, etc.) using webometric
techniques. His ORCID iD is 0000-0001-9857-1511.
a web researcher partner of the Cybermetrics Lab at the Spanish
National Research Council (CSIC). He has published more than 40 research
papers about web metrics (link analysis, altmetrics, etc.), information
consumption, web usage mining and academic search engines (Google
Scholar, Microsoft Academic Search). Recently, he has released the
monograph Social Network Sites for Scientists: A Quantitative Survey
where he analyses the most relevant academic social networks
(ResearchGate, Academia.edu, Mendeley, etc.) using webometric
techniques. His ORCID iD is 0000-0001-9857-1511.
Impact of Social Sciences – Twitter can help with scientific dissemination but its influence on citation impact is less clear
No comments:
Post a Comment