Will this post be cited more often? Non-content factors that influence citation rates.
concept. Most ranking algorithms use citations to determine a journal’s
influence or impact. Publication in “high impact” journals is often
the key to tenure and promotion, and the number of times an article has
been cited is often widely touted in tenure and promotion packets.
Image courtesy of Flickr user futureatlas.com
useful for scholars and librarians to know why a particular item gets
cited. We’d all like to think that the only reason an article is cited
is because it’s content is relevant (and more relevant than other items)
to the study at hand.
Unfortunately, there is some evidence to suggest that other,
non-content, factors influence the likelihood of an item being cited.
Big caveat: The quality of these studies is highly variable and their
results are sometimes contradictory. Correlation does not equal
causation.
Nevertheless, most of the non-content factors influencing citation
rate relate to article discoverability. You can’t be cited if you can’t
be read, and you can’t be read if you can’t be found. How likely is an
article to be found in a database? Was the article discussed in a
newspaper or other popular science forum? Does the title clearly
explain what the article is about (and make you want to read more)? Is
the article already connected to a wide circle of readers via multiple
authors or large universities? While there are classic examples of
important scientific publications published in obscure journals, those
are the exception and not the norm.
So, in no particular order, here are a few things that folks suggest might influence how often your article is cited:
A lot of research has looked into various aspects of article titles on subsequent citations.
- Type of title – In an interesting study looking at article titles from PLoS
journals, Jamali and Nikzad (2011) wondered if the type of article
title affected the citation rate of an article. In general, they found
that article titles that asked a question were downloaded more but cited
less than descriptive or declarative titles. Interestingly, Ball
(2009) found that the number of such interrogative titles have increased
50% – 200% in the last 40 years. - Length of title – Jamali and Nikzad suggest that articles with
longer titles are downloaded and cited less, and Moore (2010) in a quick
study found no correlation. However, Habibzadeh and Yadollahie (2010)
suggested that longer titles are cited more (especially in high impact
factor journals) and a positive correlation between article title length
and citation rate was found by Jacques and Sabire (2009). - Specific terms – Disciplinary abbreviations (very specific keywords)
may lead to more citations (Moore, 2010), where as articles with
specific country names in the title might be cited less (Jacques and
Sabire, 2009). - Humorous titles – To my disappointment, a study of articles with
amusing titles in prestigious psychology journals by Sagi and Yechiam
(2008) found that these articles were less likely to be cited than other
articles with unfunny articles. Since funny titles are often less
descriptive of the actual research, these articles could be more
difficult to find in databases.
Articles with funny titles, like this one featured on Discover's Discoblog, may not be cited as much as others.
than negative results. It seems as though positive results are also
more likely to be cited. Banobi et al. (2011) found that rebuttal
articles (either technical reports or full length articles) were less
likely to be cited than the original articles, i.e. the articles with
positive results were more likely to be cited. This correlates well
with the results of Leimy and Koricheva (2005) who found that articles
that successfully proved their original hypothesis were more likely to
be cited than articles that disproved the original hypothesis
Number of authors – Leimu and Koricheva (2005) found
a positive correlation between the number of authors and the number of
citations in the ecological literature, while Kulkarni et al. (2007)
found that group authorship in medical journals increased citation
counts by 11.1. However, a blog post by Moore (2010) suggested that
isn’t wasn’t the number of authors that were important, but their
reputation. A recent study of the chemical literature that was able to
account for article quality (as measured by reviewers rating) found a
correlation with author reputation but no correlation to the number of
authors (Bornmann et al. 2012).
Industry relationship – Studying medial journals,
Kulkarni et al. (2007) found that industry funded research that reported
results beneficial to the industry (i.e. a medical device that worked
or a drug that didn’t show harmful side effects) was more likely to be
cited than non-instustry funded, negative research.
Data sharing – Piwowar et al. (2007) found that
within a specific scholarly community (cancer microarray clinical trial
publications) free availability of research data let to a higher
citation rate, independent of journal impact factor.
Open Access – Lots of studies have been done with
mixed results. A slightly higher number of studies seem to suggest that
open access leads to higher citations (See the excellent review article
by Wagner (2010)).
Popular press coverage – It makes intuitive sense
that journal articles spotlighted by the popular press might be cited
more, but this is difficult to prove. Perhaps the press is merely good
at identifying those articles that would be highly cited anyway.
Phillips et. al (1991) were able to take advantage of an interesting
situation when the New York Times went on strike in 1978 but continued
to produce a “paper of record” that was never published. Phillips et.
al. (1991) found that items written about in the “paper of record” but
not published were no more likely to be cited than other articles.
Length of your bibliography – A 2009 study by
Webster et al. (2009) suggests a correlation between the length of an
articles bibliography and the number of times it is later cited. They
suggest a “I’ll cite you since you cited me” mentality, but online
commentators suggest that this is merely a specious relationship (See
Corbyn, 2010, and comments therein).
So, if you want to publish a paper that gets the highest number of
citations, what should you do? Do your study with a large number of
prestigious co-authors. Submit your long article containing positive
results and a big bibliography to a open access journal. Say something
nice about a pharmaceutical company. Share your data and get the New
York Times to write about it.
Oh, and it might be useful to have some interesting and solid science in there somewhere.
Really Long Bibliography:
Ball, R. (2009). Scholarly communication in transition: The use ofquestion marks in the titles of scientific articles in medicine, life
sciences and physics 1966–2005. Scientometrics, 79(3), 667–679.
Retrieved from: http://www.akademiai.com/index/UH466Q5P3722N37L.pdf
Banobi, J. A., Branch, T. A., & Hilborn, R. (2011). Do rebuttals affect future science? Ecosphere, 2(3), art37. doi:10.1890/ES10-00142.1
Bornmann, L., Schier, H., Marx, W., & Daniel, H. D. (2012). What
factors determine citation counts of publications in chemistry besides
their quality? Journal of Informetrics, 6(1), 11-18. Elsevier Ltd. doi:10.1016/j.joi.2011.08.004
Corbyn, Z. (2010). An easy way to boost a paper’s citations. Nature. Nature Publishing Group. doi:10.1038/news.2010.406
Habibzadeh, F., & Yadollahie, M. (2010). Are Shorter Article
Titles More Attractive for Citations? Cross-sectional Study of 22
Scientific Journals. Croatian Medical Journal, 51(2), 165-170. doi:10.3325/cmj.2010.51.165
Jacques, T. S., & Sebire, N. J. (2010). The impact of article
titles on citation hits: an analysis of general and specialist medical
journals. JRSM short reports, 1(1), 2. doi:10.1258/shorts.2009.100020
Jamali, H. R., & Nikzad, M. (2011). Article title type and its
relation with the number of downloads and citations. Scientometrics,
(49), 653-661. doi:10.1007/s11192-011-0412-z
Kulkarni, A. V., Busse, J. W., & Shams, I. (2007).
Characteristics associated with citation rate of the medical literature.
PloS one, 2(5), e403. doi:10.1371/journal.pone.0000403
Leimu, R., & Koricheva, J. (2005). What determines the citation
frequency of ecological papers? Trends in ecology & evolution,
20(1), 28-32. doi:10.1016/j.tree.2004.10.010
Moore, A. (2010). Do Article Title Attributes Influence Citations?
Wiley-Blackwell Publishing News. Retrieved December 16, 2011, from http://blogs.wiley.com/publishingnews/2010/09/02/do-article-title-attributes-influence-citations/
Phillips, D. P., Kanter, E. J., Bednarczyk, B., & Tastad, P. L.
(1991). Importance of the Lay Press in the Transmission of Medical
Knowledge to the Scientific Community. The New England Journal of
Medicine, 325(16), 1180-1183. Available via: http://www.ncbi.nlm.nih.gov/pubmed/1891034
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007). Sharing
detailed research data is associated with increased citation rate. PloS
ONE, 2(3), e308. doi:10.1371/journal.pone.0000308
Sagi, I., & Yechiam, E. (2008). Amusing titles in scientific
journals and article citation. Journal of Information Science, 34(5),
680-687. doi:10.1177/0165551507086261
Wagner, A. B. (2010). Open access citation advantage: an annotated
bibliography. Issues in Science and Technology Librarianship, (60).
Retrieved from http://www.istl.org/10-winter/article2.html
Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot
Topics and Popular Papers in Evolutionary Psychology : Analyses of Title
Words and Citation Counts in Evolution and Human Behavior , 1979 –
2008. Evolutionary Psychology, 7(3), 348-362. Retrieved from http://www.epjournal.net/filestore/ep07348362.pdf
Will this post be cited more often? Non-content factors that influence citation rates. | the Undergraduate Science Librarian
No comments:
Post a Comment