Source: https://library.smu.edu.sg/topics-insights/Open_Access_Citation_Advantage

The Open Access Citation Advantage - Do Open Access Papers Get Cited More in Policy Documents? 

04 Jan 2023

By Yeo Pin Pin, Head, Research Services

What is Open Access Citation Advantage (OACA)?

When a publication is available free and open access (OA), the OACA effect postulates that there is an increased citation to these OA publications compared to similar paywalled articles.

For example, there are many studies including those by Eysenbach (2006), Norris (2008) and Piwowar (2018) that showed that there was an open access citation advantage for journal articles.

There are a few routes in which Open Access might cause higher citations, but the most obvious one is the idea that increased access makes a paper more likely to be read, shared and ultimately cited. There is some support for this idea given that papers that are downloaded more via Sci-hub are cited more frequently. (Correa, Laverde-Rojas, Tejada, & Marmolejo-Ramos, 2022)

Evidence for OACA effect

WIn fact, there are many papers studying the evidence for the existence of OACA effect. Even as far back as in 2010, there were enough studies for an Open Access Citation Advantage - Annotated Bibliography that listed 5 reviews and 46 articles (Wagner, 2010), while SPARC Europe lists over 70 such studies. More recently, Langham-Putrow, Bakker, & Riegelman (2021), conducted a systematic review that identified 134 papers that were included in the review and found 47.8% of them found evidence for OACA, 27.6% found that they did not exist, 23.9% found OACA in subsets of their samples and only 1 study (0.8%) did not find any OACA.

Why so many studies? It is fair to say that studies around OACA and the existence of the effect are considered controversial by some.

One of the major issues is that most studies are purely observational and do not properly control confounding effects (Crotty, 2016). One obvious confounder for example is that authors tend to be motivated to pay Article Processing Charges (APCs) to make their publications open for what they consider their best works. As such comparing citations from these presumably higher quality papers which are open access versus paywalled papers clearly is problematic.

Controlling confounders is extremely difficult, given the difficulty of finding suitable groups for comparison. For example, quality of a paper is one of the major determinants of citations but is extremely difficult to construct a suitable proxy to control for this factor.

In SMU Libraries, we have done informal analysis of the existence of the OACA effect and like the literature out there we do see a OACA effect over SMU affiliated papers when simply comparing two groups of papers Open Access vs Paywalled articles, but this comes with the same issues already mentioned.

While research designs with Randomised Control Trials would get around these problems and be suited to showing causality rather than just correlation, there have been very few such studies. One early study in 2008, shows no such effect (Davis, 2018).

In the Systematic Review mentioned earlier by Langham-Putrow, Bakker, & Riegelman (2021), of the 134 papers they included they found only two papers where the design was using Randomised Control Trials. During a critical appraisal of the 134 studies for risks of bias in the domains of population, data collection, study design and results, they found only three studies with low overall risk of bias. Incidentally these three studies were split in their verdict, one found an effect, one did not, and one was inconclusive.

New study for OACA for policy citations

Almost all studies on OACA in the past focused on cites to and from journal articles as they drew from traditional citation indexes like Web of Science and Scopus which did not track cites from policy documents. This is now changing with the Overton database.

Katherine Stephan (2022) from Overton did a study of scholarly articles that were cited in their database from policy documents and determined the proportion that were open access. It was found that there was a higher percentage of open access articles cited in policy papers. They found that 70% of articles cited in UK government policy papers and 80% of articles cited in UK Parliamentary Research briefings were open access. It was an interesting study that points to OACA for policy citations.

Studies have also shown that references in Wikipedia tend to favor citations to open access or free to read papers. Given that unlike academic researchers in Universities, Wikipedians and possibly producers of policy papers may not have as good access to paywalled content which may lead them to rely more heavily on Open Access content than the typical researcher with access to the resources subscribed by their library and this might point to a possibly stronger OACA effect.

More study is needed of course.

SMU Libraries subscribe to Overton and you can access it here. Using Overton, we see that publications with SMU affiliation were cited in over 2,600 policy documents mainly by think tanks, governments and international government organizations published by NBER, IFO Institute, World Bank, IZA Institute of Labor Economics, European Union and more. Through InK, our SMU repository, and other platforms, the publications by SMU are made open access and we hope this has contributed to the high citations of SMU publications in policy documents.

It was also suggested by Stephan that further studies be done on related questions like; Is the work cited because it is relevant/because it is open access/is a certain type of work more likely to be open access? We will be on the lookout for further studies on OACA and aim to keep you informed.

References

Correa, J. C., Laverde-Rojas, H., Tejada, J., & Marmolejo-Ramos, F. (2022). The Sci-Hub effect on papers’ citations. Scientometrics, 127(1), 99-126. https://doi.org/10.1007/s11192-020-03806-w

Crotty, D. (2016 August 31). “When bad science wins, or ‘I’ll see it when I believe it.’” The Scholarly Kitchen. https://scholarlykitchen.sspnet.org/2016/08/31/when-bad-science-wins-or-ill-see-it-when-i-believe-it/.

Davis, Philip M., et al. (2008)."Open access publishing, article downloads, and citations: Randomised controlled trial." BMJ 337. https://doi.org/10.1136/bmj.a568

Eysenbach, G. (2006). Citation advantage of open access articles. PLoS Biology, 4(5), e157. https://doi.org/10.1371/journal.pbio.0040157

Lawrence, S. (2001). Free online availability substantially increases a paper's impact. Nature, 411, 521. https://doi.org/10.1038/35079151

Langham-Putrow, A., Bakker, C., & Riegelman, A. (2021). Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PLoS One 16(6), e0253129. https://doi.org/10.1371/journal.pone.0253129

Norris, M., Oppenheim, C., & Rowland, F. (2008). The citation advantage of open‐access articles. Journal of the American Society for Information Science and Technology, 59(12), 1963-1972. https://doi.org/10.1002/asi.20898

Piwowar, H., Priem, J., Larivière, V., Alperin, J. P., Matthias, L., Norlander, B., et al. (2018). The state of OA: A large-scale analysis of the prevalence and impact of Open Access articles. PeerJ, 6, e4375. https://doi.org/10.7717/peerj.4375

Stephan, Katherine. (2022, September 27). Is there an open access advantage for policy citations? We think so (but there’s more to explore). https://blog.overton.io/is-there-an-open-access-advantage-for-policy-citations-we-think-so-but-theres-more-to-explore

Wagner, A. B. (2010). Open access citation advantage: An annotated bibliography. Issues in Science and Technology Librarianship, 60(2). https://doi.org/10.5062/F4Q81B0W