Introduction

Open Access (OA) is a term used to refer to the unrestricted access to scientific articles online, or as Suber’s (2012, p. 4) often referenced definition for OA states: “Open Access literature is digital, online, free of charge, and free of most copyright and licensing restrictions.” OA is generally divided into Gold OA and Green OA, with the former referring to articles in journals that are made completely open by the publisher and the latter referring to parallel publishing of a preprint version of articles by the authors in for instance institutional repositories, possibly after a period of embargo set by the journal publishing the article. In addition, some journals are offering so called Hybrid OA, where an optional author-side payment enables the article to be published OA in an otherwise subscription-based journal. The concept of Hybrid OA, however, has become somewhat more complex recently, as some subscription agreements made between countries or universities and some publishers may include free Hybrid OA publishing for the researchers. The influence that OA might have on citations to research articles has been the focus of many studies throughout the years. Overall, research has provided evidence that OA articles do receive more citations than articles that are behind paywalls (e.g. Harnad and Brody 2004; Hajjem et al. 2005; Kousha and Abdoli 2010). Focusing on a 12-year sample of nearly 14 million articles published between 1992 and 2003 in a hybrid OA journal Hajjem et al. (2005) showed that OA articles received more citations than non-OA articles in the same journal. Wang et al. (2015) compared OA and non-OA papers in Nature Communications and found that the OA papers received more citations and page views, they were downloaded over a longer period, and they were more prominently featured on social media. In a large-scale study Archambault et al. (2014) found an overall citation advantage to articles being freely available, caused by self-archiving and making articles available elsewhere on the web than on the journal websites. On the other hand, a general citation disadvantage for articles in OA journals was found in all research discipline fields outside of physics and astronomy. A citation disadvantage for OA journals have also been discovered by Van Leeuwen et al. (2015), Torres-Salinas et al. (2016) and Dorta-González et al. (2017).

The influence of various OA mechanisms on citations was compared by Piwowar et al. (2018), where 100,000 WoS-indexed articles and reviews published between 2009 and 2015 were sampled. The results showed an 18% increase in citations by publications being available OA, however, that positive effect was due to other mechanisms (self-archiving was found to increase citations by 33%) than publishing in OA journals which was found to have a negative effect of 17% less citations than all publications on average. The authors suggest the reasons for this being an increase in the number of newer and smaller open access journals, a proportion of which are non-English, and the growth of OA mega-journals such as Plos ONE. However, recently it has been found that some of the so called mega-journals have “started to decline in all bibliometric parameters”, of which most notable is the citation counts (Heneberg 2019). Dorta-González and Santana-Jiménez (2018) found OA journals to be in minority in most disciplines (ranging from 6.6 to 27.5%), but that the negative gap in citation frequency to open access journals seems to be closing as journals age. This corroborates the discussions in Piwowar et al. (2018) and may indicate that when OA journals mature and become more widely used in the research community the citation disadvantage fades away.

Altmetrics are considered as “alternative metrics” to traditional bibliometric indicators and they have been defined as “scholarly impact measures based on activity in online tools and environments” (Priem 2014, p. 266). This online activity, or these online events around research outputs on various platforms on the web and in social media (e.g. Twitter, Facebook, blogs, and mainstream news sources), are widely identified, collected and analyzed as potential signals of research impact or online attention the research has received. Some authors have suggested that altmetrics may be able to reflect some aspects of societal impact of research (Bornmann 2014; Bornmann et al. 2019) or to have value in mapping of audiences beyond academia (Holmberg et al. 2014; Robinson-Garcia et al. 2017), as the aggregated attention is not solely generated by academics, but on the other hand, depending on the platform a significant proportion of it may be (Tsou et al. 2015; Mohammadi et al. 2018; Vainio and Holmberg 2017). Altmetrics are considered to function as complements (rather than alternatives) to more traditional citation-based indicators of research impact (Haustein et al. 2015). With OA articles the audiences that the research outputs can reach is thought to potentially be much wider, which might result in more online mentions and increased interaction around the articles and thus be reflected in altmetrics of the articles. In other words, OA articles are accessible even to audiences beyond academia, thus potentially having more societal impact, which would be reflected as higher altmetrics counts. Research measuring the influence of OA on altmetrics has been conducted only to a limited degree but earlier research has already uncovered some evidence of an OA Altmetrics Advantage, i.e. OA articles receiving more online attention (e.g., Adie 2014; Shema et al. 2014; Alhoori et al. 2015; Wang et al. 2015; Cintra et al. 2018). Adie (2014) found that OA articles published in Nature Communications generate significantly more tweets and attract more readers on Mendeley when compared to non-OA articles in the same journal. Teplitskiy et al. (2016) researched the influences that OA status of the journal has on the probability of an article being referenced in the English Wikipedia site. Controlling for field and impact factor the authors found that the odds of an OA journal being referenced in Wikipedia is 47% higher compared to non-OA journals. This result, however, may have been influenced by Wikipedia’s efforts to encourage linking to OA articles (see https://en.wikipedia.org/wiki/Wikipedia:OABOT). In one of the most recent studies Cintra et al. (2018) investigated whether OA status of articles in so-called hybrid OA journals would result in more citations and online mentions when compared to closed articles in the same journals. Their results indicate that OA status does indeed have a positive relationship to citation counts and the number of mentions the articles receives, but that the quality of the paper is still the most important factor driving citations.

The present research continues this line of inquiry and investigates how the OA Altmetrics (and Citation) Advantage holds for a specific case of research articles, namely the research outputs from universities in Finland between 2012 and 2014, and what kind of field specific differences there may exist in that advantage. More specifically, this research investigates whether scientific articles with at least one of the authors affiliated to a Finnish university and published in OA journals (Gold OA) are more often cited as indexed by Web of Science and mentioned on different online platforms such as Mendeley, Twitter, Facebook, mainstream news, blogs and Wikipedia. Furthermore, this research examines possible differences between fields of science in observed OA (dis)advantage. The research enables future studies involving comparisons to research outputs of other countries and between countries. As we will here demonstrate with case Finland, a field by field analysis of OA (dis)advantage by various platforms will yield results concerning visibility and receptivity of research publications, which are obtainable mutatis mutandis in any other country also, provided availability of appropriate data.

Data and methods

Data

Data about research articles with at least one author affiliated with a Finnish university was retrieved from the national Juuli database, which annually collects publication data from all Finnish universities. Juuli provides a unique and comprehensive national data set of the research publications, in contrast to those found in Web of Science and Scopus (see Mongeon and Paul-Hus 2015). Information from 114,496 publications published from 2012 through 2014 was retrieved. Of these 38,819 publications had a DOI attached to them and they were used to search the altmetric data provided by Altmetric.com (data version June 2016), retrieve readership counts from the Mendeley API, and fetch citation counts from Web of Science. A total of 11,716 publications had at least one identified altmetric event (i.e. a tweet, blog post, news post, or Wikipedia article, which mentions the publication). A total of 27% of these publications were published in 2012, 36% in 2013, and 37% in 2014. The DOAJ list of OA journals was retrieved and used to identify which of the publications were published in an OA journal, i.e. a journal in which all articles are openly available, often referred to as Gold OA. This research focused thus only on Gold OA, OA publications in so called hybrid OA journals and Green OA articles were not identified or investigated in this research. In the remainder of this article the abbreviation OA thus refers to articles published in OA journals only. Of the 11,716 articles 17.3% (2027 papers) were published in OA journals.

The 11,716 Finnish research publications under analyses have accumulated 460,437 citations or online mentions, i.e. events, in total by June 4, 2016, which yields 39.3 events per publication (Table 1). Vast majority of events is observed in Mendeley (23.6 events per publication), WoS (7.9) and Twitter (6.5). In each of Facebook, blogs, Wikipedia and mainstream news, however, there is less than one event per paper. To ensure comparability of event counts between different platforms we combine the events in Facebook, blogs, Wikipedia and news into one category we call “other four platforms”, which then yields 1.3 events per publication. Thus, each paper under analyses is likely to have gained an event in at least one of the four platforms.

Table 1 Events of Finnish research publications between 2012 and 2014 by platforms

In determining the field of each research publication, we apply OECD’s field of science (FOS) classification (sub-levels) for classifying the journals in which the publication is published. Consequently, research publications from Finnish universities between 2012 and 2014 under analyses represent 38 different fields. The number of publications by fields range from three in other agricultural sciences to 3096 in Clinical medicine. As we will be utilizing a relational approach in measuring the OA advantage, for each field we will set a threshold value of fifteen publications published in OA journals, which is the lowest possible value still ensuring that each field included will have recorded events for both OA and non-OA publications. Altogether 14 out of the 38 fields exceed the threshold (Table 2). Remaining 24 fields, not meeting the threshold, will be grouped into one category we call “other 24 fields”.

Table 2 Finnish research publications between 2012 and 2014 by fields

Data analysis

While the focus of the paper lies on empirical examination of differences between disciplines and platforms in OA (dis)advantage, at the same time it will take some preliminary developmental steps toward an apt operationalization of disciplinary differences in OA (dis)advantage for (internationally) comparative research. As said, we will here examine how scientific publications of Finnish universities in different fields become visible in various platforms and further what kind of reception they get in those platforms.

As we are inquiring real world questions and working with ‘live’ data sources, we rely on logistic measures and relational methodology as opposed to experimental designs or sample-based statistical inference (cf. Kivinen et al. 2017). This means, for instance, that we will not make such causal statements like “x causes, predicts, affects, influences, explains, or is an antecedent of y”, that “y depends on x” or that “y is in association with x”. Accordingly we will discard all variations of mean citation ratios, mean normalized citation scores and alike (see Archambault et al. 2014; Dorta-González et al. 2017; Harnad and Brody 2004; Wang et al. 2015; Van Leeuwen et al. 2015).

To examine how the OA advantage varies between platforms and between fields of science we use two measures. For demonstrative reasons we will break down our measure for OA (dis)advantage by two dimensions, namely relative visibility and relative receptivity. Conceptually relative visibility precedes relative receptivity in a sense that by definition all research papers are invisible in a certain platform until their very first event, whether it is a citation by some later paper that has been recorded in Web of Science, or a mention in a blog entry or a tweet. Papers, which collect events, become visible in the platform and thus it makes sense to consider further what kind of reception the paper will eventually have within the platform in question. With that, relative visibility indicates how often publications in open access journals gain at least one event as compared to those published in subscription-based journals, while relative receptivity indicates how frequently publications in open access journals gain events as compared to those published in subscription-based journals.Footnote 1

The first measure is relative visibility, which indicates how much more/less often OA publications in field X remain without a single event in platform Y as compared to non-OA publications in respective field and respective platform. For each combination of field X and platform Y relative visibility is obtained by \(\ln \left( {\frac{{1 - {\raise0.7ex\hbox{${f^{\text{OA}} }$} \!\mathord{\left/ {\vphantom {{f^{\text{OA}} } {F^{\text{OA}} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${F^{\text{OA}} }$}}}}{{1 - {\raise0.7ex\hbox{${f^{{n{\text{OA}}}} }$} \!\mathord{\left/ {\vphantom {{f^{{n{\text{OA}}}} } {F^{{n{\text{OA}}}} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${F^{{n{\text{OA}}}} }$}}}}} \right)\), where f is the number of research publications with no events (in platform Y) and F number of all publications (in field X). OA denotes publications in open access journals and nOA publications in subscription-based journals.

Note that only in cases, where exactly equal shares of publications in OA journals and of those in subscription-based journals (in field X) remain without events (in platform Y), relative visibility, as well as relative receptivity, gets value zero. Thus, we will measure both relative visibility and relative receptivity as logarithmic distances—positive or negative—from zero. In the case of relative visibility, a positive log-distance indicates that OA publications gain events more often than subscription-based publications, whereas negative log-distance indicates that OA publications gain events less often than non-OA publications. Relative visibility answers the question, whether publications in open access journals gain at least one event more often than those published in subscription-based journals. With that, relative visibility deals with distributions of 11,716 papers by fields and platforms. On the other hand, in the case of relative receptivity a positive log-distance indicates that OA publications gain events more frequently than non-OA publications, whereas negative log-distance indicates that OA publications gain events less frequently than non-OA publications. Relative receptivity, will thus answer the question, whether publications in open access journals gain events more frequently than those published in subscription-based journals. It, thus, deals with distributions of 460,437 events by fields and platforms. With that relative receptivity indicates how much more frequently OA publications in field X receive events in platform Y as compared to non-OA publications in respective field and respective platform. For each combination of field X and platform Y relative receptivity is obtained by \(\ln \left( {\frac{{{\raise0.7ex\hbox{${C^{\text{OA}} }$} \!\mathord{\left/ {\vphantom {{C^{\text{OA}} } {F^{\text{OA}} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${F^{\text{OA}} }$}}}}{{{\raise0.7ex\hbox{${C^{{n{\text{OA}}}} }$} \!\mathord{\left/ {\vphantom {{C^{{n{\text{OA}}}} } {F^{{n{\text{OA}}}} }}}\right.\kern-0pt} \!\lower0.7ex\hbox{${F^{{n{\text{OA}}}} }$}}}}} \right)\) where C is the total number of events in platform Y.

We will reserve the term likelihood for discussing OA-advantage. Thus for the sake of rigor, when discussing relative visibility we will use the term more/less often, when discussing relative receptivity we will use the term more/less frequently and when discussing OA advantage we will use the term more/less likely.

Finally, OA advantage and disadvantage are measured in terms of log-distance, and defined as linear combinations of relative visibility (α) and relative receptivity (β). OA advantage is obtained by \(\sqrt {\alpha^{2} + \beta^{2} }\), iff α > 0 and β > 0. OA disadvantage in turn is obtained by \(- \sqrt {\alpha^{2} + \beta^{2} }\), iff α < 0 and β < 0.

Results

Relative visibility

Finnish research publications in OA journals generally gain events in the group of other four platforms more often than publications in subscription-based journals; relative visibility of other platforms is 0.17. However, publications in OA journals gain events in Twitter, Mendeley and WoS less often than publications in subscription-based journals; relative visibility in Twitter − 0.04, in Mendeley − 0.15 and in WoS − 0.18 (see bottom row in Table 3).

Table 3 Relative visibility of Finnish research publications in OA journals between 2012 and 2014 by fields and platforms

When examining relative visibility field by field we find that publications in OA journals of earth and related environmental sciences as well as in physical sciences gain events in Mendeley, WoS and other four platforms more often than publications in subscription-based journals of the same fields (Table 3). At the other end, publications in OA journals of medical engineering do not gain events more often in any of the platforms when compared to publications in subscription-based journals of the same field.

When examining relative receptivity field by field we find that publications in OA journals of physical sciences as well as in veterinary sciences gain events in all platforms more frequently than publications in subscription-based journals of the same fields (Table 4). At the other end, publications in OA journals of psychology, clinical medicine, basic medicine, medical engineering and other natural sciences gain events less frequently in all platforms as compared to publications in subscription-based journals of the same field.

Table 4 Relative receptivity of Finnish research publications in OA journals between 2012 and 2014 by fields and platforms. (log-distance; zero indicates no difference in receptivity between publications in OA versus non-OA journals)

OA advantage versus disadvantage

OA advantage for Finnish research publications is found for 13 different combinations of field and platform, including eight different fields; physical sciences, veterinary sciences, social and economic geography, earth and related environmental sciences, psychology, educational sciences, environmental biotechnology and environmental engineering (Table 5). OA disadvantage for Finnish research publications is found for 26 different combinations of field and platform, including 12 different fields as well as the group other 24 fields. Physical sciences and veterinary sciences are the only two fields, where no OA disadvantage is observed in any platform.

Table 5 OA advantage vs. disadvantage of Finnish research publications between 2012 and 2014 by fields and platforms

When examining OA advantage field by field we find that the greatest disadvantage is with research publications in the field of veterinary science referenced in other four platforms (Fig. 1). OA advantage for publications in veterinary science is also found for events in Mendeley. Other fields of publications in which OA advantage is observed for events in two different platforms are social and economic geography (other four platforms and Wos) and earth and related environmental sciences (other four platforms and Wos). In physical sciences OA advantage is observed for events in three different platforms (WoS, other four platforms and Mendeley). In psychology OA advantage is observed for events in Twitter, in educational sciences, environmental biotechnology and environmental engineering for altmetric events in other four platforms.

Fig. 1
figure 1

OA advantage and disadvantage of Finnish research publications between 2012 and 2014 by combinations of fields and platforms

When examining OA disadvantage field by field we find that the greatest disadvantage is with research publications in the field of other natural sciences cited in WoS (Fig. 1). OA disadvantage for publications in other natural sciences is also found for events in Twitter and other four platforms. Other fields of publications in which OA advantage is observed in three different platforms are clinical medicine (WoS, Mendeley and Twitter) and basic medicine (other four platforms, WoS and Mendeley). In medical engineering OA advantage is observed for references in all platforms. In psychology, and in other 24 fields OA advantage is observed for references in WoS and Mendeley.

As mentioned earlier, instead of ruling out 24 fields and 4 platforms due to reasons of not being able to collect enough papers and events over a given time span, we grouped them into other 24 fields and other four platforms. In order to do justice for these fields and platforms we make a separate comparison limited to these 24 fields and four platforms only. However, let us strongly emphasize that the OA (dis)advantages reported in appendix Tables 6 and 7 are comparable within each table only due to the differences in the number of occurrences and should not be compared with any of the other OA (dis)advantages presented in the paper.

On Facebook we find OA advantage for as many as seven different fields and for the group of other 24 fields, while OA disadvantage was found for only two different fields. In both news and blogs OA advantage is found for three different fields and for the group of other 24 fields, while in news OA disadvantage is found for eight and in blogs for six different fields. As for Wikipedia only OA disadvantages were found for five different fields.

In above explained way we also examined the other 24 fields by calculating OA (dis)advantages, where there were enough cases. In philosophy, ethics and religion we find OA advantage for the three main platforms WoS, Twitter and Mendeley. In each of economics and business, sociology, civil engineering and other engineering & technology OA advantage was found in Twitter and in other four platforms. From these four fields, however, also OA disadvantage in Mendeley was found for sociology and civil engineering as well as OA disadvantage in WoS for other engineering & technology. In both of news and blogs OA advantage is found for three different fields and for the group other 24 fields, while in news OA disadvantage is found for eight and in blogs for six different fields. As for Wikipedia only OA disadvantages were found for five different fields. In each of animal & dairy science, language & literature, materials engineering and other social sciences OA advantage was found in one platform. From these, however, also OA disadvantage was found in two platforms for languages & literature and materials engineering. More detailed results are presented in appendix Tables 6 and 7.

Discussion

The emphasis of the paper was mainly on analyzing differences between fields as well as between platforms in OA (dis)advantage in Finnish context. The results showed significant platform specific differences in the OA advantage, as well as disciplinary differences, as could be expected. Psychology for instance had a clear OA advantage on Twitter, while the advantage turned to disadvantage on WoS and Mendeley. Physical sciences and veterinary science had an OA advantage across all platforms, while many other fields showed an OA disadvantage across all platforms. The differences between platforms are to be expected, as the platforms in themselves are very different and attract different types of audiences. Whereas Mendeley is solely used by academics and Mendeley readership counts have been found to correlate significantly with later citations (Thelwall 2018), other platforms such as Twitter for instance, is used by a much more heterogeneous user base. The results from the present research may also echo the findings of Piwowar et al. (2018) and Dorta-González and Santana-Jiménez (2018) that discovered a negative influence of OA on citations. Similar reasoning may also explain the results found here; although not investigated in this research, newer (OA) journals have not had time to establish a reputation within the research community and are thus not attracting attention, not through citations nor altmetrics. This may, however, change with time (Dorta-González and Santana-Jiménez 2018).

The results of this study also reflect the complexity of assessing the influence of OA status on altmetrics and the heterogeneity of altmetrics. Considering the existing disciplinary differences and the dynamic nature of the web and of altmetrics, earlier approaches estimating average counts of online mentions may not do justice to the various ways in which published research in different fields gain visibility. Some methodological outlines to serve future comparative research were also developed in this study. The approach taken here, to derive the possible OA advantage from calculating both the relative visibility and relative receptivity of the articles, may be better adapted to take the complexity of altmetrics into account. The results presented are mostly descriptive and the methodological approach adopted was demonstrated in classificatory purposes only. In the future the full potential of the methodological approach could be explored in various internationally and inter-temporally comparative settings. Further methodological developments include, for instance, operationalization of changes in differences between fields. It will be interesting to find out later on how the OA (dis)advantage patterns for Finnish universities observed here will compare to respective patterns of universities in other Nordic countries for instance.

The study is not without limitations and the results may have been influenced by some factors, which require further investigation. For instance, focusing the analysis only on publications that had received some online attention may have had some influence on the results, effects of which future research should test for. The limitation of only considering WoS-tracked citations is something that could be improved in the future as more inclusive and open citation information sources become available, Furthermore, as DOIs were used as the sole means of detecting altmetric events, the degree of DOI adoption among the journals may have had some influence on the results. In addition, this research focused on articles published in DOAJ listed OA journals, thus the possible OA advantage or disadvantage of Green OA or OA articles in hybrid OA journals influenced the altmetrics and citations for such articles within the subscription journal category to an unknown degree. Future research could for instance use the oaDOI data (used for instance by Piwowar et al. 2018) to determine the influence OA status of articles may have on altmetrics. Furthermore, future research could also employ more qualitative approaches to investigate motivations for mentioning research online and with that try to explain why some research receive more online attention than others.