Why aren’t published works cited? Exploring the influences of bibliographic characteristics on uncitedness phenomenon in social sciences and arts and humanities

Abstract

French

The uncitedness phenomenon was noticed in the 1960s, and Hamilton’s seminal paper in 1991 on uncitedness provoked debate about uncited scientific work. Uncitedness is usually associated with poor quality and zero-impact papers, but some have argued that the community has been misguided to think uncited papers are intellectually worthless. This study aims to help arts and humanities and social sciences scholars take necessary measures to avoid uncitedness by examining the relationship between uncitedness and document features, including document type, publication language, the number of authors, the length of the document, and the number of references. The study collected publications in information science and library science, economics in social sciences, and history and philosophy in arts and humanities. The publication period was from 1997 to 2016, and the citation window was 5 years. In total, we retrieved 653,796 documents from the Web of Science database. The results show inverse relations between uncitedness and document length, number of authors, and number of references. The findings suggest more uncited papers in arts and humanities than in social sciences. The findings also confirmed that the number of uncited research articles, reviews, and proceeding papers had decreased steadily from 1997 to 2016.

Keywords

bibliographic characteristics cited references document length document type number of authors publication language uncitedness

Introduction

Citation counts are measures used to evaluate the impact of academic work. Hence, scholars are committed to publishing highly cited articles. Nevertheless, uncitedness is common across disciplines (Egghe et al., 2011; Nicolaisen and Frandsen, 2019; Thelwall, 2016). Uncitedness refers to the scientific publications, authors, or journals that are not cited within a defined time window (Dorta-González et al., 2020; Liang et al., 2015). Okubo (1997) pointed out that more than half of the scientific publications in the Science Citation Index (SCI) were never cited once in the 5 years following their publications. The uncitedness rate varies by discipline. Pendlebury (1991) found that arts and humanities (AH) have the highest uncitedness rate, followed by social sciences (SS), engineering, and hard sciences. Sugimoto and Larivière (2018) studied articles published in 1990 with a citation window of 27 years and noted that the uncitedness rate for Medical Science was 12% and 70% for AH. The uncitedness rate also varies by sub-disciplines. Pendlebury (1991) found that among hard sciences, physics and chemistry have the lowest uncited rates – 36.7% and 38.8%; atomic, molecular, and chemical physics had only 9.2% of articles uncited.

An uncited article is often perceived as having poorer quality and no impact than a highly cited article (Hou and Ye, 2020; Hu and Wu, 2014). Some disagree with such an association. Glänzel et al. (2006: 267) argued that the uncited does not reveal the quality of a paper or any information about the authorship status in the community. Besides, by examining the distribution of usage count of the uncited articles, Dias et al. (2023) discovered that uncited articles are not necessarily unused.

The study presented here aims to help AH and SS scholars take necessary measures to avoid uncitedness by addressing the following research questions:

RQ1. What are the bibliographic characteristics of an uncited article?

RQ2. Are there differences in the characteristics of the uncited work in AH and SS?

The study explored document characteristics such as discipline, document types, language, author number, and reference number to answer the questions. The study focused on the publications in AH and social science from 1997 to 2006. A total of 653,796 documents were retrieved from the Web of Science^®. The results showed inverse relations between the uncited and document length, author number, and reference number. The results supported the findings of other studies that the number of uncited decreased in the later period, possibly due to the improvement of dissemination and utilization environment of literature (Wallace et al., 2009).

Related works on uncitedness

Hamilton’s work published in the 1990s drew attention to the uncited scientific work. The opening statement suggests that ‘New evidence raises the possibility that a majority of scientific papers make negligible contributions to knowledge’ (131). Hamilton (1991) reported that 55% of published papers between 1981 and 1985 covered in the Institute for Scientific Information (ISI) analysis received no citation in the first 5 years following the publication. Hamilton’s essay was later criticized for being misguiding but influential (Garfield, 1991). Pendlebury (1991) posited that the uncited rates reported in Hamilton’s essay were correct, but the interpretations of these rates were not. Pendlebury argued that knowing what is in the numbers is necessary before interpreting them and drawing a conclusion. Pendlebury suggested several factors which could contribute to the uncitedness rate, including field, document type, country, and time. Pendlebury concluded that the uncitedness phenomenon was not as alarming as Hamilton previously suggested.

Sugimoto and Larivière (2018) argued that uncitedness is often misunderstood and misinterpreted. First, the assumption that scientific value and citation are linked leads to the impression that the uncited is worthless or of poor quality. They asserted that, in reality, the importance of research is not exclusively determined by its citation and that, ‘The scientific community should move from a pejorative to descriptive understanding of uncitedness’ (76).

Second, the time element must be considered in the analysis. Sugimoto and Larivière described the process of an article being cited and pointed out that the process’s duration varies by discipline. Thus, the citation window can affect the analysis. The concern about the citation window is shared by other scholars (Clermont et al., 2020; Nicolaisen and Frandsen, 2019). The length of the citation window varies from one discipline to another. For instance, it takes longer for an article to be cited in AH and SS; hence, a shorter citation window leads to high uncited rates (Nane, 2015). Garfield (1991) and Glänzel et al. (2003) found the delayed recognition phenomenon and argued that in sciences, uncited work might receive a citation in the future. Sugimoto and Larivière found that about a third of documents in AH received citations within 25 years following publication. Therefore, they concluded that different citation windows need to be used in different disciplines to assess uncitedness better.

Third, the uncitedness rate depends on genre conventions in the discipline. Research tends to be published in outlets not indexed by the primary sources in many applied fields, such as engineering. Sugimoto and Larivière explained that this is a reason for the higher uncitedness rate in engineering than in physics. The coverage of the database determines citing items in the analysis, thus the uncited rates. They pointed out that the uncited rates are lower in the medical sciences because most references cited by the documents are indexed in the databases. In contrast, the high uncited rates in AH are due to the lower coverage of journals, books, and so on.

Liang et al. (2015) conducted a comprehensive study exploring uncited papers, authors, and topics in Library Information Science. They collected articles published between 1991 and 2010 from four major journals: Information Processing and Management, Journal of the American Society for Information Science and Technology, Journal of Documentation, and Scientometrics. The bibliometric indicators used to compare cited and uncited articles include the average number of pages (length of the article), references, and authors per paper. They found that the uncited were shorter, cited fewer references, and had fewer authors. Other studies have found relationships between these bibliometric indicators and uncitedness (Peters and van Raan, 1994; Stern, 1990).

Document type is significant to uncitedness. Nicolaisen and Frandsen (2019) reported that there was a higher uncitedness rate for conference proceedings (0.33) compared with articles (0.16) in Computer Science. Reviews are published more frequently in AH, yet a high fraction of these reviews is uncited (e.g., 0.44 compared with 0.12 in physics and astronomy). Heneberg (2013) showed that over 90% of uncited from highly visible scientists fall into editorial materials and progress reports presented at international meetings (meeting abstracts), discussion items (letters to the editor, discussion), personalia (biographic items), while only a few original articles and reviews (0.9% and 0.3%, respectively) are uncited.

The number of references cited in a scientific research article has been consistently identified as a strong predictor of citation count, with fewer references correlating with a higher probability of uncitedness. Kousha and Thelwall (2024) found a positive association between several bibliographic characteristics of cited references and citation rates. These characteristics included volume, age, impact factor, and international scope. The association aligns with Stern’s (1990) findings in the biomedical field, where a significant inverse relationship was observed between uncitedness and the number of references. Stern suggests that works demonstrating a strong connection to prior research are more likely to be acknowledged in subsequent publications. Similarly, Hou and Ye (2020) posit that the number of references reflects the richness of a paper’s content, leading to the perception of a more comprehensive work with more citations. Baruch et al. (2022) investigated uncited publications in management and found a positive correlation between reference list length and markers of academic rigor and breadth of knowledge. They further suggest that a lengthy reference list reduces the likelihood of uncitedness and improves the chances of publication in high-ranking journals. Elango (2016) reinforces this notion, arguing that a limited number of references implies a lack of substantial information offered to the reader. These findings are further supported by additional studies demonstrating a positive correlation between the number of references and citation counts (Didegah and Thelwall, 2013; Onodera and Yoshikane, 2015).

Peters and van Raan’s (1994) study showed a positive correlation between the number of authors and citations. They found that a paper with four or more authors is cited most and almost twice as often as a paper with one or two authors. Borsuk et al. (2009) studied the effect of the number of authors on citations in ecology and found an inverse correlation between them. Stern (1990) asserted that the fewer authors, the higher the uncitedness rate. Elango (2016) concluded that tribology papers with fewer references and fewer authors are at an increased risk of not receiving a citation.

Publication language contributes to uncitedness (Borsuk et al., 2009; Elango, 2016). The existing literature argues that publication language impacts the citation frequency of a paper. Van Raan (2005) found that non-English publications can dilute the academic impact of a university or a department, and only English-language journals after removing German-language journals can improve the academic impact of a university by 25%. It was explained that non-English publications have less impact and receive fewer citations because of language biases in the coverage of citation databases (Van Leeuwen et al., 2001; Van Raan, 2005). Poomkottayil et al. (2011) studied the influence of publication language on citation in the Swiss context and analyzed 301 oral surgery research papers published between 2002 and 2007. The results revealed that English publications received 6 (ISI database) and 7 (Google Scholar) times more citations than publications in German and French. The authors hence suggested that to make their work more accessible to the international scientific community, researchers should publish their work in English. The work mentioned did not directly study the relation between uncitedness and publication language; nevertheless, an inference can be made based on their findings. Elango (2016) claimed about 45% of uncited papers were published in languages other than English.

Data collection and method

Data for the study was drawn from the Web of Science^®. The study selected articles published from 1997 to 2016 in Information Science and Library Science and Economics in SS and History and Philosophy in AH. We chose 5-year citation window time because, first, the existing research suggests that it takes longer for articles in SS and AH to receive citations. Second, the longer citation time window allows some highly cited publications to be included in the analysis. Table 1 summarizes the distribution of documents collected across the four subject areas.

Table 1.

Distribution of the 653,796 documents across the subject areas in social sciences and arts and humanities.

Subject category	Number of documents	Percentage (%)
Information Science and Library Science	99,715	15
History	86,016	13
Economics	35,9333	55
Philosophy	108,732	17

This study explored the uncitedness of three document types, including research articles, reviews, and proceeding papers. A total of 653,796 documents were retrieved, and bibliographic information was collected, including the number of authors, title, source journal, volume, number of pages, publication language, cited references, and the annual distribution of citations per document between 1997 and 2016. The distributions of cited and uncited in each document type are shown in Table 2. The distribution of 653,796 documents between AH and SS is 194,748 and 459,048, respectively. Of the publications in AH, 78,125 (40.12%) were cited at least once, while 116,623 (59.88%) were uncited in the 5-year publication window. Similarly, 304,694 (66.38%) publications were cited at least once, while 154,354 (33.62%) were uncited in SS. There were more uncited research articles, reviews, and proceeding papers in AH than in SS.

Table 2.

Distributions of the uncited in each and all document types (n = 653,796).

Document type	Total number of documents	Number of uncited	Uncitedness rate %
Article	538,835	185,637	34.45
Review	9562	3366	35.20
Proceedings paper	105,399	81,974	77.77

Results

Publication language and uncitedness

Table 3 shows that work published in English received more citations than work published in other languages. Comparing work in European and non-European languages, the latter received fewer or no citation. For example, the uncitedness rate of English publications in AH is 45%, whereas it is 81%, 83%, 88%, 97%, and 100% for Spanish, Russian, Italian, Chinese, and Japanese, respectively. The same result is also observed in the SS publications.

Table 3.

Publication language and uncitedness in articles, reviews, and proceedings papers in arts and humanities and social sciences.

Publication language	Arts and humanities (n = 194,748)		Social sciences (n = 459,048)
Publication language	Cited	Uncited	Cited	Uncitedness
English	66,047 (55%)	54,941 (45%)	29,7117 (69%)	130,454 (31%)
Spanish	3654 (19%)	15,705 (81%)	2243 (30%)	5286 (70%)
Italian	939 (12%)	7005 (88%)	136 (29%)	335 (71%)
French	1474 (14%)	9442 (86%)	786 (24%)	2514 (76%)
Russian	2114 (17%)	10,652 (83%)	1182 (20%)	4772 (80%)
Chinese	29 (3%)	855 (97%)	77 (4%)	2022 (96%)
Croatian	162 (13%)	1048 (87%)	192 (24%)	617 (76%)
Japanese	0	5 (5%)	17 (12%)	129 (88%)

Document length and uncitedness

We organized documents into nine groups according to length to explore the relation between document length and uncitedness (Figure 1). A total of 1139 papers were excluded from the analysis because the information on their page numbers was unavailable. The correlation test showed an inverse relation between document length and uncitedness (r = –0.81), meaning that the shorter the document length, the higher the uncitedness rate. Papers lengths between 1 and 5 pages have the highest uncitedness rate (71.24%), but the rate falls sharply once the length increases to 6 pages and more. The uncitedness rate of papers between 6 and 10 pages is 51.38%, which falls to 36.68% when the length exceeds 10 pages. The uncitedness rate continues to fall steadily as the length of papers increases (from 36.68% to 31.30%). The uncitedness rate falls below 30% when the paper length reaches 31 pages and more. Notably, 30–40 pages appear to be the threshold. Papers between 36 and 40 pages have the lowest uncitedness rate of 28.57%, but the rate increases again after this threshold.

Figure 1.

Distribution of uncited and cited according to the length of articles, reviews, and proceeding papers.

The one-way analysis of variance (ANOVA) test suggests a significant difference in the pattern of uncitedness and document length between AH and SS (p = 0.0006, p < 0.01). The correlation analyses show a stronger negative correlation between uncitedness and document length in AH (r = –0.87) than in SS (r = –0.79728).

We compared documents’ uncitedness rates in relation to their length in different citation time windows: 1997–2001 and 2012–2016. A one-way ANOVA test reveals that there is no significant difference in the pattern of uncitednesses in these two periods (p = 0.6317, p > 0.05) (Figure 2). We also compared the patterns of uncitednesses of each period against the overall pattern of uncitednesses (1997–2016). The result of the one-way ANOVA test suggests that there is no significant difference in the pattern of uncitednesses between the two periods and 1997–2016 (p = 0.6939, 0.9538). Hence, the pattern of uncitednesses in 1997–2001 and 2012–2016 is in line with the pattern of 1997–2016.

Figure 2.

A comparison of length of articles, reviews, and proceedings paper and uncitednesses during 1997–2001 and 2012–2016.

Number of authors and uncitedness

We organized the documents into eight groups according to the number of authors. Figure 3 shows an inverse relationship between the number of authors and uncitedness (r = –0.87424) and a positive relationship between the number of authors and citedness (r = 0.87423). Figure 2 shows a skew distribution of the number of authors in which 79.85% of AH and SS work has been carried out by one to two authors, and so has the distribution of uncited. The distribution exhibits that work done by a single author had the highest uncitedness rate. The uncitedness rate falls significantly from 52% to 32%; at the same time, the citedness rate increases considerably from 48% to 68% when the number of authors increases to 2. The uncitedness rate decreases from 32% to 27% when the number of authors increases to 3. This observation reveals that even though it may be a community norm to publish work independently (i.e., more than 53.24% of the work published was by a single author), single-author papers have a higher uncitedness rate than papers with multiple authorships.

Figure 3.

Patterns of uncited and cited according to the number of authors of articles, reviews, and proceedings paper.

A one-way ANOVA test suggests a significant difference in the pattern of uncitedness and number of authors between AH and SS (p = 0.0010, p < 0.05). The correlation tests show a strong negative correlation between the number of authors and uncitedness in AH (r = –0.7302) and in SS (r = –0.9394). In AH, 90.65% are single-author papers (Table 4). These papers contribute to 87.25% of cited, yet, at the same time, they contribute to the majority of uncited (92.93%). In SS, papers with 1 (37.37%), 2 (34.82), and 3 (18.52%) authors contribute to 89.29% of cited and to 93.49% of the uncited. Among those, single-author papers are the major contributor to uncited counts for 47.44%.

Table 4.

Citedness, uncitedness, and number of authors in arts and humanities and social sciences.

Number of authors	Arts and humanities (n = 194,748)			Social sciences (n = 459,048)
Number of authors	Cited	Uncited	Uncitedness rate %	Cited	Uncited	Uncitedness rate %
1	68,167	108,374	92.93	98,306	73,223	47.44
2	7574	6588	5.65	111,303	48,524	31.44
3	1359	1122	0.96	62,445	22,565	14.62
4	442	293	0.25	20,144	6660	4.31
5	184	111	0.10	6296	2051	1.33
6	76	41	0.04	2732	697	0.45
7	61	37	0.03	1415	279	0.18
⩾8	81	57	0.05	2053	355	0.23
Total	78,125	116,623	100	304,694	154,354	100

The one-way ANOVA test suggests that there is a significant difference in the pattern of uncitednesses and the number of authors between 1997–2001 and 2012–2016 (p = 0.04, p < 0.05). This is shown in Figure 4 where there were strong inverse relations between number of authors and uncitedness rate in both 1997–2001 (r = –0.6748) and 2012–2016 (r = –0.8688) periods. Compared with the uncitedness rates in the 1997–2001 period, the uncited rates in 2012–2016 were generally lower.

Figure 4.

A comparison of number of authors of articles, reviews, and proceedings paper and uncitednesses during 1997–2001 and 2012–2016.

Number of references and uncitedness

The documents were categorized based on the number of cited references to examine the relationship between the number of references and uncitedness. Figure 5 shows that the uncitedness rate decreases (r = –0.8926) while the number of cited references increases. The gap between the uncitedness and citedness rates is the smallest when the reference number is between 11 and 15. The citedness rate starts to surpass the uncitedness rate after this point, and the gap between the two continues to widen as the number of cited references increases. The papers with 61–70 references had the lowest uncitedness and highest citedness rates (21% and 79%). In contrast, the papers with no reference or less than 6 references had the highest or second highest uncitedness rate (Table 5).

Figure 5.

The relations between the number of cited references in articles, reviews, and proceedings paper and uncitedness in AH and SS.

Table 5.

Number of references and citedness and uncitedness in arts and humanities and social sciences.

Number of references	Arts and humanities (n = 194,748)			Social sciences (n = 459,048)
Number of references	Citedness	Uncitedness	Uncitedness rate %	Citedness	Uncitedness	Uncitedness rate %
0	2024	10,169	8.72	3986	16,490	10.68
1–5	2629	10,173	8.72	8032	21,453	13.90
6–10	4701	13,536	11.61	18,579	29,859	19.34
11–15	6129	14,267	12.23	25,477	22,324	14.46
16–20	6762	12,969	11.12	29,935	16,446	10.65
21–25	6594	10,803	9.26	32,365	12,716	8.24
26–30	6160	8669	7.43	32,468	9410	6.10
31–35	5828	6863	5.88	29,513	6975	4.52
36–40	5136	5442	4.67	26,006	5037	3.26
41–45	4487	4321	3.71	21,183	3626	2.35
46–50	3886	3562	3.05	16,948	2614	1.69
51–55	3331	2834	2.43	13,441	1910	1.24
56–60	2922	2278	1.95	10,206	1341	0.87
61–65	2533	1818	1.56	7939	981	0.64
66–70	2165	1508	1.29	6224	759	0.49
⩾71	12,838	7411	6.35	22,392	2413	1.56
Total	78,125	116,623	100	304,694	154,354	100

A one-way ANOVA test reveals a significant difference in the pattern of uncitedness and number of references between AH and SS (p = 0.00043, p < 0.05). The correlation test suggests a strong negative correlation between the number of references and uncitedness in both disciplines (r = –0.9825 in AH and r = –0.8908 in SS). AH papers with references between 11 and 15 contribute 12.23% of uncited. After this point, the uncitedness rate drops steadily as the number of references increases. However, the uncitedness rate surges again by about 5% when the number of references is 71 or more. SS papers with reference numbers between 6 and 10 have the highest uncitedness rate, around 19.34%, but the rate drops sharply to 14.46% with 11–15 references. The uncitedness rate continues decreasing until the reference number reaches 71 or more. The turning point where the uncitedness rate drops afterward for AH is 11–15 references, and for SS is 6–10 references.

The patterns of uncitednesses in 1997–2001 and 2012–2016 are similar to the overall trend and according to a one-way ANOVA test there is no significant difference in uncitedness in these two periods (p = 0.9724, p > 0.05). However, it is evident that the uncitedness rates in the period 2012–2016 are lower than in 1997–2001 as the growing number references, especially after the threshold 31–35 (Figure 6).

Figure 6.

A comparison of number of cited references in articles, reviews, and proceedings papers and uncitedness phenomenon during 1997–2001 and 2012–2016.

Discussion

Research articles, reviews, and proceeding papers are favorable forms of scholarly communications, and their uncitedness rates are lower than other document types in AH and SS. Journal articles are the most favorable communication medium among the three document types and attract the most citations. This finding supports Nicolaisen and Frandsen’s (2019) work which suggests that a journal article is the preferred form of communication across six subject categories. Nicolaisen and Fransen discovered that the number of conference proceeding papers is twice as many as journal articles in Computer Science, yet the uncitedness ratio of conference papers is also twice of articles. This study also found a similar result, where the ratio of uncited conference proceeding papers (0.78) is more than twice the number of research articles (0.35). Review papers are published frequently in medicine, AH, and SS, but the uncitedness ratio of review papers in AH is high (0.40) suggested by Nicolaisen and Fransen. In our study, the uncitedness ratio of review papers in AH and SS combined is 0.35.

Our findings suggest a correlation between publication language and uncitedness. The English language is the dominant publication language in AH and SS. Articles, reviews, and proceeding papers published in English received more citations than non-English publications. This result supports Czerwon and Havemann (1993) work which quantified the influence of publication languages on citations and showed that publications in English exhibited a significantly higher citations-per-paper average. Articles in non-English European languages received more citations than those in non-English and non-European (Table 3). Elango (2016) explained that it could be because fewer researchers can understand papers in other languages, especially Chinese, Japanese, or Korean, which prevents the publications from being read widely and thus cited. Thus, Elango commented, ‘These types of publications transfer knowledge to local or national end-users only’ (2).

A negative correlation exists between the number of authors and the likelihood of a paper remaining uncited. This finding aligns with Rousseau’s (1992) Bayesian probability model predictions, which suggest that multi-authored articles garner more citations than single-authored works. Other studies have corroborated this relationship (Leimu and Koricheva, 2005; Thelwall and Maflahi, 2020). Hu et al. (2019) proposed that the scholarly networks of authors contribute to the observed negative correlation. Multi-authored research likely benefits from additional rounds of editing and a wider range of expertise, leading to the perception of higher quality (Borsuk et al., 2009). Furthermore, such collaborations may be viewed as more trustworthy due to the notion that ‘they can’t all be wrong’ (Borsuk et al., 2009: 28). Another potential explanation lies in the ‘audience effect’ – larger authorship teams may generate greater interest through more extensive networking efforts, ultimately increasing citations (Wagner et al., 2019). However, self-citation within large author groups is a potential confounding factor. Van Raan (1998) argued that collaboration naturally leads to more self-citations (‘more authors to cite themselves’) and acts as a citation amplifier (427). Therefore, self-citation practices may partially inflate the observed correlation between authorship size and citations.

Our findings provide further evidence to support the effect of the length of the document on uncitedness. The analysis revealed a strong inverse relation between document length and uncitedness, for example, 1–5 (71.24%), 6–10 (51.38%), 11–30 (34.33%), 31–40 (28.79%), and ⩾41 (29.96%). The cited and uncited work in each group suggests that most articles in AH and SS are between 11 and 30 pages (61.12%). Although the longer the papers, the lower the uncitedness rate, there are only a few papers which are longer than 30 pages (7.77%) and even fewer papers longer than 40 pages (2.01%). It should be noted that, as other established studies, we measured article length using the total number of pages, even though these depend on page layout, format, and number of references. Like other studies, we interpret the correlation as the result that longer articles contain more citable contents compared with shorter articles (Kousha and Thelwall, 2024).

We show that the gap between rates of citedness and uncitedness widened as the number of references increased. We echo explanations by previous studies and argue that the number of references indicates the reliability of the papers and the connection of the papers to the existing research (Onodera and Yoshikane, 2015). Scholars are more likely to find papers with many references through citation links or random searches. Furthermore, the number of references indicates the depth or breadth of an article concerning the literature. In either case, scholars are likely to perceive the paper as useful or relevant to their research. This interpretation coincides with Hu and Wu’s (2018) survey results which identified that the relevance of the literature to the research is a key reason for a paper being cited.

In line with the literature (Katchanov et al., 2023; Larivière et al., 2008; Pendlebury, 1991), we have found that the number of uncited decreases over time (Table 4 and Figures 2, 4, and 6). The uncitedness rates of the work published from 2012 to 2016 are lower than from 1997 to 2001. A possible explanation could be that the younger documents are more current and relevant to the research agenda than the older ones (Baruch et al., 2022). Larivière et al. (2008) suggested the improved accuracy of bibliographic databases, for example, Thomson, could be a reason for the trend. In addition to the above, Wallace et al. (2009) suggested the emergence of electronic access to scientific papers, increased cited references per paper, and increased scientific publications.

Conclusion

We explored the uncitedness phenomenon in AH and SS. The bibliographic characteristics explored include the publication language, the number of authors, the number of references, and the length of the document. Overall, our findings are in line with that of the previous studies. Among these characteristics, the length of the document and publication language has been studied the least in the literature. Therefore, our study contributes to this area by showing correlations between uncitedness and the length of the articles and publication language.

The findings have a few implications for AH and SS scholars regarding their strategy to avoid uncitedness. First, to increase the influence of their work, scholars in both disciplines should consider publishing their work as full research articles in journals. Reviews may be frequently published in both disciplines, but the number of uncited reviews is also high. Second, scholars could consider collaborating and publishing papers with their colleagues. Single authorship seems to be a norm in both disciplines, but a large proportion of single-author work is uncited. This is especially true in AH. Levitt (2015) suggested that papers with at least two authors receive high citation impacts in SS. Based on our findings, we suggest that two to four authors are the optimal numbers. Interdisciplinary collaborations have a high citation impact in other disciplines but not in AH and SS (Haustein et al., 2014). Institutional collaborations generate more citations than in-house collaborations in natural sciences but not in AH and SS (Franceschet and Costantini, 2010). Hence, it is not necessary for AH and SS scholars to find co-authors outside their own disciplines. Third, the length of the paper and the number of references relate to the perceived quality of the paper and the perceived quantity of information contained in the paper. However, the length of the document is highly regulated by journals nowadays. Scholars often have to decide between the main body of text and the number of references. Our findings suggest that the number of cited references plays a significant role in scholarly impact, as demonstrated by research highlighting the importance of both relevance (Hu and Wu, 2018) and currency (Kousha and Thelwall, 2024) of cited works. Therefore, scholars are advised to prioritize citing current and highly relevant references to enhance the discoverability and citability of their research. Articles with more recent references presumably indicate a more current topic that is more likely to be cited by new articles (Ahlgren et al., 2018; Onodera and Yoshikane, 2015). For the same reasons, publishers may consider not including references in the word counts to increase journal citation impact. Finally, for knowledge diffusion internationally, we recommend authors consider publishing their work in English so that it can be read widely (Elango, 2016).

Our dataset was extracted from the Web of Science, yet there is bias in the Web of Science where English journals are overrepresented, and articles in AH and SS are underrepresented (Mongeon et al., 2016). Papers written in languages such as Russian, Italian, Chinese, and Japanese, respectively, may receive many more citations in journals that the Web of Science does not cover. Currently, there is no ideal solution to these problems, as all databases have inbuilt language and field coverage biases. Therefore, further research on language and uncitedness or disciplines and uncitedness also need to consider the limitations of existing databases when concluding. Another limitation is that the current study did not separate the main body of text from the reference list; therefore, it is difficult to differentiate the following combinations, which lead to variation in article length: (1) a long main body of text with a short reference list, (2) a short main body of text with a long reference list, and (3) a long main body of text with a long reference list. Since the length and number of references are related, future research should differentiate the two to better understand the effects of the length of the papers and number of references.

Footnotes

Acknowledgements

This work was supported by the Major Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu Province of China ‘Research on Intelligent Recognition and Cultivation Mechanism of High Value Patents for Future Industries in Jiangsu Province’ and supported by Qing Lan Project. We are grateful for contributions of Qiangqiang Hao (student) in drawing of partial charts.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the Key Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu Province ‘Research on Intelligent Recognition and Cultivation Mechanism of High Value Patents for Future Industries in Jiangsu Province’ (Grant No. 2024SJZD065), the Project of Social Science Foundation of Jiangsu Province ‘Research on the Identification and Tackling Mechanism of Crossing disciplinary Technologies in Future Industries for Jiangsu Province Based on High Value Patents’ (Grant No. 24TQB005), and Qing Lan Project.

ORCID iD

Zewen Hu

Data availability statement

The data that support the findings of this study are openly available in Web of Science at . The data that support the findings of this study are available from the corresponding author upon reasonable request.

Author biographies

Angela Lin is a lecturer (assistant professor) in information systems at the Information School, University of Sheffield. Her principal research interests are in the areas of information systems, management, digital transformation, digital business, the business applications of the Internet of Things, and citation analysis. She has published papers in information systems and information science.

Jingjing Cui is a student who owns a master’s degree of School of Management Science and Engineering, Nanjing University of Information Science and Technology, mainly focusing on data mining and intelligence analysis.

Zewen Hu is a professor and doctoral supervisor in the School of Management Engineering, Nanjing University of Information Science & Technology. His research interests include scientometrics, machine learning and knowledge mining. He has successfully led 2 National Natural Science Foundation projects and published over 30 papers in SSCI journals such as the Journal of Informetrics, Information Processing & Management, and so on.

References

Ahlgren

Colliander

Sjögårde

(2018) Exploring the relation between referencing practices and citation impact: A large-scale study based on Web of Science data. Journal of the Association for Information Science and Technology 69(5): 728–743.

Baruch

Homberg

Alshaikhmubarak

(2022) Are half of the published papers in top-management-journals never cited? Refuting the myth and examining the reason for its creation. Studies in Higher Education 47(6): 1134–1149.

Borsuk

Budden

Leimu

, et al. (2009) The influence of author gender, national language and number of authors on citation rate in ecology. The Open Ecology Journal 2(1): 25–28.

Clermont

Krolak

Tunger

(2020) Does the citation period have any effect on the informative value of selected citation indicators in research evaluations? Scientometrics 126(2): 1019–1047.

Czerwon

Havemann

(1993) Influence of publication languages on the citation rate of scientific articles: A case study of East German journals. Scientometrics 26(1): 51–63.

Dias

Lev

Anderson

(2023) Low cited articles in operations research/management science. Omega 115(C): 102792.

Didegah

Thelwall

(2013) Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics 7(4): 861–873.

Dorta-González

Suárez -Vega

Dorta-González

(2020) Open access effect on uncitedness: A large-scale study controlling by discipline, source type and visibility. Scientometrics 124(3): 2619–2644.

Egghe

Guns

Rousseau

(2011) Thoughts on uncitedness: Nobel laureates and field medalists as case studies. Journal of the American Society for Information Science and Technology 63(2): 1637–1644.

10.

Elango

(2016) Uncitedness in scientific publications: A case study of tribology research. SRELS Journal of Information Management 53(4): 293–296.

11.

Franceschet

Costantini

(2010) The effect of scholar collaboration on impact and quality of academic papers. Journal of Informetrics 4(4): 540–553.

12.

Garfield

(1991) To be an uncited scientist is no cause for shame. The Scientist 5(6): 12.

13.

Glänzel

Debackere

Thijs

, et al. (2006) A concise review on the role of author self-citations in information science, bibliometrics and science policy. Scientometrics 67(2): 263–277.

14.

Glänzel

Schlemmer

Thijs

(2003) Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon. Scientometrics 58(3): 571–586.

15.

Hamilton

(1991) Research papers: Who’s uncited now? Science 251: 25.

16.

Haustein

Larivière

Börner

(2014) Long-distance interdisciplinary research leads to higher citation impact. In: 19th international conference on science and technology indicators, Liden, 3–5 September.

17.

Heneberg

(2013) Supposedly uncited articles of Nobel laureates and Fields medalists can be prevalently attributed to the errors of omission and commission. Journal of the American Society for Information Science and Technology 64(3): 448–454.

18.

Hou

(2020) Are uncited papers necessarily all nonimpact papers? A quantitative analysis. Scientometrics 124(2): 1631–1662.

19.

Lin

Willett

(2019) Identification of research communities in cited and uncited publications using a co-authorship network. Scientometrics 118(01): 1–19.

20.

(2014) Regularity in the time-dependent distribution of the percentage of never-cited papers: An empirical pilot study based on the six journals. Journal of Informetrics 8(1): 136–146.

21.

(2018) A probe into causes of non-citation based on survey data. Social Science Information 57(1): 139–151.

22.

Katchanov

Markova

Shmatko

(2023) Uncited papers in the structure of scientific communication. Journal of Informetrics 17(2): 101391.

23.

Kousha

Thelwall

(2024) Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper. Journal of the Association for Information Science and Technology 75(3): 215–244.

24.

Larivière

Archambault

Gingras

, et al. (2008) The fall of uncitedness. In Book of Abstracts of the 10th International Conference on Science and Technology Indicators. Vienna: University of Vienna, pp.279–282.

25.

Leimu

Koricheva

(2005) What determines the citation frequency of ecological papers? Trends in Ecology and Evolution 20(1): 28–32.

26.

Levitt

(2015) What is the optimal number of researchers for social science research? Scientometrics 102(1): 213–225.

27.

Liang

Zhong

Rousseau

(2015) Uncited papers, uncited authors and uncited topics: A case study in library and information science. Journal of Informetrics 9(1): 50–58.

28.

Mongeon

Paul-Hus

(2016) The journal coverage of Web of Science and Scopus: A comparative analysis. Scientometrics 106(1): 213–228.

29.

Nane

(2015) Time to first citation estimation in the presence of additional information. In: Salah

Tonta

Salah

AAA

, et al. (eds) Proceedings of the 15th International Society of Scientometrics and Informetrics Conference. Istanbul: Bogazici University Print-house, pp.249–260.

30.

Nicolaisen

Frandsen

(2019) Zero impact: A large-scale study of uncitedness. Scientometrics 119(2): 1227–1254.

31.

Okubo

(1997) Bibliometric indicators and analysis of research systems: Methods and examples. Available at: https://www.oecd-ilibrary.org/docserver/208277770603.pdf?expires=1731491397&id=id&accname=guest&checksum=54A325C3A5BE2D68424C101F2517C004

32.

Onodera

Yoshikane

(2015) Factors affecting citation rates of research articles. Journal of the Association for Information Science and Technology 66(4): 739–764.

33.

Pendlebury

(1991) Letter to the editor. Science 251: 1410–1411.

34.

Peters

HPF

Van Raan

AFJ

(1994) On determinants of citation scores: A case study in chemical engineering. Journal of the American Society for Information Science 45(1): 39–49.

35.

Poomkottayil

Bornstein

Sendi

(2011) Lost in translation: The impact of publication language on citation frequency in the scientific dental literature. Swiss Medical Weekly 141(0304): w13148.

36.

Rousseau

(1992)) Why am I not cited, or why are multi-authored papers more cited than others. Journal of Documentation 48(1): 79–80.

37.

Stern

(1990) Uncitedness in the biomedical literature. Journal of the American Society for Information Science 41(3): 193–196.

38.

Sugimoto

Larivière

(2018) Measuring Research: What Everyone Needs to Know. Oxford: Oxford University Press.

39.

Thelwall

(2016) Are there too many uncited articles? Zero inflated variants of the discretized lognormal and hooked power law distributions. Journal of Informetrics 10(2): 622–633.

40.

Thelwall

Maflahi

(2020) Academic collaboration rates and citation associations vary substantially between countries and fields. Journal of the Association for Information Science and Technology 71(8): 968–978.

41.

Van Leeuwen

Moed

Tijssen

, et al. (2001) Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance. Scientometrics 51: 335–346.

42.

Van Raan

(2005) For your citations only? Hot topics in bibliometric analysis. Measurement: Interdisciplinary Research and Perspectives 3(1): 50–62.

43.

Van Raan

AFJ

(1998) The influence of international collaboration on the impact of research results: Some simple mathematical considerations concerning the role of self-citations. Scientometrics 42(3): 423–428.

44.

Wagner

Whetsell

Mukherjee

(2019) International research collaboration: Novelty, conventionality, and atypicality in knowledge recombination. Research Policy 48(5): 1260–1270.

45.

Wallace

Larivière

Gingras

(2009) Modeling a Century of Citation Distributions. Journal of Informetrics 3(4): 296–303.