Content-based filtering yields the highest CTR with 0.118%, while stereotype recommendations yield the highest ACCS (1.28). While CTR and CSR do not indicate choice overload, ACCS and TTFC point toward 5–6 recommendations as being optimal for Sowiport. Similarly, TTFC increases with larger recommendation set size, but increases more slowly for sets of more than five recommendations. When displaying 15 recommendations, the average clicks per set is at a maximum (1.15). ACCS increases with set size, but increases more slowly for six recommendations and more. Most recommendation sets only receive one click. We found that with increasing recommendation set size, i.e., the numbers of displayed recommendations, CTR decreases from 0.41% for one recommendation to 0.09% for 15 recommendations. These metrics help us to analyze choice overload and can yield evidence for finding the ideal number of recommendations to display. We use four metrics to analyze 41.3 million delivered recommendations: click-through rate (CTR), percentage of clicked recommendation sets (clicked set rate, CSR), average clicks per clicked recommendation set (ACCS), and time to first click (TTFC), which is the time between delivery of a set of recommendations to the first click. We then conducted our experimental evaluation through GESIS’ digital library Sowiport, with recommendations delivered by recommendations-as-a-service provider Mr. Of these libraries, the majority (74%) displays 3–5 related articles 28% of them display 6–10 related articles and no digital library displayed more than ten related-article recommendations. We first analyzed existing digital libraries, and found that only 30% of digital libraries show related-article recommendations to their users. We examine choice overload when displaying related-article recommendations in digital libraries, and examine the effectiveness of recommendation algorithms in this domain. The proposed citation function-based citation recommendation can generate intuitive explanations of the results for users, improving the transparency, persuasiveness, and effectiveness of recommender systems.Ĭhoice overload describes a situation in which a person has difficulty in making decisions due to too many options. The citation recommendation strategy can be helpful for valuable citation discovery, semantic scientific retrieval, and automatic literature review generation. More recent deep learning models should be performed to future validate the proposed approach. The dataset is insufficient due to the complexity of annotating citation functions for paragraphs in the related studies section. The experiments show that the term function-based methods outperform the baseline methods regarding the recall, precision, and F1-score measurement, demonstrating that term functions are useful in identifying valuable citations. Then the term function information is applied to enhance the performance. BM25 and Word2vec with VSM are implemented as the baseline models for the recommendation.

Using these term functions as labels, we annotate 531 research papers in three topics to evaluate our proposed recommendation strategy. We present nine term functions with three newly created and six identified from existing literature. This study focuses on citation recommendation in the related studies section by applying the term function of a citation context, potentially improving the efficiency of writing a literature review. (2) The literature review is not conducted extensively, which leads to working on a research problem that others have well addressed. Researchers frequently encounter the following problems when writing scientific articles: (1) Selecting appropriate citations to support the research idea is challenging.

Docear download