Go Wide, Go Deep: Quantifying the Impact of Scientific Papers through Influence Dispersion Trees

by   Dattatreya Mohapatra, et al.

Despite a long history of use of citation count as a measure to assess the impact or influence of a scientific paper, the evolution of follow-up work inspired by the paper and their interactions through citation links have rarely been explored to quantify how the paper enriches the depth and breadth of a research field. We propose a novel data structure, called Influence Dispersion Tree (IDT) to model the organization of follow-up papers and their dependencies through citations. We also propose the notion of an ideal IDT for every paper and show that an ideal (highly influential) paper should increase the knowledge of a field vertically and horizontally. Upon suitably exploring the structural properties of IDT, we derive a suite of metrics, namely Influence Dispersion Index (IDI), Normalized Influence Divergence (NID) to quantify the influence of a paper. Our theoretical analysis shows that an ideal IDT configuration should have equal depth and breadth (and thus minimize the NID value). We establish the superiority of NID as a better influence measure in two experimental settings. First, on a large real-world bibliographic dataset, we show that NID outperforms raw citation count as an early predictor of the number of new citations a paper will receive within a certain period after publication. Second, we show that NID is superior to the raw citation count at identifying the papers recognized as highly influential through Test of Time Award among all their contemporary papers (published in the same venue). We conclude that in order to quantify the influence of a paper, along with the total citation count, one should also consider how the citing papers are organized among themselves to better understand the influence of a paper on the research field. For reproducibility, the code and datasets used in this study are being made available to the community.


page 1

page 2

page 3

page 4


Citation Analysis of Computer Systems Papers

Citation analysis is used extensively in the bibliometrics literature to...

Predicting Long-Term Citations from Short-Term Linguistic Influence

A standard measure of the influence of a research paper is the number of...

Code Replicability in Computer Graphics

Being able to duplicate published research results is an important proce...

NNCP: A citation count prediction methodology based on deep neural network learning techniques

With the growing number of published scientific papers world-wide, the n...

Unbiased evaluation of ranking metrics reveals consistent performance in science and technology citation data

Despite the increasing use of citation-based metrics for research evalua...

Geographic Citation Gaps in NLP Research

In a fair world, people have equitable opportunities to education, to co...

Measuring Influence in Science: Standing on the Shoulders of Which Giants?

I study the measurement of the influence of scientists based on bibliogr...

Please sign up or login with your details

Forgot password? Click here to reset