相关论文: Introducing multiverse analysis to bibliometrics: …

Through case studies, we demonstrate how multiverse analysis can strengthen the robustness and transparency of computational social science findings against alternative methodological decisions. We conduct multiverse analyses of three…

Amid mounting concern about the reliability and credibility of machine learning research, we present a principled framework for making robust and generalizable claims: the multiverse analysis. Our framework builds upon the multiverse…

This article introduces the Multidimensional Research Assessment Matrix of scientific output. Its base notion holds that the choice of metrics to be applied in a research assessment process depends upon the unit of assessment, the research…

As science transitions from the age of lone geniuses to an era of collaborative teams, the question of whether large teams can sustain the creativity of individuals and continue driving innovation has become increasingly important. Our…

Following Funk and Owen-Smith (2017), Wu et al. (2019) proposed the disruption index (DI1) as a bibliometric indicator that measures disruptive and consolidating research. When we summarized the literature on the disruption index for our…

Multiverse analysis, a paradigm for statistical analysis that considers all combinations of reasonable analysis choices in parallel, promises to improve transparency and reproducibility. Although recent tools help analysts specify…

A multiverse analysis evaluates all combinations of "reasonable" analytic decisions to promote robustness and transparency, but can lead to a combinatorial explosion of analyses to compute. Long delays before assessing results prevent users…

The influence of gender diversity on the success of scientific teams is of great interest to academia. However, prior findings remain inconsistent, and most studies operationalize diversity in aggregate terms, overlooking internal role…

A vast number of systems across the world use algorithmic decision making (ADM) to (partially) automate decisions that have previously been made by humans. The downstream effects of ADM systems critically depend on the decisions made during…

As machine learning models become increasingly prevalent in critical decision-making models and systems in fields like finance, healthcare, etc., ensuring their robustness against adversarial attacks and changes in the input data is…

This paper shows how bibliometric assessment can be implemented at individual level. This has been successfully done at the University of Vienna carried out by the Bibliometrics and Publication Strategies Department of the Vienna University…

The rapid development of machine learning (ML) and artificial intelligence (AI) applications requires the training of large numbers of models. This growing demand highlights the importance of training models without human supervision, while…

Incorporating fresh members in teams is considered a pathway to team creativity. However, whether freshness improves team performance or not remains unclear, as well as the optimal involvement of fresh members for team performance. This…

Marketing scholars have underscored the importance of conceptual articles in providing theoretical foundations and new perspectives to the field. This paper supports the argument by employing two network-based measures -- the number of…

The potential occurrence of variable returns to size in research activity is a factor to be considered in choices about the size of research organizations and also in the planning of national research assessment exercises, so as to avoid…

Understanding the relationship between the composition of a research team and the potential impact of their research papers is crucial as it can steer the development of new science policies for improving the research enterprise. Numerous…

When language models answer open-ended problems, they implicitly make hidden decisions that shape their outputs, leaving users with uncontextualized answers rather than a working map of the problem; drawing on multiverse analysis from…

When building statistical models for Bayesian data analysis tasks, required and optional iterative adjustments and different modelling choices can give rise to numerous candidate models. In particular, checks and evaluations throughout the…

We present a framework consisting of three approaches that can enhance meta-analyses: 1) scoping reviews (evidence map), 2) bibliometrics, and 3) alternative impact metrics. These three "enrichment" approaches facilitate the research…

As multimodal learning finds applications in a wide variety of high-stakes societal tasks, investigating their robustness becomes important. Existing work has focused on understanding the robustness of vision-and-language models to…