Related papers: The Ever-Evolving Science Exam
Causal inference is central to scientific discovery, yet choosing appropriate methods remains challenging because of the complexity of both statistical methodology and real-world data. Inspired by the success of artificial intelligence in…
When the world changes, so does the text that humans write about it. How do we build language models that can be easily updated to reflect these changes? One popular approach is retrieval-augmented generation, in which new documents are…
Scientific knowledge is constantly subject to a variety of changes due to new discoveries, alternative interpretations, and fresh perspectives. Understanding uncertainties associated with various stages of scientific inquiries is an…
The evidence-based approach has increasingly been employed to synthesize empirical findings from the primary research in software engineering. Nevertheless, the reproducibility of evidence-based software engineering (EBSE) studies seems to…
Open science represents a transformative research approach essential for enhancing sustainability and impact. Data generation encompasses various methods, from automated processes to human-driven inputs, creating a rich and diverse…
Context: Empirical Software Engineering (ESE) drives innovation in SE through qualitative and quantitative studies. However, concerns about the correct application of empirical methodologies have existed since the 2006 Dagstuhl seminar on…
The rapid growth of Artificial Intelligence and Machine Learning in scientific research has highlighted a gap between industry-standard MLOps tools and platforms, and the unique requirements of modern and Open Science, particularly…
Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the…
We briefly review some of the scientific challenges and epistemological issues related to climate science. We discuss the formulation and testing of theories and numerical models, which, given the presence of unavoidable uncertainties in…
Open Science aims to foster openness and collaboration in research, leading to more significant scientific and social impact. However, practicing Open Science comes with several challenges and is currently not properly rewarded. In this…
The flourish of web-based services gave birth to the research area \textit{services computing}, a rapidly-expanding academic community since nearly 20 years ago. Consensus has been reached on a set of representative research problems in…
Extracting key information from scientific papers has the potential to help researchers work more efficiently and accelerate the pace of scientific progress. Over the last few years, research on Scientific Information Extraction (SciIE)…
Current evaluation frameworks for foundation models rely heavily on static, manually curated benchmarks, limiting their ability to capture the full breadth of model capabilities. This paper introduces Active learning for Capability…
Despite persistent efforts in revealing the temporal patterns in scientific careers, little attention has been paid to the spatial patterns of scientific activities in the knowledge space. Here, drawing on millions of papers in six…
The rapid advancement of large language models (LLMs) is fundamentally reshaping software engineering (SE), driving a paradigm shift in both academic research and industrial practice. While top-tier SE venues continue to show sustained or…
The central challenge of AI for Science is not reasoning alone, but the ability to create computational methods in an open-ended scientific world. Existing LLM-based agents rely on static, pre-defined tool libraries, a paradigm that…
The Architecture, Engineering, and Construction (AEC) industry is undergoing rapid digital transformation, producing diverse digital assets such as datasets, computational models, use cases, and educational materials across the built…
Capitalizing on the enthusiasm about space science in the general public, our goal as an interdisciplinary group of scholars is to design and teach a new team-taught interdisciplinary course, "Philosophy and Science of Space Exploration…
Automatic scoring engines have been used for scoring approximately fifteen million test-takers in just the last three years. This number is increasing further due to COVID-19 and the associated automation of education and testing. Despite…
Understanding the earth's climate system and how it might be changing is a preeminent scientific challenge. Global climate models are used to simulate past, present, and future climates, and experiments are executed continuously on an array…