Related papers: Robust benchmarking in noisy environments
In order to increase their robustness against environmental fluctuations, many biological populations have developed bet-hedging mechanisms in which the population `bets' against the presence of prolonged favorable environmental conditions…
This paper introduces a novel approach to quantifying ecological resilience in biological systems, particularly focusing on noisy systems responding to episodic disturbances with sudden adaptations. Incorporating concepts from…
Prescribed-time algorithms based on time-varying gains may have remarkable properties, such as regulation in a user-prescribed finite time that is the same for every nonzero initial condition and that holds even under matched disturbances.…
Ecosystems tend to fluctuate around stable equilibria in response to internal dynamics and environmental factors. Occasionally, they enter an unstable tipping region and collapse into an alternative stable state. Our understanding of how…
We study the effects of time-varying environmental noise on nonequilibrium phase transitions in spreading and growth processes. Using the examples of the logistic evolution equation as well as the contact process, we show that such temporal…
To overcome the limitations of automated metrics (e.g. BLEU, METEOR) for evaluating dialogue systems, researchers typically use human judgments to provide convergent evidence. While it has been demonstrated that human judgments can suffer…
Recent advancements in Large Language Models (LLMs) have significantly enhanced interactions between users and models. These advancements concurrently underscore the need for rigorous safety evaluations due to the manifestation of social…
Large Language Models (LLMs) have witnessed remarkable advancements in recent years, prompting the exploration of tool learning, which integrates LLMs with external tools to address diverse real-world challenges. Assessing the capability of…
Many robot control scenarios involve assessing system robustness against a task specification. If either the controller or environment are composed of "black-box" components with unknown dynamics, we cannot rely on formal verification to…
In this work, we present a benchmark that consists of Jupyter notebooks development trajectories and allows measuring how large language models (LLMs) can leverage runtime information for predicting code output and code generation. We…
As a scientific programming language, Julia strives for performance but also provides high-level productivity features. To avoid performance pathologies, Julia users are expected to adhere to a coding discipline that enables so-called type…
When training large language models (LLMs), it is common practice to track downstream task performance throughout the training process and select the checkpoint with the highest validation score. However, downstream metrics often exhibit…
This paper studies the classical problem of detecting the locations of signal occurrences in a one-dimensional noisy measurement. Assuming the signal occurrences do not overlap, we formulate the detection task as a constrained likelihood…
Existing benchmarks do not test language agents on their interaction with human users or ability to follow domain-specific rules, both of which are vital for deploying them in real world applications. We propose $\tau$-bench, a benchmark…
Increasing emphasis on data and quantitative methods in the biomedical sciences is making biological research more computational. Collecting, curating, processing, and analysing large genomic and imaging data sets poses major computational…
This paper examines synchronization of computer clocks connected via a data network and proposes a skewless algorithm to synchronize them. Unlike existing solutions, which either estimate and compensate the frequency difference (skew) among…
System-environment interactions are intrinsically nonlinear and dependent on the interplay between many degrees of freedom. The complexity may be even more pronounced when one aims to describe biologically motivated systems. In that case,…
BlockSystems.jl and NetworkDynamics.jl are two novel software packages which facilitate highly efficient transient stability simulations of power networks. Users may specify inputs and power system design in a convenient modular and…
Performance testing aims at uncovering efficiency issues of software systems. In order to be both effective and practical, the design of a performance test must achieve a reasonable trade-off between result quality and testing time. This…
In a standard classification framework a set of trustworthy learning data are employed to build a decision rule, with the final aim of classifying unlabelled units belonging to the test set. Therefore, unreliable labelled observations,…