相关论文: A nonmanipulable test

Constructive election control considers the problem of an adversary who seeks to sway the outcome of an electoral process in order to ensure that their favored candidate wins. We consider the computational problem of constructive election…

The workshop is devoted to model-based testing of both software and hardware. Model-based testing uses models describing the required behavior of the system under consideration to guide such efforts as test selection and test results…

Conformalized multiple testing offers a model-free way to control predictive uncertainty in decision-making. Existing methods typically use only part of the available data to build score functions tailored to specific settings. We propose a…

Testing the validity of claims made by self-proclaimed experts can be impossible when testing them in isolation, even with infinite observations at the disposal of the tester. However, in a multiple expert setting it is possible to design a…

The validity of instrumental variable (IV) designs is typically tested using two types of falsification tests. We characterize these tests as conditional independence tests between negative control variables -- proxies for unobserved…

In a well-calibrated risk prediction model, the average predicted probability is close to the true event rate for any given subgroup. Such models are reliable across heterogeneous populations and satisfy strong notions of algorithmic…

A joint characterisation of the controllability and observability of a particular kind of discrete system has been developed. The key idea of the procedure can be reduced to a correct choice of the sampling sequence. This freedom, owing to…

In contrast with software-generated randomness (called pseudo-randomness), quantum randomness is provable incomputable, i.e.\ it is not exactly reproducible by any algorithm. We provide experimental evidence of incomputability --- an…

A statistical test based on the geometric mean is proposed to determine if a predictive model should be rejected or not, when the quantity of interest is a strictly positive continuous random variable. A simulation study is performed to…

There is a strong consensus that combining the versatility of machine learning with the assurances given by formal verification is highly desirable. It is much less clear what verified machine learning should mean exactly. We consider this…

In adaptive-sampling control, the control frequency can be adjusted during task execution. Ensuring that these changes do not jeopardize the safety of the system being controlled requires attention. We introduce robust M-step hold model…

We propose a rigorous decomposition of predictive error, highlighting that not all 'irreducible' error is genuinely immutable. Many domains stand to benefit from iterative enhancements in measurement, construct validity, and modeling. Our…

Given observations from a stationary time series, permutation tests allow one to construct exactly level $\alpha$ tests under the null hypothesis of an i.i.d. (or, more generally, exchangeable) distribution. On the other hand, when the null…

A central paradigm behind process semantics based on observability and testing is that the exact moment of occurring of an internal nondeterministic choice is unobservable. It is natural, therefore, for this property to hold when the…

Consider a case-control study in which we have a random sample, constructed in such a way that the proportion of cases in our sample is different from that in the general population---for instance, the sample is constructed to achieve a…

Surrogate data testing is a method frequently applied to evaluate the results of nonlinear time series analysis. Since the null hypothesis tested against is a linear, gaussian, stationary stochastic process a positive outcome may not only…

Scalable oversight protocols aim to empower evaluators to accurately verify AI models more capable than themselves. However, human evaluators are subject to biases that can lead to systematic errors. We conduct two studies examining the…

The usual advantages put forward for including nullability declarations in the type systems of programming languages are that they improve program reliability or performance. But there is another, entirely different, reason for doing so. In…

Automated testing tools typically create test cases that are different from what human testers create. This often makes the tools less effective, the created tests harder to understand, and thus results in tools providing less support to…

In safety-critical applications a probabilistic model is usually required to be calibrated, i.e., to capture the uncertainty of its predictions accurately. In multi-class classification, calibration of the most confident predictions only is…