Related papers: Independence Tests Without Ground Truth for Noisy …
Binary classification is widely used in ML production systems. Monitoring classifiers in a constrained event space is well known. However, real world production systems often lack the ground truth these methods require. Privacy concerns may…
In various situations one is given only the predictions of multiple classifiers over a large unlabeled test data. This scenario raises the following questions: Without any labeled data and without any a-priori knowledge about the…
The effectiveness of binary analysis tools and techniques is often measured with respect to how well they map to a ground truth. We have found that not all ground truths are created equal. This paper challenges the binary analysis community…
The evaluation of noisy binary classifiers on unlabeled data is treated as a streaming task: given a data sketch of the decisions by an ensemble, estimate the true prevalence of the labels as well as each classifier's accuracy on them. Two…
We consider the following problem: given the weights of two models, can we test whether they were trained independently -- i.e., from independent random initializations? We consider two settings: constrained and unconstrained. In the…
We consider the problem of whether a given decision model, working with structured data, has individual fairness. Following the work of Dwork, a model is individually biased (or unfair) if there is a pair of valid inputs which are close to…
Algebraic independence is an advanced notion in commutative algebra that generalizes independence of linear polynomials to higher degree. Polynomials {f_1, ..., f_m} \subset \F[x_1, ..., x_n] are called algebraically independent if there is…
We wish to test whether a real-valued variable $Z$ has explanatory power, in addition to a multivariate variable $X$, for a binary variable $Y$. Thus, we are interested in testing the hypothesis $\mathbb{P}(Y=1\, | \, X,Z)=\mathbb{P}(Y=1\,…
In hypothesis testing problems the property of strict unbiasedness describes whether a test is able to discriminate, in the sense of a difference in power, between any distribution in the null hypothesis space and any distribution in the…
Rule based classifiers that use the presence and absence of key sub-strings to make classification decisions have a natural mechanism for quantifying the uncertainty of their precision. For a binary classifier, the key insight is to treat…
A distributed binary hypothesis testing problem is studied with one observer and two decision centers. Achievable type-II error exponents are derived for testing against conditional independence when the observer communicates with the two…
In order to better understand the structure of classical rings of invariants for binary forms, Dixmier proposed, as a conjectural homogeneous system of parameters, an explicit collection of invariants previously studied by Hilbert. We…
Many binary classification problems minimize misclassification above (or below) a threshold. We show that instances of ranking problems, accuracy at the top or hypothesis testing may be written in this form. We propose a general framework…
Using ideas from automata theory we design a new efficient (deterministic) identity test for the \emph{noncommutative} polynomial identity testing problem (first introduced and studied in \cite{RS05,BW05}). We also apply this idea to the…
We study the problem of fair binary classification using the notion of Equal Opportunity. It requires the true positive rate to distribute equally across the sensitive groups. Within this setting we show that the fair optimal classifier is…
Binary classification is a task that involves the classification of data into one of two distinct classes. It is widely utilized in various fields. However, conventional classifiers tend to make overconfident predictions for data that…
An open problem in the theory of inverse semigroups was whether the variety of such semigroups, when viewed as algebras with a binary operation and a unary operation, is 2-based, that is, has a base for its identities consisting of 2…
In a broad range of classification and decision making problems, one is given the advice or predictions of several classifiers, of unknown reliability, over multiple questions or queries. This scenario is different from the standard…
Medical researchers have solved the problem of estimating the sensitivity and specificity of binary medical diagnostic tests without gold standard tests for comparison. That problem is the same as estimating confusion matrices for…
This paper analyzes the eigenvalue distribution of neutral differential systems and the corresponding difference systems, and establishes the relationship between the eigenvalue distribution and delay-independent stability of neutral…