Related papers: Coarse-graining and the Blackwell order
As artificial intelligence (AI) systems are increasingly used in ethically sensitive domains such as education, healthcare, and transportation, balancing accuracy and interpretability has become a central concern. Coarse ethics (CE)…
The data processing inequality is the most basic requirement for any meaningful measure of information. It essentially states that distinguishability measures between states decrease if we apply a quantum channel and is the centerpiece of…
We introduce an information order on experiments based on weighted garbling, a generalization of the standard notion of garbling. In this order, an experiment is more informative than another if the latter is a weighted garbling of the…
According to Blackwell's Theorem it is equivalent to compare channels by either a garbling order or optimal decision making. This equivalence does not hold anymore if also allowing pre-garbling, i. e. for the so-called Shannon-order (see…
A thermodynamic formalism describing the efficiency of information learning is proposed, which is applicable for stochastic thermodynamic systems with multiple internal degree of freedom. The learning rate, entropy production rate (EPR),…
I establish a translation invariance property of the Blackwell order over experiments, show that garbling experiments bring them closer together, and use these facts to define a cardinal measure of informativeness. Experiment $A$ is…
Partial information decomposition has recently found applications in biological signal processing and machine learning. Despite its impacts, the decomposition was introduced through an informal and heuristic route, and its exact operational…
The ability to control complex networks is of crucial importance across a wide range of applications in natural and engineering sciences. However, issues of both theoretical and numerical nature introduce fundamental limitations to…
We first define the coarse-graining of probability measures in terms of stochastic kernels. We define when a probability measure is part of another probability measure and say that two probability measures coexist if they are both parts of…
We survey information-theoretic approaches to the reduction of Markov chains. Our survey is structured in two parts: The first part considers Markov chain coarse graining, which focuses on projecting the Markov chain to a process on a…
We investigate the information complexity of mixed-integer convex optimization under different types of oracles. We establish new lower bounds for the standard first-order oracle, improving upon the previous best known lower bound. This…
Learning can be seen as approximating an unknown function by interpolating the training data. Kriging offers a solution to this problem based on the prior specification of a kernel. We explore a numerical approximation approach to kernel…
Answering one of the main questions of [FHK14, Chapter 7], we show that there is a tight connection between the depth of a classifiable shallow theory $T$ and the Borel rank of the isomorphism relation $\cong^\kappa_T$ on its models of size…
Can complex engineered and biological networks be coarse-grained into smaller and more understandable versions in which each node represents an entire pattern in the original network? To address this, we define coarse-graining units (CGU)…
The nature of coarse graining is intuitively "obvious", but it is rather difficult to find explicit and calculable models of the coarse graining process (and the resulting entropy flow) discussed in the literature. What we would like to…
We propose a data-driven, coarse-graining formulation in the context of equilibrium statistical mechanics. In contrast to existing techniques which are based on a fine-to-coarse map, we adopt the opposite strategy by prescribing a…
In previous work, an ordering result was given for the symbolwise probability of error using general Markov channels, under iterative decoding of LDPC codes. In this paper, the ordering result is extended to mutual information, under the…
A fundamental question in information theory is to quantify the loss of information under a noisy channel. Partial orders and contraction coefficients are typical tools to that end, however, they are often also challenging to evaluate. For…
We present a loss function for neural networks that encompasses an idea of trivial versus non-trivial predictions, such that the network jointly determines its own prediction goals and learns to satisfy them. This permits the network to…
Information-theoretical quantities such as statistical distinguishability typically result from optimisations over all conceivable observables. Physical theories, however, are not generally considered valid for all mathematically allowed…