Related papers: Coarse-graining and the Blackwell order
We study optimization problems in which a linear functional is maximized over probability measures that are dominated by a given measure according to an integral stochastic order in an arbitrary dimension. We show that the following four…
Coarse-graining techniques play a central role in reducing the complexity of stochastic models, and are typically characterised by a mapping which projects the full state of the system onto a smaller set of variables which captures the…
This paper quantifies the intuitive observation that adding noise reduces available information by means of non-linear strong data processing inequalities. Consider the random variables $W\to X\to Y$ forming a Markov chain, where $Y=X+Z$…
We consider the problem of coarse-graining in the context of finite-volume fluid models. If a variable is defined on a high-resolution grid it may be coarse-grained so that it is defined on a grid of lower resolution. In general this will…
Key to effective generic, or "black-box", variational inference is the selection of an approximation to the target density that balances accuracy and speed. Copula models are promising options, but calibration of the approximation can be…
Order is one of the main instruments to measure the relationship between objects in (empirical) data. However, compared to methods that use numerical properties of objects, the amount of ordinal methods developed is rather small. One reason…
Grokking is a phenomenon where a model trained on an algorithmic task first overfits but, then, after a large amount of additional training, undergoes a phase transition to generalize perfectly. We empirically study the internal structure…
We study an information design problem with continuous state and discrete signal space. Under convex and S-shaped value functions, the optimal information structure is interval-partitional and exhibits a dual expectations property: each…
Post-training is routinely evaluated through aggregate benchmark scores that treat multi-hop reasoning as a single capability -- as if a model that answers more questions correctly must be better at assembling facts. We show that this…
Quantum channel capacities give the fundamental performance limits for information flow over a communication channel. However, the prevalence of superadditivity is a major obstacle to understanding capacities, both quantitatively and…
Kendall transformation is a conversion of an ordered feature into a vector of pairwise order relations between individual values. This way, it preserves ranking of observations and represents it in a categorical form. Such transformation…
The current paper answers an open question of abs/1007.2426 We say that a countable model M characterizes an infinite cardinal kappa, if the Scott sentence of M has a model in cardinality kappa, but no models in cardinality kappa plus. If M…
Information theory is gaining popularity as a tool to characterize performance of biological systems. However, information is commonly quantified without reference to whether or how a system could extract and use it; as a result,…
One of the basic tenets in information theory, the data processing inequality states that output divergence does not exceed the input divergence for any channel. For channels without input constraints, various estimates on the amount of…
We study notions of generic and coarse computability in the context of computable structure theory. Our notions are stratified by the $\Sigma_\beta$ hierarchy. We focus on linear orderings. We show that at the $\Sigma_1$ level all linear…
Multivariate information decompositions hold promise to yield insight into complex systems, and stand out for their ability to identify synergistic phenomena. However, the adoption of these approaches has been hindered by there being…
We introduce a new type of adversary for online graph problems. The new adversary is parameterized by a single integer $\kappa$, which upper bounds the number of connected components that the adversary can use at any time during the…
A simple method to enhance the quality of communication is to send a carrier with its copies. Classical information theory says that information behaves quantitatively under copying. In other words, if a carrier is more informatic than…
Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. This has prompted the development of principled and effective data curation methods in recent…
Classically, no information can be transmitted through a depolarising, that is a completely noisy, channel. We show that by combining a depolarising channel with another channel in an indefinite causal order---that is, when there is…