Related papers: Coarse-graining and the Blackwell order
Grover's algorithm is a primary algorithm offered as evidence that quantum computers can provide an advantage over classical computers. It involves an "oracle" specified for a given application whose structure is not part of the formal…
Solid State Drives (SSDs) are critical to datacenters, consumer platforms, and mission-critical systems. Yet diagnosing their performance and reliability is difficult because data are fragmented and time-disjoint, and existing methods…
Conventional theoretical machine learning studies generally assume explicitly or implicitly that there are enough or even infinitely supplied computational resources. In real practice, however, computational resources are usually limited,…
Parallel aggregation is a ubiquitous operation in data analytics that is expressed as GROUP BY in SQL, reduce in Hadoop, or segment in TensorFlow. Parallel aggregation starts with an optional local pre-aggregation step and then repartitions…
We revisit first-order optimization under local information constraints such as local privacy, gradient quantization, and computational constraints limiting access to a few coordinates of the gradient. In this setting, the optimization…
The partial information decomposition (PID) aims to quantify the amount of redundant information that a set of sources provides about a target. Here, we show that this goal can be formulated as a type of information bottleneck (IB) problem,…
We introduce a straightforward numerical coarse-graining scheme to estimate quantum states for a set of noisy measurement outcomes, which are difficult to calibrate, that is based solely on the measurement data collected from these…
Real-world data often follows a long-tailed distribution, which makes the performance of existing classification algorithms degrade heavily. A key issue is that samples in tail categories fail to depict their intra-class diversity. Humans…
We consider distributed optimization where the objective function is spread among different devices, each sending incremental model updates to a central server. To alleviate the communication bottleneck, recent work proposed various schemes…
There are currently a plurality of capacity theories of fading channels, including the ergodic capacity for fast fading channels and outage capacity for slow fading channels. However, analyses show that the outage capacity is a…
A common phenomena in modern recommendation systems is the use of feedback from one user to infer the `value' of an item to other users. This results in an exploration vs. exploitation trade-off, in which items of possibly low value have to…
Evaluating the quantum capacity of quantum channels is an important but difficult problem, even for channels of low input and output dimension. Smith and Smolin showed that the quantum capacity of the Clifford-twirl of a qubit amplitude…
There has been growing attention on fairness considerations recently, especially in the context of intelligent decision making systems. Explainable recommendation systems, in particular, may suffer from both explanation bias and performance…
Given black-box access to the input and output systems, we develop the first efficient quantum causal order discovery algorithm with polynomial query complexity with respect to the number of systems. We model the causal order with quantum…
In this paper, fading Gaussian multiuser channels are considered. If the channel is perfectly known to the transmitter, capacity has been established for many cases in which the channels may satisfy certain information theoretic orders such…
Knowledge distillation is a critical technique to transfer knowledge between models, typically from a large model (the teacher) to a more fine-grained one (the student). The objective function of knowledge distillation is typically the…
We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter…
Consider a channel with a given input alphabet size and a given input distribution. Our aim is to degrade or upgrade it to a channel with at most L output letters. The paper contains four main results. The first result, from which the paper…
Knowledge distillation deploys complex machine learning models in resource-constrained environments by training a smaller student model to emulate internal representations of a complex teacher model. However, the teacher's representations…
Downsampling produces coarsened, multi-resolution representations of data and it is used, for example, to produce lossy compression and visualization of large images, reduce computational costs, and boost deep neural representation…