Related papers: Decision Trees, Protocols, and the Fourier Entropy…
Changes in input distribution can induce shifts in the average predictions of machine learning models. Such prediction shifts may impact downstream business outcomes (e.g. a bank's loan approval rate), so understanding their causes can be…
A function $f:\mathbb{Z}_n \to \mathbb{C}$ can be represented as a linear combination $f(x)=\sum_{\alpha \in \mathbb{Z}_n}\widehat{f}(\alpha) \chi_{\alpha,n}(x)$ where $\widehat{f}$ is the (discrete) Fourier transform of $f$. Clearly, the…
Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that…
We consider a random field $\phi(\mathbf{r})$ in $d$ dimensions which is largely concentrated around small `hotspots', with `weights', $w_i$. These weights may have a very broad distribution, such that their mean does not exist, or else is…
In 1974, M. Shub stated Topological Entropy Conjecture, that is, the inequality $\log\rho\leq ent(f)$ is valid or not, where $f$ is a continuous self-map on a compact manifold $M$, $ent(f)$ is the topological entropy of $f$ and $\rho$ is…
We develop a categorical foundation for belief propagation on factor graphs. We construct the free hypergraph category \(\Syn_\Sigma\) on a typed signature and prove its universal property, yielding compositional semantics via a unique…
Automated story generation remains a difficult area of research because it lacks strong objective measures. Generated stories may be linguistically sound, but in many cases suffer poor narrative coherence required for a compelling,…
The entropy of network ensembles characterizes the amount of information encoded in the network structure, and can be used to quantify network complexity, and the relevance of given structural properties observed in real network datasets…
FeFETs hold strong potential for advancing memory and logic technologies, but their inherent randomness arising from both operational cycling and fabrication variability poses significant challenges for accurate and reliable modeling.…
Making decisions freely presupposes that there is some indeterminacy in the environment and in the decision making engine. The former is reflected on the behavioral changes due to communicating: few changes indicate rigid environments;…
The frequency response function (FRF) is an established way to describe the outcome of experiments in posture control literature. The FRF is an empirical transfer function between an input stimulus and the induced body segment sway profile,…
The estimation of categorical distributions under marginal constraints summarizing some sample from a population in the most-generalizable way is key for many machine-learning and data-driven approaches. We provide a parameter-agnostic…
Shannon's entropy power inequality (EPI) can be viewed as a statement of concavity of an entropic function of a continuous random variable under a scaled addition rule: $$f(\sqrt{a}\,X + \sqrt{1-a}\,Y) \ge a f(X) + (1-a) f(Y) \quad \forall…
A novel formalism, called H-theory, is applied to the problem of statistical equilibrium of a hierarchical complex system with multiple time and length scales. In this approach, the system is formally treated as being composed of a small…
In an attempt to show that the acceptance probability of a quantum query algorithm making $q$ queries can be well-approximated almost everywhere by a classical decision tree of depth $\leq \text{poly}(q)$, Aaronson and Ambainis proposed the…
This article provides a completion to theories of information based on entropy, resolving a longstanding question in its axiomatization as proposed by Shannon and pursued by Jaynes. We show that Shannon's entropy function has a…
The minimum error entropy (MEE) criterion has been successfully used in fields such as parameter estimation, system identification and the supervised machine learning. There is in general no explicit expression for the optimal MEE estimate…
The classical Shannon sampling theorem states that a signal f with Fourier transform F in L^2(R) having its support contained in (-\pi,\pi) can be recovered from the sequence of samples (f(n))_{n in Z} via f(t)=\sum_{n in Z} f(n) (sin(\pi…
Let $(X_1,\ldots,X_n)$ be an i.i.d. sequence of random variables in $\mathbb{R}^d$, $d\geq 1$. We show that, for any function $\varphi :\mathbb{R}^d\rightarrow\mathbb{R}$, under regularity conditions, \[n^…
From the output produced by a memoryless deletion channel from a uniformly random input of known length $n$, one obtains a posterior distribution on the channel input. The difference between the Shannon entropy of this distribution and that…