Related papers: Algorithmic information and simplicity in statisti…
We propose a coefficient of conditional dependence between two random variables $Y$ and $Z$ given a set of other variables $X_1,\ldots,X_p$, based on an i.i.d. sample. The coefficient has a long list of desirable properties, the most…
Symmetry shares an entwined history with the structure of physical theory. We propose a consequence of symmetry towards the axiomatic derivation of Hilbert space quantum theory. We introduce the notion of information symmetry (IS) and show…
Building upon the work of Chebyshev, Shannon and Kontoyiannis, it may be demonstrated that Chebyshev's asymptotic result: \begin{equation} \ln N \sim \sum_{p \leq N} \frac{1}{p} \cdot \ln p \end{equation} has a natural information-theoretic…
We propose a new interpretation of objective deterministic chances in statistical physics based on physical computational complexity. This notion applies to a single physical system (be it an experimental set--up in the lab, or a subsystem…
From a dynamical viewpoint, basic phase transitions of statistical mechanics can be regarded as a breaking of ergodicity. While many random models exhibiting such transitions at the thermodynamics limit exist, finite-dimensional examples…
The concept of information has emerged as a language in its own right, bridging several disciplines that analyze natural phenomena and man-made systems. Integrated information has been introduced as a metric to quantify the amount of…
In Part I, we defined a LASSO condition number and developed an algorithm -- for computing support sets (feature selection) of the LASSO minimisation problem -- that runs in polynomial time in the number of variables and the logarithm of…
Factorizing low-rank matrices has many applications in machine learning and statistics. For probabilistic models in the Bayes optimal setting, a general expression for the mutual information has been proposed using heuristic statistical…
As inductive inference and machine learning methods in computer science see continued success, researchers are aiming to describe ever more complex probabilistic models and inference algorithms. It is natural to ask whether there is a…
We study stochastic approximation algorithms with Markovian noise and constant step-size $\alpha$. We develop a method based on infinitesimal generator comparisons to study the bias of the algorithm, which is the expected difference between…
We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability $1- {\alpha}$. Although there exist numerous algorithms for this problem, it remains theoretically unexplored how the…
We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…
We investigate quantum information masking for arbitrary dimensional quantum states. We show that mutually orthogonal quantum states can always be served for deterministic masking of quantum information. We further construct a probabilistic…
The debate on the nature of quantum probabilities in relation to Quantum Non Locality has elevated Quantum Mechanics to the level of an "Operational Epistemic Theory". In such context the quantum superposition principle has an extraneous…
The statistical mechanics of particles that populate indistinguishable energy sub-states is explored. In particular, the mathematical treatment of the microstates differs from conventional statistical mechanics where for a given degeneracy,…
In the course of computer modeling of the most probable stationary macrostates of non-ergodic closed systems, a forecast was obtained about the existence of limits of applicability of the basic axiomatic postulate of statistical physics,…
We consider the task of estimating a high-dimensional directed acyclic graph, given observations from a linear structural equation model with arbitrary noise distribution. By exploiting properties of common random graphs, we develop a new…
Statistical sufficiency formalizes the notion of data reduction. In the decision theoretic interpretation, once a model is chosen all inferences should be based on a sufficient statistic. However, suppose we start with a set of procedures…
The statistical physics approach to the number partioning problem, a classical NP-hard problem, is both simple and rewarding. Very basic notions and methods from statistical mechanics are enough to obtain analytical results for the phase…
A new measure of information in quantum mechanics is proposed which takes into account that for quantum systems the only feature known before an experiment is performed are the probabilities for various events to occur. The sum of the…