相关论文: Constant conditional entropy and related hypothese…

In this paper, we study VC-density over indiscernible sequences (denoted VC_ind-density). We answer an open question in [1], showing that VC_ind-density is always integer valued. We also show that VC_ind-density and dp-rank coincide in the…

Information theory establishes the fundamental limits on data transmission, storage, and processing. Quantum information theory unites information theoretic ideas with an accurate quantum-mechanical description of reality to give a more…

During a spontaneous change, a macroscopic physical system will evolve towards a macro-state with more realizations. This observation is at the basis of the Statistical Mechanical version of the Second Law of Thermodynamics, and it provides…

The principle of entropy increase is not only the basis of statistical mechanics, but also closely related to the irreversibility of time, the origin of life, chaos and turbulence. In this paper, we first discuss the dynamic system…

Entanglement entropies of two-dimensional gapped ground states are expected to satisfy an area law, with a constant correction term known as the topological entanglement entropy (TEE). In many models, the TEE takes a universal value that…

These lectures deal with the problem of inductive inference, that is, the problem of reasoning under conditions of incomplete information. Is there a general method for handling uncertainty? Or, at least, are there rules that could in…

We define a notion of entropy for an infinite family $\mathcal{C}$ of measurable sets in a probability space. We show that the mean ergodic theorem holds uniformly for $\mathcal{C}$ under every ergodic transformation if and only if…

The uncertainty principle sets a bound on our ability to predict the measurement outcomes of two incompatible observables which are measured on a quantum particle simultaneously. In quantum information theory, the uncertainty principle can…

Evaluating whether large language models (LLMs) capture the structure of natural language beyond local fluency remains an open challenge. Existing evaluation methods, largely based on task performance or short-context behavior, provide…

The dependence with text length of the statistical properties of word occurrences has long been considered a severe limitation quantitative linguistics. We propose a simple scaling form for the distribution of absolute word frequencies…

An important body of quantitative linguistics is constituted by a series of statistical laws about language usage. Despite the importance of these linguistic laws, some of them are poorly formulated, and, more importantly, there is no…

Many quantum chemical similarity measures have been derived and substantiated by applying concepts and quantities from information theory to the electron density. To justify the use of information theory, the electron density is usually…

For a family of stochastic differential equations, we investigate the asymptotic behaviors of its corresponding Picard's iteration, establishing convergence results in terms of relative entropy. Our convergence results complement the…

The uncertainty principle sets limit on our ability to predict the values of two incompatible observables measured on a quantum particle simultaneously. This principle can be stated in various forms. In quantum information theory, it is…

The entropy rate of printed English is famously estimated to be about one bit per character, a benchmark that modern large language models (LLMs) have only recently approached. This entropy rate implies that English contains nearly 80…

In this article, I investigate the use of Bayesian updating rules applied to modeling social agents in the case of continuos opinions models. Given another agent statement about the continuous value of a variable $x$, we will see that…

The speed with which a learning algorithm converges as it is presented with more data is a central problem in machine learning --- a fast rate of convergence means less data is needed for the same level of performance. The pursuit of fast…

We propose an effect called information constraint which is characterized by the existence of different decay rates of signal strengths propagating along opposite directions. It is an intrinsic property of a type of open quantum system,…

Oliinychenko, Bugaev and Sorin [arXiv:1204.0103 [hep-ph]] considered the role of conservation laws in discussing possible weaknesses of thermal models which are utilized in describing the hadron multiplicities measured in central…

Information estimates such as the ``direct method'' of Strong et al. (1998) sidestep the difficult problem of estimating the joint distribution of response and stimulus by instead estimating the difference between the marginal and…