Related papers: Entropy Formulas For Dynamical Systems With Mistak…
We propose a new way to measure the balance between freedom and coherence in a dynamical system and a new measure of its internal variability. Based on the concept of entropy and ideas from neuroscience and information theory, we define…
As access to high-quality, domain-specific data grows increasingly scarce, multi-epoch training has become a practical strategy for adapting large language models (LLMs). However, autoregressive models often suffer from performance…
We study the Bowen topological entropy of generic and irregular points for certain dynamical systems. We define the topological entropy of noncompact sets for flows, analogous to Bowen's definition. We show that this entropy coincides with…
We consider suspension semiflows of an angle multiplying map on the circle and study the distributions of periods of their periodic orbits. Under generic conditions on the roof function, we give an asymptotic formula on the number $\pi(T)$…
The efficiency of Monte Carlo samplers is dictated not only by energetic effects, such as large barriers, but also by entropic effects that are due to the sheer volume that is sampled. The latter effects appear in the form of an entropic…
We describe an approach to improving model fitting and model generalization that considers the entropy of distributions of modelling residuals. We use simple simulations to demonstrate the observational signatures of overfitting on ordered…
We obtain large deviation results for non-uniformly expanding maps with non-flat singularities or criticalities and for partially hyperbolic non-uniformly expanding attracting sets. That is, given a continuous function we consider its space…
Maximum entropy models provide the least constrained probability distributions that reproduce statistical properties of experimental datasets. In this work we characterize the learning dynamics that maximizes the log-likelihood in the case…
Recurrence rate, determinism, average line length, and entropy of line lengths are measures of complexity in recurrence quantification analysis, that help to understand the structure, predictability and complexity of dynamical systems. In…
One of the most influential results in neural network theory is the universal approximation theorem [1, 2, 3] which states that continuous functions can be approximated to within arbitrary accuracy by single-hidden-layer feedforward neural…
In this paper, we extend the concept of generalized entropy to uniform spaces, allowing computations beyond metrizable settings. We apply this to parabolic dynamics - systems with a unique fixed point uniformly attracting all compact…
We describe a framework in which is possible to develop and implement algorithms for the approximation of invariant measures of dynamical systems with a given bound on the error of the approximation. Our approach is based on a general…
The entropy production is one of the most essential features for systems operating out of equilibrium. The formulation for discrete-state systems goes back to the celebrated Schnakenberg's work and hitherto can be carried out when for each…
We investigate the role of entropic concepts for the relaxation dynamics in granular systems. In these systems the existence of a geometrical frustration induces a drastic modification of the allowed phase space, which in its turn induces a…
We study nontrivial entropy invariants in the class of parabolic flows on homogeneous spaces, quasi-unipotent flows. We show that topological complexity (ie, slow entropy) can be computed directly from the Jordan block structure of the…
We consider shift spaces in which elements of the alphabet may overlap nontransitively. We define a notion of entropy for such spaces, give several techniques for computing lower bounds for it, and show that it is equal to a limit of…
We analyze phase transitions in the conditional entropy of a sequence caused by a change in the conditional variables. Such transitions happen, for example, when training to learn the parameters of a system, since the transition from the…
Non-reciprocal interactions are present in many systems out of equilibrium. The rate of entropy production is a measure that quantifies the time irreversibility of a system, and thus how far it is from equilibrium. In this work, we…
Many learning algorithms can be represented as Markov processes, and understanding their generalization error is a central topic in learning theory. For specific continuous-time noisy algorithms, a prominent analysis technique relies on…
We show that under local detailed balance the expected entropy production rate is always bounded in terms of the dynamical activity. The activity refers to the time-symmetric contribution in the action functional for path-space…