Related papers: Entropy, Computing and Rationality

In the paper the memory effect in the system consisting from a trajectory of process and an environment is considered. The environment is presented by scalar potential and noise. The evolution of system is interpreted as process of the…

Large language models achieve strong reasoning performance, yet existing decoding strategies either explore blindly (random sampling) or redundantly (independent multi-sampling). We propose Entropy-Tree, a tree-based decoding method that…

This work demonstrates how the concept of the entropic potential of events -- a parameter quantifying the influence of discrete events on the expected future entropy of a system -- can enhance uncertainty quantification, decision-making,…

The thermodynamic definition of entropy can be extended to nonequilibrium systems based on its relation to information. To apply this definition in practice requires access to the physical system's microstates, which may be prohibitively…

Entropy is a measure of self-information which is used to quantify losses. Entropy was developed in thermodynamics, but is also used to compare probabilities based on their deviating information content. Corresponding model uncertainty is…

Rational decision making in its linguistic description means making logical decisions. In essence, a rational agent optimally processes all relevant information to achieve its goal. Rationality has two elements and these are the use of…

Quantum information theory, particularly its entropic formulations, has made remarkable strides in characterizing quantum systems and tasks. However, a critical dimension remains underexplored: computational efficiency. While classical…

Automated grading systems can efficiently score short-answer responses, yet they often fail to indicate when a grading decision is uncertain or potentially contentious. We introduce semantic entropy, a measure of variability across multiple…

As one of the most widespread social dynamics, cooperative behavior is among the most fascinating collective phenomena. Several animal species, from social insects to human beings, feature social groups altruistically working for a common…

Finding better solutions to combinatorial optimization problems could have a large positive impact on many real-world application areas, such as logistics. For this reason, significant efforts have been made to design novel optimisation…

This paper aims to overcome a major obstacle in scaling RL for reasoning with LLMs, namely the collapse of policy entropy. Such phenomenon is consistently observed across vast RL runs without entropy intervention, where the policy entropy…

By 'informational entropy', we understand an inherent boundary to knowability, due e.g. to perceptual, theoretical, evidential or linguistic limits. In this paper, we discuss a logical framework in which this boundary is incorporated into…

Entropy is a concept that has traditionally been reliant on a definite notion of causality. However, without a definite notion of causality, the concept of entropy is not all lost. Indefinite causal structure results from combining…

The uncertainty principle brings out intrinsic quantum bounds on the precision of measuring non-commuting observables. Statistical outcomes in the measurement of incompatible observables reveal a trade-off on the sum of corresponding…

The uncertainty principle is one of the key concepts in quantum theory. This principle states that it is not possible to measure two incompatible observables simultaneously and accurately. In quantum information theory, the uncertainty…

Entropy is a central concept in physics, but can be challenging to calculate even for systems that are easily simulated. This is exacerbated out of equilibrium, where generally little is known about the distribution characterizing simulated…

In this thesis we aim to analyze and quantify the energetic and information contents that can be extracted from a dynamical system subject to the external environment. The latter is usually assumed to be deleterious for the feasibility of…

It has been demonstrated that excitable media with a tree structure performed better than other network topologies, it is natural to consider neural networks defined on Cayley trees. The investigation of a symbolic space called tree-shift…

The interrelationships of the fundamental biological processes natural selection, mutation, and stochastic drift are quantified by the entropy rate of Moran processes with mutation, measuring the long-run variation of a Markov process. The…

Large Reasoning Models (LRMs) often suffer from overthinking, generating unnecessarily long reasoning chains even for simple tasks. This leads to substantial computational overhead with limited performance gain, primarily due to redundant…