Related papers: A Possible Role for Entropy in Creative Cognition
We propose to examine the predictability and the complexity characteristics of the Standard&Poor500 dynamics behaviors in a coarse-grained way using the symbolic dynamics method and under the prism of the Information theory through the…
An effective way to scale up test-time compute of large language models is to sample multiple responses and then select the best one, as in Grok Heavy and Gemini Deep Think. Existing selection methods often rely on external reward models,…
How do we imagine visual objects and combine them to create new forms? To answer this question, we need to explore the cognitive, computational and neural mechanisms underlying imagery and creativity. The body of research on deep learning…
What is information originating in observation? Until now it has no scientifically conclusive definition. Information is memorized entropy cutting in random observations which processing interactions. Randomness of various interactive…
Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide…
Enormous questions still loom for the emerging science of spontaneous thought: what, exactly, is spontaneous thought? Why does our brain engage in spontaneous forms of thinking, and when is this most likely to occur? And perhaps the…
Maximum Entropy models can be inferred from large data-sets to uncover how collective dynamics emerge from local interactions. Here, such models are employed to investigate neurons recorded by multielectrode arrays in the human and monkey…
The entropy production of an open system coupled to a reservoir initialized in a canonical state can be expressed as a sum of two microscopic information-theoretic contributions: the system-bath mutual information and the relative entropy…
Societal complexity may be at a historical peak. Distinct from entropy, complexity tends to rise as systems move away from order, crest at an intermediate state, and decline as entropy continues increasing. The use of a thermodynamic…
Both entropy-minimizing and entropy-maximizing (curiosity) objectives for unsupervised reinforcement learning (RL) have been shown to be effective in different environments, depending on the environment's level of natural entropy. However,…
It is widely appreciated that well-balanced excitation and inhibition are necessary for proper function in neural networks. However, in principle, such balance could be achieved by many possible configurations of excitatory and inhibitory…
In order to become aware of inconsistencies, one must first construe of the world in a way that reflects its consistencies. This paper begins with a tentative model for how a set of discrete memories transforms into an interconnected…
Immersion in a task is a prerequisite for creativity. However, excessive arousal in a single task has drawbacks, such as overlooking events outside of the task. To examine such a negative aspect, this study constructs a computational model…
Consciousness is supported by complex patterns of brain activity which are indicative of irreversible non-equilibrium dynamics. While the framework of stochastic thermodynamics has facilitated the understanding of physical systems of this…
Encouraging exploration is a critical issue in deep reinforcement learning. We investigate the effect of initial entropy that significantly influences the exploration, especially at the earlier stage. Our main observations are as follows:…
Do reasoning models have "Aha!" moments? Prior work suggests that models like DeepSeek-R1-Zero undergo sudden mid-trace realizations that lead to accurate outputs, implying an intrinsic capacity for self-correction. Yet, it remains unclear…
The rapid integration of digital technology into daily life has prompted sustained concern regarding its impact on human cognition. This integrative review synthesizes documented risks and negative associations across more than 500…
This position paper proposes a fundamental shift in designing code generation models: treating reasoning depth as a controllable resource. Rather than being an incidental byproduct of prompting, we argue that the trade-off between rapid,…
Simultaneously recorded neurons exhibit correlations whose underlying causes are not known. Here, we use a population of threshold neurons receiving correlated inputs to model neural population recordings. We show analytically that small…
One among the most intriguing results coming from the application of statistical mechanics to the study of brain is the understanding that it, as a dynamical system, is inherently out of equilibrium. In the realm of non-equilibrium…