Related papers: KnowTD-An Actionable Knowledge Representation Syst…
Knowledge Transfer (KT) techniques tackle the problem of transferring the knowledge from a large and complex neural network into a smaller and faster one. However, existing KT methods are tailored towards classification tasks and they…
Information thermodynamics relates the rate of change of mutual information between two interacting subsystems to their thermodynamics when the joined system is described by a bipartite stochastic dynamics satisfying local detailed balance.…
Knowledge distillation is a technique to imitate a performance that a deep learning model has, but reduce the size on another model. It applies the outputs of a model to train another model having comparable accuracy. These two distinct…
Turing Machines (TMs) are the canonical model of computation in computer science and physics. We combine techniques from algorithmic information theory and stochastic thermodynamics to analyze the thermodynamic costs of TMs. We consider two…
In the theoretical modelling of a physical system a crucial step consists in the identification of those degrees of freedom that enable a synthetic, yet informative representation of it. While in some cases this selection can be carried out…
We introduce an algorithmic model of heat conduction, the thermodynamic graph. The thermodynamic graph is analogous to meshes in the finite difference method in the sense that the calculation of temperature is carried out at the vertices of…
Knowledge distillation aims at transferring knowledge acquired in one model (a teacher) to another model (a student) that is typically smaller. Previous approaches can be expressed as a form of training the student to mimic output…
Machine learning models, particularly the black-box models, are widely favored for their outstanding predictive capabilities. However, they often face scrutiny and criticism due to the lack of interpretability. Paradoxically, their strong…
Building on the fundamental equation, this study revisits key thermodynamic concepts in a cohesive and innovative manner. It demonstrates the consistency of thermodynamic theory while addressing and clarifying common misconceptions and…
We develop the laws of thermodynamics in terms of general exponential families. By casting learning (log-loss minimization) problems in max-entropy and statistical mechanics terms, we translate thermodynamics results to learning scenarios.…
Thermodynamics plays an important role both in the foundations of physics and in technological applications. An operational perspective adopted in recent years is to formulate it as a quantum resource theory. At the core of this theory is…
Improved understanding of molecular systems has only emphasised the sophistication of networks within the cell. Simultaneously, the advance of nucleic acid nanotechnology, a platform within which reactions can be exquisitely controlled, has…
The problem of the insensitivity of the macroscopic behavior of any thermodynamical system to partitioning generates a bias between the reproducibility of its macroscopic behavior viewed as the simplest form of causality and its long-term…
On the one hand, the dissipated heat of a thermodynamic work extraction process upper bounds the non-predictive information, which the associated system encodes about its environment. Thus, emergent information processing capabilities can…
In analogy to Brownian computers we explicitly show how to construct stochastic models, which mimic the behaviour of a general purpose computer (a Turing machine). Our models are discrete state systems obeying a Markovian master equation,…
Human cognition can leverage fundamental conceptual knowledge, like geometric and kinematic ones, to appropriately perceive, comprehend and interact with novel objects. Motivated by this finding, we aim to endow machine intelligence with an…
The framework of algorithmic knowledge assumes that agents use algorithms to compute the facts they explicitly know. In many cases of interest, a deductive system, rather than a particular algorithm, captures the formal reasoning used by…
We develop inductive biases for the machine learning of complex physical systems based on the port-Hamiltonian formalism. To satisfy by construction the principles of thermodynamics in the learned physics (conservation of energy,…
Information dynamics is an emerging description of information processing in complex systems which describes systems in terms of intrinsic computation, identifying computational primitives of information storage and transfer. In this paper…
Operational quantum stochastic thermodynamics is a recently proposed theory to study the thermodynamics of open systems based on the rigorous notion of a quantum stochastic process or quantum causal model. In there, a stochastic trajectory…