Related papers: KnowTD-An Actionable Knowledge Representation Syst…
Physics-informed neural network architectures have emerged as a powerful tool for developing flexible PDE solvers which easily assimilate data, but face challenges related to the PDE discretization underpinning them. By instead adapting a…
In a supervisory control system the human agent knowledge of past, current, and future system behavior is critical for system performance. Being able to reason about that knowledge in a precise and structured manner is central to effective…
In his Comment [1], Philip Strasberg (PS) argues from the analysis of different examples that the framework we have presented in [2] does not recover known results of macroscopic textbook thermodynamics. Here, we show that such apparent…
Three different approaches to derive mutual information via thermodynamics are presented where the temperature-dependent energy is given by: (a) $\beta \mathcal{E} = -\ln[P(X,Y)]$, (b) $\beta \mathcal{E} =-\ln[P(Y|X)]$ or (c) $\beta…
Knowledge distillation (KD) is a new method for transferring knowledge of a structure under training to another one. The typical application of KD is in the form of learning a small model (named as a student) by soft labels produced by a…
Thermodynamics relies on the possibility to describe systems composed of a large number of constituents in terms of few macroscopic variables. Its foundations are rooted into the paradigm of statistical mechanics, where thermal properties…
Temporal knowledge graphs (TKGs) represent time-stamped relational facts and support a wide range of reasoning tasks over evolving events. However, existing methods produce entity representations that are static at the entity level, in that…
A central idea of knowledge distillation is to expose relational structure embedded in the teacher's weights for the student to learn, which is often facilitated using a temperature parameter. Despite its widespread use, there remains…
Existing conversational models are handled by a database(DB) and API based systems. However, very often users' questions require information that cannot be handled by such systems. Nonetheless, answers to these questions are available in…
Knowledge distillation (KD) is a technique for transferring knowledge from complex teacher models to simpler student models, significantly enhancing model efficiency and accuracy. It has demonstrated substantial advancements in various…
The usual formulation of thermodynamics is based on the additivity of macroscopic systems. However, there are numerous examples of macroscopic systems that are not additive, due to the long-range character of the interaction among the…
Knowledge distillation (KD) is widely used for training a compact model with the supervision of another large model, which could effectively improve the performance. Previous methods mainly focus on two aspects: 1) training the student to…
We numerically show that a deep neural network (DNN) can learn macroscopic thermodynamic laws purely from microscopic data. Using molecular dynamics simulations, we generate the data of snapshot images of gas particles undergoing adiabatic…
The application of thermodynamic reasoning in the study of learning systems has a long tradition. Recently, new tools relating perfect thermodynamic adaptation to the adaptation process have been developed. These results, known as…
We introduce thermodynamic networks, a general framework for autonomous, physics-based computation using non-equilibrium steady states. These networks are modeled as a collection of finite-size reservoirs that exchange conserved…
In this study, we present a reformulation of classical equilibrium thermodynamics by replacing the obscure and ambiguous concept of entropy with the clear and intuitive concept of information stored in a thermodynamic system. Specifically,…
Knowledge tracing (KT) is a crucial technique to predict students' future performance by observing their historical learning processes. Due to the powerful representation ability of deep neural networks, remarkable progress has been made by…
An analytical characterization of thermodynamically rational agent behaviour is obtained for a simple, yet non--trivial example of a ``Maxwell's demon" operating with partial information. Our results provide the first fully transparent…
A thermodynamically motivated neural network model is described that self-organizes to transport charge associated with internal and external potentials while in contact with a thermal reservoir. The model integrates techniques for rapid,…
Knowledge distillation (KD) is one of the most potent ways for model compression. The key idea is to transfer the knowledge from a deep teacher model (T) to a shallower student (S). However, existing methods suffer from performance…