Related papers: Stochastic Thermodynamics of Learning
This work proposes to model the space environment as a stochastic dynamic network where each node is a group of objects of a given class, or species, and their relationship is represented by stochastic links. A set of stochastic dynamic…
We study nonequilibrium thermodynamics of complex information flows induced by interactions between multiple fluctuating systems. Characterizing nonequilibrium dynamics by causal networks (i.e., Bayesian networks), we obtain novel…
Equations of State model relations between thermodynamic variables and are ubiquitous in scientific modelling, appearing in modern day applications ranging from Astrophysics to Climate Science. The three desired properties of a general…
We study a distributed learning process observed in human groups and other social animals. This learning process appears in settings in which each individual in a group is trying to decide over time, in a distributed manner, which option to…
We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule including passive forgetting and different time scales for neuronal activity and learning…
This tutorial investigates the convergence of statistical mechanics and learning theory, elucidating the potential enhancements in machine learning methodologies through the integration of foundational principles from physics. The tutorial…
Spiking neural network is a type of artificial neural network in which neurons communicate between each other with spikes. Spikes are identical Boolean events characterized by the time of their arrival. A spiking neuron has internal…
Recent experimental and theoretical studies show that energy efficiency, which measures the amount of information processed by a neuron with per unit of energy consumption, plays an important role in the evolution of neural systems. Here,…
Fluctuations in biochemical networks, e.g., in a living cell, have a complex origin that precludes a description of such systems in terms of bipartite or multipartite processes, as is usually done in the framework of stochastic and/or…
We develop inductive biases for the machine learning of complex physical systems based on the port-Hamiltonian formalism. To satisfy by construction the principles of thermodynamics in the learned physics (conservation of energy,…
The dynamics of noise-resilient Boolean networks with majority functions and diverse topologies is investigated. A wide class of possible topological configurations is parametrized as a stochastic blockmodel. For this class of networks, the…
We define a neural network as a septuple consisting of (1) a state vector, (2) an input projection, (3) an output projection, (4) a weight matrix, (5) a bias vector, (6) an activation map and (7) a loss function. We argue that the loss…
Neural dynamics of energy-based models are governed by energy minimization and the patterns stored in the network are retrieved when the system reaches equilibrium. However, when the system is driven by time-varying external input, the…
Many recent generative models make use of neural networks to transform the probability distribution of a simple low-dimensional noise process into the complex distribution of the data. This raises the question of whether biological networks…
We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior…
Social learning is defined as the ability of a population to aggregate information, a process which must crucially depend on the mechanisms of social interaction. Consumers choosing which product to buy, or voters deciding which option to…
We study the learning problem of revealed preference in a stochastic setting: a learner observes the utility-maximizing actions of a set of agents whose utility follows some unknown distribution, and the learner aims to infer the…
How should we present training examples to learners to teach them classification rules? This is a natural problem when training workers for crowdsourcing labeling tasks, and is also motivated by challenges in data-driven online education.…
We develop a novel framework for costly information acquisition in which a decision-maker learns about an unobserved state by choosing a signal distribution, with the cost of information determined by the distribution of noise in the…
Neural networks are not learning optimal decision boundaries. We show that decision boundaries are situated in areas of low training data density. They are impacted by few training samples which can easily lead to overfitting. We provide a…