相关论文: Mean-field theory of learning: from dynamics to st…

Using the cavity method and diagrammatic methods, we model the dynamics of batch learning of restricted sets of examples, widely applicable to general learning cost functions, and fully taking into account the temporal correlations…

Rich out of equilibrium collective dynamics of strongly interacting large assemblies emerge in many areas of science. Some intriguing and not fully understood examples are the glassy arrest in atomic, molecular or colloidal systems,…

Dynamical mean-field theory is a powerful physics tool used to analyze the typical behavior of neural networks, where neurons can be recurrently connected, or multiple layers of neurons can be stacked. However, it is not easy for beginners…

The dynamical and stationary properties of on-line learning from finite training sets are analysed using the cavity method. For large input dimensions, we derive equations for the macroscopic parameters, namely, the student-teacher…

Dynamical Mean-Field Theory (DMFT) is a powerful theoretical framework for analyzing systems with many interacting degrees of freedom. This tutorial provides an accessible introduction to DMFT. We begin with a linear model where the DMFT…

We analyze the learning of noisy teacher-generated examples by nonlinear and differentiable student perceptrons using the cavity method. The generic activation of an example is a function of the cavity activation of the example, which is…

The dynamics of minority games with agents trading on different time scales is studied via dynamical mean-field theory. We analyze the case where the agents' decision-making process is deterministic and its stochastic generalization with…

We discuss here the mean-field theory for a cellular automata model of meta-learning. The meta-learning is the process of combining outcomes of individual learning procedures in order to determine the final decision with higher accuracy…

Using the cavity method we consider the learning of noisy teacher-generated examples by a nonlinear student perceptron. For insufficient examples and weak weight decay, the activation distribution of the training examples exhibits a gap for…

Mean-field theory is a powerful tool for studying large neural networks. However, when the system is composed of a few neurons, macroscopic differences between the mean-field approximation and the real behavior of the network can arise.…

Although real-world complex systems typically interact through sparse and heterogeneous networks, analytic solutions of their dynamics are limited to models with all-to-all interactions. Here, we solve the dynamics of a broad range of…

Dynamical systems driven by strong external signals are ubiquituous in nature and engineering. Here we study "echo state networks", networks of a large number of randomly connected nodes, which represent a simple model of a neural network,…

We study inference and reconstruction of couplings in a partially observed kinetic Ising model. With hidden spins, calculating the likelihood of a sequence of observed spin configurations requires performing a trace over the configurations…

The main difficulty that arises in the analysis of most machine learning algorithms is to handle, analytically and numerically, a large number of interacting random variables. In this Ph.D manuscript, we revisit an approach based on the…

In this paper, we examine the convergence landscape of multi-agent learning under uncertainty. Specifically, we analyze two stochastic models of regularized learning in continuous games -- one in continuous and one in discrete time with the…

The driven transport of plastic systems in various disordered backgrounds is studied within mean field theory. Plasticity is modeled using non-convex interparticle potentials that allow for phase slips. This theory most naturally describes…

A dynamical model of an ecological community is analyzed within a "mean-field approximation" in which one of the species interacts with the combination of all of the other species in the community. Within this approximation the model may be…

We study the dynamics of supervised learning in layered neural networks, in the regime where the size $p$ of the training set is proportional to the number $N$ of inputs. Here the local fields are no longer described by Gaussian probability…

The ability to learn from a small number of examples has been a difficult problem in machine learning since its inception. While methods have succeeded with large amounts of training data, research has been underway in how to accomplish…

A dynamic mean field theory is developed for finite state and action Bayesian reinforcement learning in the large state space limit. In an analogy with statistical physics, the Bellman equation is studied as a disordered dynamical system;…