Related papers: Construction of a Neural Network with Temperature-…

This study proposes a method to enhance neural network performance when training data and application data are not very similar, e.g., out of distribution problems, as well as pattern and regime shifts. The method consists of three main…

Many complex systems exhibit interactions that depend not only on pairwise connections, but also group structures and memory effects. To capture such effects, we develop a unified tensor framework for modeling higher-order Markov chains…

In this study, we improve a neural network (NN) parameterization of deep convection in the global atmosphere model ARP-GEM. To take into account the sporadic nature of convection, we develop a NN parameterization that includes a triggering…

A thermal simulation methodology is developed for interconnects enabled by a data-driven learning algorithm accounting for variations of material properties, heat sources and boundary conditions (BCs). The methodology is based on the…

We present a stochastic neural automata in which activity fluctuations and synaptic intensities evolve at different temperature, the latter moving through a set of stored patterns. The network thus exhibits various retrieval phases,…

We present, theoretical predictions and Monte Carlo simulations, for a simple three matrix model that exhibits an exotic phase transition. The nature of the transition is very different if approached from the high or low temperature side.…

Coded recurrent neural networks with three levels of sparsity are introduced. The first level is related to the size of messages, much smaller than the number of available neurons. The second one is provided by a particular coding rule,…

Recently, there have been many works on the deep learning of statistical ensembles to determine the critical temperature of a possible phase transition. We analyze the detailed structure of an optimized deep learning machine and prove the…

We study the interplay between memorization and generalization of overparameterized networks in the extreme case of a single training example and an identity-mapping task. We examine fully-connected and convolutional networks (FCN and CNN),…

We design a neural network to extract and process features from absorption images taken of one-dimensional Bose gases in the quasi-condensate regime. Specifically, the network is trained to predict both the temperature of single…

Motivated by biological considerations, we study sparse neural maps from an input layer to a target layer with sparse activity, and specifically the problem of storing $K$ input-target associations $(x,y)$, or memories, when the target…

Simulated tempering (ST) has attracted a great deal of attention in the last years, due to its capability to allow systems with complex dynamics to escape from regions separated by large entropic barriers. However its performance is…

We consider the problem of neural association for a network of non-binary neurons. Here, the task is to first memorize a set of patterns using a network of neurons whose states assume values from a finite number of integer levels. Later,…

Interpreting spectropolarimetric observations of the solar atmosphere takes much longer than the acquiring the data. The most important reason for this is that the model fitting, or "inversion", used to infer physical quantities from the…

Graph embedding methods represent nodes in a continuous vector space, preserving information from the graph (e.g. by sampling random walks). There are many hyper-parameters to these methods (such as random walk length) which have to be…

We use temperature tuning to control signal propagation in simple one-dimensional arrays of masses connected by hard anharmonic springs and with no local potentials. In our numerical model a sustained signal is applied at one site of a…

We develop a thermodynamic theory for machine learning (ML) systems. Similar to physical thermodynamic systems which are characterized by energy and entropy, ML systems possess these characteristics as well. This comparison inspire us to…

Sequential activation of neurons is a common feature of network activity during a variety of behaviors, including working memory and decision making. Previous network models for sequences and memory emphasized specialized architectures in…

Neurophysiologists are nowadays able to record from a large number of extracellular electrodes and to extract, from the raw data, the sequences of action potentials or spikes generated by many neurons. Unfortunately these ''many neurons''…

Theoretical efforts to prove advantages of Transformers in comparison with classical architectures such as feedforward and recurrent neural networks have mostly focused on representational power. In this work, we take an alternative…