Related papers: Construction of a Neural Network with Temperature-…

Spiking Neural Networks (SNNs) are one of the most promising bio-inspired neural networks models and have drawn increasing attention in recent years. The event-driven communication mechanism of SNNs allows for sparse and theoretically…

In this paper, we propose a mechanism for storing complex patterns within a neural network and subsequently recalling them. This model is based on our work published in 2018(Inazawa, 2018), which we have refined and extended in this work.…

Recent works demonstrate that early layers in a neural network contain useful information for prediction. Inspired by this, we show that extending temperature scaling across all layers improves both calibration and accuracy. We call this…

During typical gradient-based training of deep neural networks, all of the model's parameters are updated at each iteration. Recent work has shown that it is possible to update only a small subset of the model's parameters during training,…

We show that neural networks trained by evolutionary reinforcement learning can enact efficient molecular self-assembly protocols. Presented with molecular simulation trajectories, networks learn to change temperature and chemical potential…

Efficient continual learning in humans is enabled by a rich set of neurophysiological mechanisms and interactions between multiple memory systems. The brain efficiently encodes information in non-overlapping sparse codes, which facilitates…

Here we find the increase in 1/f noise of superconducting resonators at low temperatures to be completely incompatible with the standard tunneling model (STM) of Two Level Systems (TLS), which has been used to describe low-frequency noise…

The formation of secondary structures by a random RNA sequence is studied as a model system for the sequence-structure problem omnipresent in biopolymers. Several toy energy models are introduced to allow detailed analytical and numerical…

Sequence models lie at the heart of modern deep learning. However, rapid advancements have produced a diversity of seemingly unrelated architectures, such as Transformers and recurrent alternatives. In this paper, we introduce a unifying…

By means of Metropolis Monte Carlo simulations of a coarse-grained model for flexible polymers, we investigate how the integrated autocorrelation times of different energetic and structural quantities depend on the temperature. We show…

Graphs are useful to interpret widely used image processing methods, e.g., bilateral filtering, or to develop new ones, e.g., kernel based techniques. However, simple graph constructions are often used, where edge weight and connectivity…

We consider a generalization of the Hopfield model, where the entries of patterns are Gaussian and diluted. We focus on the high-storage regime and we investigate analytically the topological properties of the emergent network, as well as…

Computer simulations of structure formation in network forming materials (such as amorphous semiconductors, glasses, or fluids containing hydrogen bonds) are challenging. The problem is that large structural changes in the network topology…

In neuronal systems, inhibition contributes to stabilizing dynamics and regulating pattern formation. Through developing mean field theories of neuronal models, using complete graph networks, inhibition is commonly viewed as one ``control…

I build a melting temperature database that contains approximately 10,000 materials. Based on the database, I build a machine learning model that predicts melting temperature in seconds. The model features graph neural network and residual…

Real-world problems often involve complex and unstructured sets of measurements, which occur when sensors are sparsely placed in either space or time. Being able to model this irregular spatiotemporal data and extract meaningful forecasts…

Transformer-based language models excel at both recall (retrieving memorized facts) and reasoning (performing multi-step inference), but whether these abilities rely on distinct internal mechanisms remains unclear. Distinguishing recall…

Using a distinguishable-particle lattice model based on void-induced dynamics, we successfully reproduce the well-known linear relation between heat capacity and temperature at very low temperatures. The heat capacity is dominated by…

Transformers have demonstrated exceptional performance across various domains due to their self-attention mechanism, which captures complex relationships in data. However, training on smaller datasets poses challenges, as standard attention…

We construct a network from climate records of different geographical sites in the North Atlantic. A link between two sites represents the cross-correlations between the records of each site. We find that within the different phases of the…