相关论文: Exploring the Temperature-Dependent Phase Transiti…

Associative memory models based on Hopfield networks and self-attention based on key-value mechanisms have been popular approaches in the study of memory mechanisms in deep learning. It has been pointed out that the state update rule of the…

The Hopfield model in a transverse field is investigated in order to clarify how quantum fluctuations affect the macroscopic behavior of neural networks. Using the Trotter decomposition and the replica method, we find that the $\alpha$ (the…

Generative models, including diffusion models, are increasingly used as foundation models and adapted through sequential fine-tuning, making continual learning an essential problem setting. However, continual learning in such generative…

The Hopfield network (HN) is a classical model of associative memory whose dynamics are closely related to the Ising spin system with 2-body interactions. Stored patterns are encoded as minima of an energy function shaped by a Hebbian…

Content-addressable memories such as Modern Hopfield Networks (MHN) have been studied as mathematical models of auto-association and storage/retrieval in the human declarative memory, yet their practical use for large-scale content storage…

We study the Hopfield model where the ratio $\alpha$ of patterns to sites grows large. We prove that the free energy with inverse temperature $\beta$ and external field $B$ behaves like $\beta\sqrt\alpha+\gamma$, where $\gamma$ is the…

Transformer-based models have demonstrated exceptional performance across diverse domains, becoming the state-of-the-art solution for addressing sequential machine learning problems. Even though we have a general understanding of the…

In this work we propose an energy functional along the lines of Modern Hopfield Networks (MNH), the stationary points of which correspond to the attention due to Vaswani et al. [12], thus unifying both frameworks. The minima of this…

Here we present a fundamental comprehension of the microscopic mechanisms leading to the emergence of inverse melting transitions by considering a thorough mean-field analysis of a variety of minimal models with different competing…

Heat transfer in semiconductor devices is dominated by chip and substrate assemblies, where heat generated within a finite chip layer dissipates into a semi-infinite substrate with much higher thermophysical properties. This mismatch…

The softmax function combined with a cross-entropy loss is a principled approach to modeling probability distributions that has become ubiquitous in deep learning. The softmax function is defined by a lone hyperparameter, the temperature,…

The lack of signed random networks in standard balance studies has prompted us to extend the Hamiltonian of the standard balance model. Random networks with tunable parameters are suitable for better understanding the behavior of standard…

Finding synthesis routes for molecules of interest is an essential step in the discovery of new drugs and materials. To find such routes, computer-assisted synthesis planning (CASP) methods are employed which rely on a model of chemical…

We develop a thermodynamic theory for machine learning (ML) systems. Similar to physical thermodynamic systems which are characterized by energy and entropy, ML systems possess these characteristics as well. This comparison inspire us to…

As the demand for computational power increases, high-bandwidth memory (HBM) has become a critical technology for next-generation computing systems. However, the widespread adoption of HBM presents significant thermal management challenges,…

We study energy transport in the paradigmatic Hamiltonian mean-field (HMF) model and other related long-range interacting models using molecular dynamics simulations. We show that energy diffusion in the HMF model is subdiffusive in nature,…

Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the evolution of tokens in deep encoder-only transformers at…

Studies have been made on the phase transition phenomena of an oscillator network model based on a standard Hebb learning rule like the Hopfield model. The relative phase informations---the in-phase and anti-phase, can be embedded in the…

We have varied the disorder in a two-dimensional electron system in silicon by applying substrate bias. When the disorder becomes sufficiently low, we observe the emergence of the metallic phase, and find evidence for a metal-insulator…

A well-known approach to describe the dynamics of an open quantum system is to compute the master equation evolving the reduced density matrix of the system. This approach plays an important role in describing excitation transfer through…