相关论文: Optimal storage capacity of neural networks at fin…
The inclusion of a threshold in the dynamics of layered neural networks with variable activity is studied at arbitrary temperature. In particular, the effects on the retrieval quality of a self-controlled threshold obtained by forcing the…
The large variation of datasets is a huge barrier for image classification tasks. In this paper, we embraced this observation and introduce the finite temperature tensor network (FTTN), which imports the thermal perturbation into the matrix…
Quantum neural networks form one pillar of the emergent field of quantum machine learning. Here, quantum generalisations of classical networks realizing associative memories - capable of retrieving patterns, or memories, from corrupted…
In this work we study, analytically and employing Monte Carlo simulations, the influence of the competition between several activity-dependent synaptic processes, such as short-term synaptic facilitation and depression, on the maximum…
The storage capacity of a binary classification model is the maximum number of random input-output pairs per parameter that the model can learn. It is one of the indicators of the expressive power of machine learning models and is important…
The performance of large neural networks can be judged not only by their storage capacity but also by the time required for learning. A polynomial learning algorithm with learning time $\sim N^2$ in a network with $N$ units might be…
We studied the computational properties of an attractor neural network (ANN) with different network topologies. Though fully connected neural networks exhibit, in general, a good performance, they are biologically unrealistic, as it is…
The heat capacity $\mathcal{C}$ of a given probe is a fundamental quantity that determines, among other properties, the maximum precision in temperature estimation. In turn, $\mathcal{C}$ is limited by a quadratic scaling with the number of…
Synaptic efficacy between neurons is known to change within a short time scale dynamically. Neurophysiological experiments show that high-frequency presynaptic inputs decrease synaptic efficacy between neurons. This phenomenon is called…
We use a formal correspondence between thermodynamics and inference, where the number of samples can be thought of as the inverse temperature, to study a quantity called ``learning capacity'' which is a measure of the effective…
We calculate the dependence of heat capacity of a free standing thin membrane on its thickness and temperature. A remarkable fact is that for a given temperature there exists a minimum in the dependence of the heat capacity on the…
This paper considers the problem of information capacity of a random neural network. The network is represented by matrices that are square and symmetrical. The matrices have a weight which determines the highest and lowest possible value…
We study a refrigerator model which consists of two $n$-level systems interacting via a pulsed external field. Each system couples to its own thermal bath at temperatures $T_h$ and $T_c$, respectively ($\theta\equiv T_c/T_h<1$). The…
We estimate the critical capacity of the zero-temperature Hopfield model by using a novel and rigorous method. The probability of having a stable fixed point is one when $\alpha\le 0.113$ for a large number of neurons. This result is an…
We present a simple model that recalls two different patterns depending on the temperature. To realize a change in recall pattern due to temperature change, we embed two patterns to different graphs: the first pattern into a fully connected…
Deep neural networks (DNNs) depend on the storage of a large number of parameters, which consumes an important portion of the energy used during inference. This paper considers the case where the energy usage of memory elements can be…
Recent experimental and theoretical studies show that energy efficiency, which measures the amount of information processed by a neuron with per unit of energy consumption, plays an important role in the evolution of neural systems. Here,…
We present results for two different kinds of high order connections between neurons acting as corrections to the Hopfield model. Equilibrium properties are analyzed using the replica mean-field theory and compared with numerical…
With the help of quantum mechanics one can formulate a model of associative memory with optimal storage capacity. I generalize this model by introducing a parameter playing the role of an effective temperature. The corresponding…
Fine-tuning large pre-trained models is a common practice in machine learning applications, yet its mathematical analysis remains largely unexplored. In this paper, we study fine-tuning through the lens of memorization capacity. Our new…