Related papers: Transfer entropy and O-information to detect grokk…

This study presents a multi-stage approach to mental health classification by leveraging traditional machine learning algorithms, deep learning architectures, and transformer-based models. A novel data set was curated and utilized to…

We show how to learn structures of generic, non-Markovian, quantum stochastic processes using a tensor network based machine learning algorithm. We do this by representing the process as a matrix product operator (MPO) and train it with a…

We identify and formalize a fundamental gradient descent phenomenon resulting in a learning proclivity in over-parameterized neural networks. Gradient Starvation arises when cross-entropy loss is minimized by capturing only a subset of…

This paper investigates, from information theoretic grounds, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, i.e., using fewer bits than needed to…

This paper demonstrates that grokking behavior in modular arithmetic with a modulus P in a neural network can be controlled by modifying the profile of the activation function as well as the depth and width of the model. Plotting the even…

Supervised learning models are typically trained on a single dataset and the performance of these models rely heavily on the size of the dataset, i.e., amount of data available with the ground truth. Learning algorithms try to generalize…

As the number of qubits in a sensor increases, the complexity of designing and controlling the quantum circuits grows exponentially. Manually optimizing these circuits becomes infeasible. Optimizing entanglement distribution in large-scale…

In quantum physics, multiparticle systems are described by quantum states acting on tensor products of Hilbert spaces. This product structure leads to the distinction between product states and entangled states; moreover, one can quantify…

Quantum phase transitions occur at zero temperature and involve the appearance of long-range correlations. These correlations are not due to thermal fluctuations but to the intricate structure of a strongly entangled ground state of the…

Beyond their origin in modeling many-body quantum systems, tensor networks have emerged as a promising class of models for solving machine learning problems, notably in unsupervised generative learning. While possessing many desirable…

Entanglement is a key property in the development of quantum technologies and in the study of quantum many-body simulations. However, entanglement measurement typically requires quantum full-state tomography (FST). Here we present a neural…

As access to high-quality, domain-specific data grows increasingly scarce, multi-epoch training has become a practical strategy for adapting large language models (LLMs). However, autoregressive models often suffer from performance…

Despite the remarkable success of large large-scale neural networks, we still lack unified notation for thinking about and describing their representational spaces. We lack methods to reliably describe how their representations are…

Recently, machine learning has been widely applied in the field of quantum information, notably in tasks such as entanglement detection, steering characterization, and nonlocality verification. However, few studies have focused on utilizing…

Grokking is the phenomenon whereby, unlike the training performance, which peaks early in the training process, the test/generalization performance of a model stagnates over arbitrarily many epochs and then suddenly jumps to usually close…

Progress in the application of machine learning techniques to the prediction of solid-state and molecular materials properties has been greatly facilitated by the development state-of-the-art feature representations and novel deep learning…

We introduce a variational algorithm based on Matrix Product States that is trained by minimizing a generalized free energy defined using Tsallis entropy instead of the standard Gibbs entropy. As a result, our model can generate the…

It has been witnessed that learned image compression has outperformed conventional image coding techniques and tends to be practical in industrial applications. One of the most critical issues that need to be considered is the…

Quantum Neural Networks (QNN) are considered a candidate for achieving quantum advantage in the Noisy Intermediate Scale Quantum computer (NISQ) era. Several QNN architectures have been proposed and successfully tested on benchmark datasets…

Knowledge Tracing (KT), which aims to model student knowledge level and predict their performance, is one of the most important applications of user modeling. Modern KT approaches model and maintain an up-to-date state of student knowledge…