Related papers: Dynamical versus Bayesian Phase Transitions in a T…

This article introduces a novel Bayesian method for asynchronous change-point detection in multivariate time series. This method allows for change-points to occur earlier in some (leading) series followed, after a short delay, by…

The Information Bottleneck theory provides a theoretical and computational framework for finding approximate minimum sufficient statistics. Analysis of the Stochastic Gradient Descent (SGD) training of a neural network on a toy problem has…

We study the dynamics of supervised on-line learning of realizable tasks in feed-forward neural networks. We focus on the regime where the number of examples used for training is proportional to the number of input channels N. Using…

Transformer architecture has shown impressive performance in multiple research domains and has become the backbone of many neural network models. However, there is limited understanding on how it works. In particular, with a simple…

The Domany Kinzel (DK) model encompasses several types of non-equilibrium phase transitions, depending on the selected parameters. We apply supervised, semi-supervised, and unsupervised learning methods to studying the phase transitions and…

While momentum-based methods, in conjunction with stochastic gradient descent (SGD), are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work,…

Transitions between distinct dynamical regimes are ubiquitous in nonequilibrium systems. As a prototypical example, deposition growth is often accompanied by irreversible morphological instabilities. Forecasting such transitions from…

Stochastic Gradient Descent (SGD) has become the method of choice for solving a broad range of machine learning problems. However, some of its learning properties are still not fully understood. We consider least squares learning in…

Previous methods for dynamic facial expression in the wild are mainly based on Convolutional Neural Networks (CNNs), whose local operations ignore the long-range dependencies in videos. To solve this problem, we propose the spatio-temporal…

In distributed and federated learning algorithms, communication overhead is often reduced by performing multiple local updates between communication rounds. However, due to data heterogeneity across nodes and the local gradient noise within…

Modeling and interpreting spike train data is a task of central importance in computational neuroscience, with significant translational implications. Two popular classes of data-driven models for this task are autoregressive Point Process…

The properties of the pure-site clusters of spin models, i.e. the clusters which are obtained by joining nearest-neighbour spins of the same sign, are here investigated. In the Ising model in two dimensions it is known that such clusters…

Modern machine learning focuses on highly expressive models that are able to fit or interpolate the data completely, resulting in zero training loss. For such models, we show that the stochastic gradients of common loss functions satisfy a…

The application of generative modeling to many-body physics offers a promising pathway for analyzing high-dimensional state spaces of spin systems. However, unlike computer vision tasks where visual fidelity suffices, physical systems…

Exploration of the QCD phase diagram and critical point is one of the main goals in current relativistic heavy-ion collisions. The QCD critical point is expected to belong to a three-dimensional (3D) Ising universality class. Machine…

In the Gaussian sequence model $Y= \theta_0 + \varepsilon$ in $\mathbb{R}^n$, we study the fundamental limit of approximating the signal $\theta_0$ by a class $\Theta(d,d_0,k)$ of (generalized) splines with free knots. Here $d$ is the…

Learning continuous-time stochastic dynamics is a fundamental and essential problem in modeling sporadic time series, whose observations are irregular and sparse in both time and dimension. For a given system whose latent states and…

Continuous time Bayesian networks (CTBNs) describe structured stochastic processes with finitely many states that evolve over continuous time. A CTBN is a directed (possibly cyclic) dependency graph over a set of variables, each of which…

In traditional models of supervised learning, the goal of a learner -- given examples from an arbitrary joint distribution on $\mathbb{R}^d \times \{\pm 1\}$ -- is to output a hypothesis that is competitive (to within $\epsilon$) of the…

The manifold hypothesis suggests that high-dimensional neural time series lie on a low-dimensional manifold shaped by simpler underlying dynamics. To uncover this structure, latent dynamical variable models such as state-space models,…