Related papers: Spin Glass Concepts in Computer Science, Statistic…

Consider a collection of competing machine learning algorithms. Given their performance on a benchmark of datasets, we would like to identify the best performing algorithm. Specifically, which algorithm is most likely to rank highest on a…

We develop a general framework for estimating function-valued parameters under equality or inequality constraints in infinite-dimensional statistical models. Such constrained learning problems are common across many areas of statistics and…

In this talk I will show that usual spin glasses are a peculiar kind of Abelian gauge theory. I will shortly review the techniques used to study them. At the end I will consider more general models (e.g. spin glasses based on non Abelian…

The concept of random dynamical system is a comparatively recent development combining ideas and methods from the well developed areas of probability theory and dynamical systems. Due to our inaccurate knowledge of the particular physical…

We introduce a statistical physics inspired supervised machine learning algorithm for classification and regression problems. The method is based on the invariances or stability of predicted results when known data is represented as…

Over the last several years, the use of machine learning (ML) in neuroscience has been rapidly increasing. Here, we review ML's contributions, both realized and potential, across several areas of systems neuroscience. We describe four…

Probabilistic submeasures generalizing the classical (numerical) submeasures are introduced and discussed in connection with some classes of aggregation functions. A special attention is paid to triangular norm-based probabilistic…

We numerically study finite-dimensional spin glasses at low and zero temperature, finding evidences for (i) strong time/space heterogeneities, (ii) spontaneous time scale separation and (iii) power law distributions of flipping times. Using…

In this chapter, we discuss recent work on learning sparse approximations to high-dimensional functions on data, where the target functions may be scalar-, vector- or even Hilbert space-valued. Our main objective is to study how the…

In this work a short overview of the development of spin glass theories, mainly long and short range Ising models, are presented.

The study of mechanistic interpretability aims to reverse-engineer a model to explain its behaviors. While recent studies have focused on the static mechanism of a certain behavior, the learning dynamics inside a model remain to be…

Deep learning algorithms have made incredible strides in the past decade, yet due to their complexity, the science of deep learning remains in its early stages. Being an experimentally driven field, it is natural to seek a theory of deep…

In this paper, we make the case that a scientific theory of deep learning is emerging. By this we mean a theory which characterizes important properties and statistics of the training process, hidden representations, final weights, and…

Manifold learning is a popular and quickly-growing subfield of machine learning based on the assumption that one's observed data lie on a low-dimensional manifold embedded in a higher-dimensional space. This thesis presents a mathematical…

We propose a framework for computing, optimizing and integrating with respect to a smooth marginal likelihood in statistical models that involve high-dimensional parameters/latent variables and continuous low-dimensional hyperparameters.…

The p-spin spin-glass model has been studied extensively at mean-field level because of the insights which it provides into the mode-coupling approach to structural glasses and the nature of the glass transition. We demonstrate explicitly…

In this work, we present a novel, machine-learning approach for constructing Multiclass Interpretable Scoring Systems (MISS) - a fully data-driven methodology for generating single, sparse, and user-friendly scoring systems for multiclass…

Statistical system models provide the basis for the examination of various sorts of distributions. Classification distributions are a very common and versatile form of statistics in e.g. real economic, social, and IT systems. The…

A random vector whose norm and overlap (inner product with an independent copy) concentrates is shown to have random low-dimensional projections that are approximately random Gaussians. Conversely, asymptotically random Gaussian projections…

In these two lectures I review our theoretical understanding of spin glasses paying a particular attention to the basic physical ideas. We introduce the replica method and we describe its probabilistic consequences (we stress the recently…