Related papers: Continuous Randomness via Transformations of 2-Ran…
LLM prompting is widely used for naturally stated tasks, yet it is unreliable it may succeed on a few test cases but fail at deployment time. We study performance prediction: given a program, either symbolic (e.g. Python) or a prompt…
Turing's famous 'machine' framework provides an intuitively clear conception of 'computing with real numbers'. A recursive counterexample to a theorem shows that the theorem does not hold when restricted to computable objects. These…
Let \mu be a computable ergodic shift-invariant measure over the Cantor space. Providing a constructive proof of Shannon-McMillan-Breiman theorem, V'yugin proved that if a sequence x is Martin-L\"of random w.r.t. \mu then the strong…
Large language models, LLMs, are increasingly deployed in multiturn settings where earlier responses shape later ones, making reliability dependent on whether a conversation remains consistent over time. When this consistency degrades…
We present an orthogonal expansion for real, function-regulated, second-order random measures over $\mathbb{R}^{d}$ with measure covariance. Such a expansion, which can be seen as a Karhunen-Lo\`eve decomposition, consists in a series of…
Continuous-output neural machine translation (CoNMT) replaces the discrete next-word prediction problem with an embedding prediction. The semantic structure of the target embedding space (i.e., closeness of related words) is intuitively…
Can autoregressive large language models (LLMs) learn consistent probability distributions when trained on sequences in different token orders? We prove formally that for any well-defined probability distribution, sequence perplexity is…
Large language models display remarkable capabilities in logical and mathematical reasoning, allowing them to solve complex tasks. Interestingly, these abilities emerge in networks trained on the simple task of next-token prediction. In…
Correlations that cannot be reproduced with local variables certify the generation of private randomness. Usually, the violation of a Bell inequality is used to quantify the amount of randomness produced. Here, we show how private…
A central problem in cognitive science and behavioural neuroscience as well as in machine learning and artificial intelligence research is to ascertain whether two or more decision makers (be they brains or algorithms) use the same…
In this paper, two tests, based on CUSUM of the residuals and least squares estimation, are studied to detect in real time a change-point in a nonlinear model. A first test statistic is proposed by extension of a method already used in the…
Many learning problems require predicting how populations evolve under an unknown transformation. A natural representation for such populations is a probability measure, with point clouds as a key example. In this work, we study the…
This paper considers the change-point problem for finite sequences of networks. To avoid the difficulty of computing the normalization coefficient, such as in Exponential random graphical models (ERGMs) and Markov networks, we construct a…
Dedicated to the memory of Professor Tze Leung Lai, this paper introduces three multi-hypothesis sequential tests. These tests are derived from one-sided versions of the sequential probability ratio test and its modifications. They are…
Let $\pi_n$ be a uniformly chosen random permutation on $[n]$. Using an analysis of the probability that two overlapping consecutive $k$-permutations are order isomorphic, the authors of a recent paper showed that the expected number of…
Detecting the emergence of an abrupt change-point is a classic problem in statistics and machine learning. Kernel-based nonparametric statistics have been used for this task which enjoy fewer assumptions on the distributions than the…
Algorithmic theories of randomness can be related to theories of probabilistic sequence prediction through the notion of a predictor, defined as a function which supplies lower bounds on initial-segment probabilities of infinite sequences.…
We consider a number of graph kernels and proximity measures including commute time kernel, regularized Laplacian kernel, heat kernel, exponential diffusion kernel (also called "communicability"), etc., and the corresponding distances as…
In recent studies, new measures of complexity for nonlinear systems have been proposed based on probabilistic grounds, as the LMC measure (Phys. Lett. A {\bf 209} (1995) 321) or the SDL measure (Phys. Rev. E {\bf 59} (1999) 2). All these…
Fine-tuning LLMs on tabular classification tasks can lead to the phenomenon of fine-tuning multiplicity where equally well-performing models make conflicting predictions on the same input. Fine-tuning multiplicity can arise due to…