Related papers: Domain size asymptotics for Markov logic networks
We characterise the likelihood function computed from a Bayesian network with latent variables as root nodes. We show that the marginal distribution over the remaining, manifest, variables also factorises as a Bayesian network, which we…
Let $(\Omega,\mathcal{F}, \mathbb{P})$ be a probability space and $E$ be a finite set. Assume that $X=(X_n)$ is an irreducible and aperiodic Markov chain, defined on $(\Omega,\mathcal{F}, \mathbb{P})$, with values in $E$ and with transition…
Let $X_1,X_2, \ldots $ be independent identically distributed random points in a convex polytopal domain $A \subset \mathbb{R}^d$. Define the largest nearest neighbour link $L_n$ to be the smallest $r$ such that every point of $\mathcal…
Continuous-time Mallows processes are processes of random permutations of the set $\{1, \ldots, n\}$ whose marginal at time $t$ is the Mallows distribution with parameter $t$. Recently Corsini showed that there exists a unique Markov…
Recent advances in prompt engineering enable large language models (LLMs) to solve multi-hop logical reasoning problems with impressive accuracy. However, there is little existing work investigating the robustness of LLMs with few-shot…
The emergence of discourse-like tokens such as "wait" and "therefore" in large language models (LLMs) has offered a unique window into their reasoning processes. However, systematic analyses of how such signals vary across training…
This work investigates the computational expressivity of language models (LMs) based on recurrent neural networks (RNNs). Siegelmann and Sontag (1992) famously showed that RNNs with rational weights and hidden states and unbounded…
The logical negation property (LNP), which implies generating different predictions for semantically opposite inputs, is an important property that a trustworthy language model must satisfy. However, much recent evidence shows that…
We consider the lossless compression bound of any individual data sequence. If we fit the data by a parametric model, the entropy quantity $nH({\hat \theta}_n)$ obtained by plugging in the maximum likelihood estimate is an underestimate of…
Large language models (LLMs) can be seen as atomic units of computation mapping sequences to a distribution over sequences. Thus, they can be seen as stochastic language layers in a language network, where the learnable parameters are the…
We introduce a formal logical language, called conditional probability logic (CPL), which extends first-order logic and which can express probabilities, conditional probabilities and which can compare conditional probabilities. Intuitively…
We propose a general approach to construct weighted likelihood estimating equations with the aim of obtaining robust parameter estimates. We modify the standard likelihood equations by incorporating a weight that reflects the statistical…
One of the questions of distribution of prime numbers is considered in the article. It is shown what error is obtained from the assumption that the asymptotic density of a sequence of primes is a probability. Various forms of an analogue of…
In this paper we show that inference in 2-variable Markov logic networks (MLNs) with cardinality and function constraints is domain-liftable. To obtain this result we use existing domain-lifted algorithms for weighted first-order model…
We study the asymptotic behavior for asymmetric neuronal dynamics in a network of linear Hopfield neurons. The interaction between the neurons is modeled by random couplings which are centered i.i.d. random variables with finite moments of…
This paper deals with a parametrized family of partially observed bivariate Markov chains. We establish that, under very mild assumptions, the limit of the normalized log-likelihood function is maximized when the parameters belong to the…
Language Models (LMs) are being proposed for mental health applications where the heightened risk of adverse outcomes means predictive performance may not be a sufficient litmus test of a model's utility in clinical practice. A model that…
Small-world networks (SWN), obtained by randomly adding to a regular structure additional links (AL), are of current interest. In this article we explore (based on physical models) a new variant of SWN, in which the probability of realizing…
Given a length $n$ sample from $\mathbb{R}^d$ and a neural network with a fixed architecture with $W$ weights, $k$ neurons, linear threshold activation functions, and binary outputs on each neuron, we study the problem of uniformly sampling…
The aim of this note is to state a couple of general results about the properties of the penalized maximum likelihood estimators (pMLE) and of the posterior distribution for parametric models in a non-asymptotic setup and for possibly large…