Related papers: The ghosts of forgotten things: A study on size af…

We consider the properties of listwise deletion when both $n$ and the number of variables grow large. We show that when (i) all data has some idiosyncratic missingness and (ii) the number of variables grows superlogarithmically in $n$,…

A central method for analyzing the asymptotic complexity of a functional program is to extract and then solve a recurrence that expresses evaluation cost in terms of input size. The relevant notion of input size is often specific to a…

The main goal of this paper was to analyze the importance that the effects of learning and forgetting might have in a lot-sizing problem. It assumes that the learning curve and the economies of scale are present in several industries yet…

In a previous work, the first author extended to higher-order rewriting and dependent types the use of size annotations in types, a termination proof technique called type or size based termination and initially developed for ML-like…

We give a decomposition of the posterior predictive variance using the law of total variance and conditioning on a finite dimensional discrete random variable. This random variable summarizes various features of modeling that are used to…

Why do we forget? Why do we remember things that never happened? The conventional answer points to biological hardware. We propose a different one: geometry. Here we show that high-dimensional embedding spaces, subjected to noise,…

Interpolation is an important property of classical and many non classical logics that has been shown to have interesting applications in computer science and AI. Here we study the Interpolation Property for the propositional version of the…

In this paper, we investigate knowledge reasoning within a simple framework called knowledge structure. We use variable forgetting as a basic operation for one agent to reason about its own or other agents\ knowledge. In our framework, two…

Construct recursively a long string of words w1. .. wn, such that at each step k, w k+1 is a new word with a fixed probability p $\in$ (0, 1), and repeats some preceding word with complementary probability 1 -- p. More precisely, given a…

We consider the following novel variation on a classical avoidance problem from combinatorics on words: instead of avoiding repetitions in all factors of a word, we avoid repetitions in all factors where each individual factor is considered…

Disordered and amorphous materials often retain memories of perturbations they have experienced since preparation. Studying such memories is a gateway to understanding this challenging class of systems, yet it often requires the ability to…

Memorization in language models is typically treated as a homogenous phenomenon, neglecting the specifics of the memorized data. We instead model memorization as the effect of a set of complex factors that describe each sample and relate it…

In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this…

Systems of enhanced memory capacity are subjected to a universal effect of memory burden, which suppresses their decay. In this paper, we study a prototype model to show that memory burden can be overcome by rewriting stored quantum…

Large Language Models (LLMs) are often evaluated against ideals of perfect Bayesian inference, yet growing evidence suggests that their in-context reasoning exhibits systematic forgetting of past information. Rather than viewing this…

In this paper we study the expressive power of Horn-formulae in dependence logic and show that they can express NP-complete problems. Therefore we define an even smaller fragment D-Horn* and show that over finite successor structures it…

The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules…

This paper revisits the conundrum faced when one attempts to understand the dynamics of black hole formation and evaporation without abandoning unitary evolution. Previous efforts to resolve this puzzle assume that information escapes in…

The Hardy-Littlewood majorant problem asks whether L^p norms of functions on the circle grow if one replaces their Fourier coefficients with their absolute values. This is clear if p is an even integer, but false if p is any other number.…

Large Language Models are affected by the phenomena of memorizing and forgetting their training data. But how do these vary by model size? We work towards this question by investigating how the model size affects the model's ability to…