Related papers: Generalized Householder transformations
In this paper, we establish a mathematical duality between utility transforms and probability distortions. These transforms play a central role in decision under risk by forming the foundation for the classic theories of expected utility,…
Transformer architectures have achieved remarkable empirical success in modeling contextual relations, yet a clear understanding of their expressive power is still lacking. In this work, we introduce a measure-theoretic framework in which…
The Transformer architecture has revolutionized the field of sequence modeling and underpins the recent breakthroughs in large language models (LLMs). However, a comprehensive mathematical theory that explains its structure and operations…
Natural language exhibits patterns of hierarchically governed dependencies, in which relations between words are sensitive to syntactic structure rather than linear ordering. While re-current network models often fail to generalize in a…
Descriptive set theory is mainly concerned with studying subsets of the space of all countable binary sequences. In this paper we study the generalization where countable is replaced by uncountable. We explore properties of generalized…
Algorithmic generalization in machine learning refers to the ability to learn the underlying algorithm that generates data in a way that generalizes out-of-distribution. This is generally considered a difficult task for most machine…
Convenient parameterizations of matrices in terms of vectors transform (certain classes of) matrix equations into covariant (hence rotation-invariant) vector equations. Certain recently introduced such parameterizations are tersely…
Most expressivity results for transformers treat them as language recognizers -- devices that accept or reject strings -- rather than as they are used in practice: as language models that generate strings autoregressively and…
The deconfounder was proposed as a method for estimating causal parameters in a context with multiple causes and unobserved confounding. It is based on recovery of a latent variable from the observed causes. We disentangle the causal…
Convolution is a ubiquitous operation in mathematics and computing. The Kripke semantics for substructural and interval logics motivates its study for quantale-valued functions relative to ternary relations. The resulting notion of…
Transformers flexibly operate over sets of real-valued vectors representing task-specific entities and their attributes, where each vector might encode one word-piece token and its position in a sequence, or some piece of information that…
While reinforcement learning has achieved remarkable successes in several domains, its real-world application is limited due to many methods failing to generalise to unfamiliar conditions. In this work, we consider the problem of…
Domain generalization (DG) aims to learn a model on several source domains, hoping that the model can generalize well to unseen target domains. The distribution shift between domains contains the covariate shift and conditional shift, both…
Generalized analytic functions over generalized analytic manifolds are build from sums of convergent real power series with non-negative real exponents (and some well-ordering condition on the support). In a paper by Mart\'in-Villaverde,…
In quantum mechanics, not everything that can be observed can be observed simultaneously. Observational data exhibits \emph{contextuality} -- a generalisation of nonlocality -- if the result of an observation is necessarily dependent on…
Syntactic structures used to play a vital role in natural language processing (NLP), but since the deep learning revolution, NLP has been gradually dominated by neural models that do not consider syntactic structures in their design. One…
In this paper, we interpret disentanglement as the discovery of local charts of the data manifold and trace how this definition naturally leads to an equivalent condition for disentanglement: commutativity between factors of variation. We…
Monads can be interpreted as encoding formal expressions, or formal operations in the sense of universal algebra. We give a construction which formalizes the idea of "evaluating an expression partially": for example, "2+3" can be obtained…
In group representations several inductions given by tensoring with appropriate bimodules may be reconstructed via homology of $G$-posets with $G$-equivariant coefficients. For this purpose, we need various local categories of a finite…
In this note, we provide some categorical perspectives on the relativization construction arising from quantum measurement theory in the presence of symmetries and occupying a central place in the operational approach to quantum reference…