Related papers: Topos Theory for Generative AI and LLMs

With a model of a geometric theory in an arbitrary topos, we associate a site obtained by endowing a category of generalized elements of the model with a Grothendieck topology, which we call the antecedent topology. Then we show that the…

Locales have been studied as "topologies without points", mainly by tools of category theory. While traditional topology presents a space as a set of points with specified neighborhoods, localic topology presents a space as a lattice of…

Colimits are a fundamental construction in category theory. They provide a way to construct new objects by gluing together existing objects that are related in some way. We introduce a complementary notion of anticolimits, which provide a…

We extend Mazzola's counterpoint model using category theory, generalizing from the category $\mathbf{Set}$ to other topoi with suitable properties. This generalization suggests that counterpoint's essential structure depends on specific…

Large language models (LLMs) have reshaped the research landscape by enabling new approaches to knowledge retrieval and creative ideation. Yet their application in discipline-specific experimental science, particularly in highly…

Tokenization is a necessary component within the current architecture of many language mod-els, including the transformer-based large language models (LLMs) of Generative AI, yet its impact on the model's cognition is often overlooked. We…

Large language models (LLMs) are increasingly used as generators in iterative neural architecture search (NAS), yet no formal convergence theory exists for this class of algorithms. We model iterative LLM-NAS as a parametric Cross-Entropy…

Large language models (LLMs) have revolutionized the field of artificial intelligence, endowing it with sophisticated language understanding and generation capabilities. However, when faced with more complex and interconnected tasks that…

Automated analysis for engineering structures offers considerable potential for boosting efficiency by minimizing repetitive tasks. Although AI-driven methods are increasingly common, no systematic framework yet leverages Large Language…

Generative models are a class of AI models capable of creating new instances of data by learning and sampling from their statistical distributions. In recent years, these models have gained prominence in machine learning due to the…

Grounded theory offers deep insights from qualitative data, but its reliance on expert-intensive manual coding presents a major scalability bottleneck. Existing computational tools either fail on full automation or lack flexible schema…

We introduce a new hierarchical deep learning framework for recursive higher-order meta-learning that enables neural networks (NNs) to construct, solve, and generalise across hierarchies of tasks. Central to this approach is a generative…

Hierarchical planning is a powerful approach to model long sequences structurally. Aside from considering hierarchies in the temporal structure of music, this paper explores an even more important aspect: concept hierarchy, which involves…

We present a way of topologizing sets of Galois types over structures in abstract elementary classes with amalgamation. In the elementary case, the topologies thus produced refine the syntactic topologies familiar from first order logic. We…

This paper develops a systematic framework for integrating local categories that model logical connectives using higher category theory. By extending these local categories into a unified two-category enriched with natural isomorphisms, the…

This paper introduces a novel approach for identifying the possible large language models (LLMs) involved in text generation. Instead of adding an additional classification layer to a base LM, we reframe the classification task as a…

The advent of Large Language Models (LLMs) has provided unprecedented capabilities for analyzing unstructured text data. However, deploying these models as reliable, robust, and scalable classifiers in production environments presents…

The purpose of this paper is to show that the dual notions of elements & distinctions are the basic analytical concepts needed to unpack and analyze morphisms, duality, and universal constructions in the Sets, the category of sets and…

The classifying topos of a geometric theory is a topos such that geometric morphisms into it correspond to models of that theory. We study classifying toposes for different infinitary logics: first-order, sub-first-order (i.e. geometric…

Ontologies are useful for automatic machine processing of domain knowledge as they represent it in a structured format. Yet, constructing ontologies requires substantial manual effort. To automate part of this process, large language models…