Related papers: Is Complexity an Illusion?
Simplicity bias is an intriguing phenomenon prevalent in various input-output maps, characterized by a preference for simpler, more regular, or symmetric outputs. Notably, these maps typically feature high-probability outputs with simple…
The article analyses foundational principles relevant to the creation of artificial general intelligence (AGI). Intelligence is understood as the ability to create novel skills that allow to achieve goals under previously unknown…
Scholars have long hypothesized that democratic forms of government are more compatible with scientific advancement. However, empirical analysis testing the democracy-science compatibility hypothesis remains underdeveloped. This article…
Despite their outstanding performance, large language models (LLMs) suffer notorious flaws related to their preference for simple, surface-level textual relations over full semantic complexity of the problem. This proposal investigates a…
Machine learning systems such as large scale recommendation systems or natural language processing systems are usually trained on billions of training points and are associated with hundreds of billions or trillions of parameters. Improving…
The description of complex physical phenomena often involves sophisticated models that rely on a large number of parameters, with many dimensions and scales. One practical way to simplify that kind of models is to discard some of the…
We define and investigate a property of mechanisms that we call "strategic simplicity," and that is meant to capture the idea that, in strategically simple mechanisms, strategic choices require limited strategic sophistication. We define a…
This paper defends an augmented cognitively oriented "generic-design hypothesis": There are both significant similarities between the design activities implemented in different situations and crucial differences between these and other…
Visual clutter affects our ability to see: objects that would be identifiable on their own, may become unrecognizable when presented close together ("crowding") -- but the psychophysical characteristics of crowding have resisted…
Studies of discrete languages emerging when neural agents communicate to solve a joint task often look for evidence of compositional structure. This stems for the expectation that such a structure would allow languages to be acquired faster…
This is a model of consciousness. The hard problem of consciousness, what it feels like, is answered. The work builds on medical research analyzing the source and mechanisms associated with our feelings. It goes further by describing a…
There is a cognitive limit in Human Mind. This cognitive limit has played a decisive role in almost all fields including computer sciences. The cognitive limit replicated in computer sciences is responsible for inherent Computational…
We examine how users perceive the limitations of an AI system when it encounters a task that it cannot perform perfectly and whether providing explanations alongside its answers aids users in constructing an appropriate mental model of the…
Traditional machine learning relies on explicit models and domain assumptions, limiting flexibility and interpretability. We introduce a model-free framework using surprisal (information theoretic uncertainty) to directly analyze and…
Recent diagnostic datasets on compositional generalization, such as SCAN (Lake and Baroni, 2018) and COGS (Kim and Linzen, 2020), expose severe problems in models trained from scratch on these datasets. However, in contrast to this poor…
A vast array of transformative technologies developed over the past decade has enabled measurement and perturbation at ever increasing scale, yet our understanding of many systems remains limited by experimental capacity. Overcoming this…
In the framework of certain general probability theories of single systems, we identify various nonclassical features such as incompatibility, multiple pure-state decomposability, measurement disturbance, no-cloning and the impossibility of…
Generalization in generative modeling is defined as the ability to learn an underlying distribution from a finite dataset and produce novel samples, with evaluation largely driven by held-out performance and perceived sample quality. In…
If we take the subjective character of consciousness seriously, consciousness becomes a matter of "being" rather than "doing". Because "doing" can be dissociated from "being", functional criteria alone are insufficient to decide whether a…
This chapter does not deal with specific tools and techniques for managing complex systems, but proposes some basic concepts that help us to think and speak about complexity. We review classical thinking and its intrinsic drawbacks when…