Related papers: Three Perspectives on Complexity $-$ Entropy, Comp…
Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although…
We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies $H(X,Y)$, $H(X|Y)$, $H(Y|X)$, and maximization of mutual information $I(X;Y)$, over convex regions. When…
In a topological dynamical system the complexity of an orbit is a measure of the amount of information (algorithmic information content) that is necessary to describe the orbit. This indicator is invariant up to topological conjugation. We…
We study the properties of Tsallis entropy and Shannon entropy from the point of view of algorithmic randomness. In algorithmic information theory, there are two equivalent ways to define the program-size complexity K(s) of a given finite…
Network or graph structures are ubiquitous in the study of complex systems. Often, we are interested in complexity trends of these system as it evolves under some dynamic. An example might be looking at the complexity of a food web as…
Complexity is an inherent attribute of any project. The purpose of defining and documenting complexity is to have an early warning tool allowing a project team to focus on certain areas and aspects of the project in order to prevent and…
According to E.T. Jaynes and E.P. Wigner, entropy is an anthropomorphic concept in the sense that in a physical system correspond many thermodynamic systems. The physical system can be examined from many points of view each time examining…
In this work, we develop a new complexity metric for an important class of low-rank matrix optimization problems in both symmetric and asymmetric cases, where the metric aims to quantify the complexity of the nonconvex optimization…
Deploying LLMs raises two coupled challenges: (1) monitoring--estimating where a model underperforms as traffic and domains drift--and (2) improvement--prioritizing data acquisition to close the largest performance gaps. We test whether an…
The Cram\'er-Rao, Fisher-Shannon and LMC shape complexity measures have been recently shown to play a relevant role to study the internal disorder of finite many-body systems (e.g., atoms, molecules, nuclei). They highlight amongst the…
The description complexity of a model is the length of the shortest formula that defines the model. We study the description complexity of unary structures in first-order logic FO, also drawing links to semantic complexity in the form of…
Complexity matching characterizes the role of information in interactions between systems and can be traced back to the 1957 Introduction to Cybernetics by Ross Ashby. We argue that complexity can be expressed in terms of crucial events,…
Since the introduction of the Kolmogorov complexity of binary sequences in the 1960s, there have been significant advancements in the topic of complexity measures for randomness assessment, which are of fundamental importance in theoretical…
Soft set theory is an important and emerging area within soft computing, owing to its attribute-oriented mathematical framework and its wide applicability in diverse domains, including science and social sciences. The theoretical…
This paper addresses the problem of measuring complexity from embedded attractors as a way to characterize changes in the dynamical behaviour of different types of systems by observing their outputs. With the aim of measuring the stability…
Complex systems are characterised by a tight, nontrivial interplay of their constituents, which gives rise to a multi-scale spectrum of emergent properties. In this scenario, it is practically and conceptually difficult to identify those…
Complex networks can be understood as graphs whose connectivity deviates from those of regular or near-regular graphs, which are understood as being `simple'. While a great deal of the attention so far dedicated to complex networks has been…
Entropy is the measure of uncertainty in any data and is adopted for maximisation of mutual information in many remote sensing operations. The availability of wide entropy variations motivated us for an investigation over the suitability…
Complexity measures are introduced, that quantify the change of the natural entropy fluctuations at different length scales in time-series emitted from systems operating far from equilibrium. They identify impending sudden cardiac death…
A practical measure for the complexity of sequences of symbols (``strings'') is introduced that is rooted in automata theory but avoids the problems of Kolmogorov-Chaitin complexity. This physical complexity can be estimated for ensembles…