Related papers: Simplicity bias, algorithmic probability, and the …
The concept of complexity appears in virtually all areas of knowledge. Its intuitive meaning shares similarities across fields, but disagreements between its details hinders a general definition, leading to a plethora of proposed…
Unexpectedness is a central concept in Simplicity Theory, a theory of cognition relating various inferential processes to the computation of Kolmogorov complexities, rather than probabilities. Its predictive power has been confirmed by…
As it was argued by Anderson [Science 177, 393 (1972)], the "reductionist" hypothesis does not by any means imply a "constructionist" one. Hence, in general, the behavior of large and complex aggregates of elementary components can not be…
Models can be simple for different reasons: because they yield a simple and computationally efficient interpretation of a generic dataset (e.g. in terms of pairwise dependences) - as in statistical learning - or because they capture the…
Simplicity is a critical inductive bias for designing data-driven controllers, especially when robustness is important. Despite the impressive results of deep reinforcement learning in complex control tasks, it is prone to capturing…
Plotting a learner's average performance against the number of training samples results in a learning curve. Studying such curves on one or more data sets is a way to get to a better understanding of the generalization properties of this…
Saliency Map, the gradient of the score function with respect to the input, is the most basic technique for interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were…
The idea is advanced that self-organization in complex systems can be treated as decision making (as it is performed by humans) and, vice versa, decision making is nothing but a kind of self-organization in the decision maker nervous…
A key challenge in machine learning is to explain how learning dynamics select among the many solutions that achieve identical loss values in overparameterized models - a phenomenon known as implicit bias. Controlling this bias provides a…
A composite likelihood is an inference function derived by multiplying a set of likelihood components. This approach provides a flexible framework for drawing inference when the likelihood function of a statistical model is computationally…
We briefly introduce noise-based logic. After describing the main motivations we outline classical, instantaneous (squeezed and non-squeezed), continuum, spike and random-telegraph-signal based schemes with applications such as circuits…
Communication signals often comprise an array of colors, lines, spots, notes or odors that are arranged in complex patterns, melodies or blends. Receiver perception is assumed to influence preference and thus the evolution of signal design,…
The aim of this Lecture Note is to introduce the Signal Processing (SP) community to a powerful yet still under-utilised tool: the semiparametric statistics. In short, the semiparametric framework allows us to estimate or perform hypothesis…
Noise is conventionally viewed as a severe problem in diverse fields, e.g., engineering, learning systems. However, this paper aims to investigate whether the conventional proposition always holds. It begins with the definition of task…
This paper reports, by way of introduction, on the advances made by our group and the broader signal processing community on the concept of sample abundance; a phenomenon that naturally arises in one-bit and few-bit signal processing…
Recent research on the dynamics of certain fluid dynamical instabilities shows that when there is a slow invariant manifold subject to fast timescale instability the dynamics are extremely sensitive to noise. The behaviour of such systems…
Applying chaos theory for secure digital communications is promising and it is well acknowledged that in such applications the underlying chaotic systems should be carefully chosen. However, the requirements imposed on the chaotic systems…
The lottery ticket hypothesis (LTH) has attracted attention because it can explain why over-parameterized models often show high generalization ability. It is known that when we use iterative magnitude pruning (IMP), which is an algorithm…
We address the problem of the relative importance of the intrinsic chaos and the external noise in determining the complexity of population dynamics. We use a recently proposed method for studying the complexity of nonlinear random…
Real-world networks are neither regular nor random, a fact elegantly explained by mechanisms such as the Watts-Strogatz or the Barabasi-Albert models, among others. Both mechanisms naturally create shortcuts and hubs, which while enhancing…