Related papers: Fitness and Overfitness: Implicit Regularization i…
In evolutionary algorithms, the fitness of a population increases with time by mutating and recombining individuals and by a biased selection of more fit individuals. The right selection pressure is critical in ensuring sufficient…
In deep learning, often the training process finds an interpolator (a solution with 0 training loss), but the test loss is still low. This phenomenon, known as benign overfitting, is a major mystery that received a lot of recent attention.…
We introduce a general framework for analyzing learning algorithms based on the notion of self-regularization, which captures implicit complexity control without requiring explicit regularization. This is motivated by previous observations…
Aging is a fundamental aspect of living systems that undergo a progressive deterioration of physiological function with age and an increase of vulnerability to disease and death. Living systems, known as complex systems, require complexity…
By training linear physical networks to learn linear transformations, we discern how their physical properties evolve due to weight update rules. Our findings highlight a striking similarity between the learning behaviors of such networks…
A major aim of evolutionary biology is to explain the respective roles of adaptive versus non-adaptive changes in the evolution of complexity. While selection is certainly responsible for the spread and maintenance of complex phenotypes,…
This chapter is about Complexity and Spatial Dynamics in Urban Systems. Strong inequalities in the size of cities and the apparent difficulty of limiting their growth raise practical issues for spatial planning. At a time when new…
Learning how to learn efficiently is a fundamental challenge for biological agents and a growing concern for artificial ones. To learn effectively, an agent must regulate its learning speed, balancing the benefits of rapid improvement…
We introduce the framework of performative reinforcement learning where the policy chosen by the learner affects the underlying reward and transition dynamics of the environment. Following the recent literature on performative…
Biological and social systems are structured at multiple scales, and the incentives of individuals who interact in a group may diverge from the collective incentive of the group as a whole. Mechanisms to resolve this tension are responsible…
The process of calibrating computer models of natural phenomena is essential for applications in the physical sciences, where plenty of domain knowledge can be embedded into simulations and then calibrated against real observations. Current…
In this work, we integrate theoretical modeling, molecular simulation, and empirical analysis to identify and characterize evolutionary hysteresis. We first show how epistatic interactions create bistable fitness landscapes and structural…
Evolving systems, be it an antibody repertoire in the face of mutating pathogens or a microbial population exposed to varied antibiotics, constantly search for adaptive solutions in time-varying fitness landscapes. Generalists correspond to…
We introduce a new notion of complexity of functions and we show that it has the following properties: (i) it governs a PAC Bayes-like generalization bound, (ii) for neural networks it relates to natural notions of complexity of functions…
The relationship between intelligence and evolution is bidirectional: while evolution can help evolve intelligences, the degree of intelligence itself can impact evolution (Baldwin, 1896). In the field of Evolutionary Computation, the…
The interaction networks of biological systems are known to take on several non-random structural properties, some of which are believed to positively influence system robustness. Researchers are only starting to understand how these…
In an attempt to better understand generalization in deep learning, we study several possible explanations. We show that implicit regularization induced by the optimization method is playing a key role in generalization and success of deep…
In evolutionary algorithms, the fitness of a population increases with time by mutating and recombining individuals and by a biased selection of more fit individuals. The right selection pressure is critical in ensuring sufficient…
We consider evolution of a large population, where fitness of each organism is defined by many phenotypical traits. These traits result from expression of many genes. We propose a new model of gene regulation, where gene expression is…
Model regularization requires extensive manual tuning to balance complexity against overfitting. Cross-regularization resolves this tradeoff by directly adapting regularization parameters through validation gradients during training. The…