Related papers: Information Theory and Direction Selectivity
Identifying the underlying time-delayed latent causal processes in sequential data is vital for grasping temporal dynamics and making downstream reasoning. While some recent methods can robustly identify these latent causal variables, they…
Cells and organisms follow aligned structures in their environment, a process that can generate persistent migration paths. Kinetic transport equations are a popular modelling tool for describing biological movements at the mesoscopic…
In latent Gaussian trees the pairwise correlation signs between the variables are intrinsically unrecoverable. Such information is vital since it completely determines the direction in which two variables are associated. In this work, we…
Data classification, the process of analyzing data and organizing it into categories, is a fundamental computing problem of natural and artificial information processing systems. Ideally, the performance of classifier models would be…
Why does time appear to pass irreversibly? To investigate, we introduce a class of partitioned cellular automata (PCAs) whose cellwise evolution is based on the chaotic baker's map. After imposing a suitable initial condition and…
We introduce an information-theoretic method for quantifying causality in chaotic systems. The approach, referred to as IT-causality, quantifies causality by measuring the information gained about future events conditioned on the knowledge…
We consider the problem of decision-making with side information and unbounded loss functions. Inspired by probably approximately correct learning model, we use a slightly different model that incorporates the notion of side information in…
Most tabular-data generators match marginal statistics yet ignore causal structure, leading downstream models to learn spurious or unfair patterns. We present TabSCM, a mixed-type generator that preserves those causal dependencies. Starting…
Investigation of the underlying physics or biology from empirical data requires a quantifiable notion of similarity - when do two observed data sets indicate nearly identical generating processes, and when they do not. The discriminating…
The uniform information density (UID) hypothesis proposes that speakers aim to distribute information evenly throughout a text, balancing production effort and listener comprehension difficulty. However, language typically does not maintain…
In this paper the theory of semi-bounded rationality is proposed as an extension of the theory of bounded rationality. In particular, it is proposed that a decision making process involves two components and these are the correlation…
Light-matter interactions inside turbid medium can be controlled by tailoring the spatial distribution of energy density throughout the system. Wavefront shaping allows selective coupling of incident light to different transmission…
Optical metrology has progressed beyond the Abbe-Rayleigh limit, unlocking (sub)atomic precision by leveraging nonlinear phenomena, statistical accumulation, and AI estimators trained on measurand variations. Here, we show that Fisher…
We show that a rate of conditional Shannon entropy reduction, characterizing the learning of an internal process about an external process, is bounded by the thermodynamic entropy production. This approach allows for the definition of an…
Neuroscientists formulate very different hypotheses about the nature of neural code. At one extreme, it has been argued that neurons encode information in relatively slow changes of individual spikes arriving "rates codes" and the…
Prior to receiving visual stimuli, spontaneous, correlated activity called retinal waves drives activity-dependent developmental programs. Early-stage waves mediated by acetylcholine (ACh) manifest as slow, spreading bursts of action…
We introduce a new criterion to determine the order of an autoregressive model fitted to time series data. It has the benefits of the two well-known model selection techniques, the Akaike information criterion and the Bayesian information…
Though image-to-sequence generation models have become overwhelmingly popular in human-computer communications, they suffer from strongly favoring safe generic questions ("What is in this picture?"). Generating uninformative but relevant…
Tabular data is more challenging to generate than text and images, due to its heterogeneous features and much lower sample sizes. On this task, diffusion-based models are the current state-of-the-art (SotA) model class, achieving almost…
Information causality (IC) was one of the first principles that have been invoked to bound the set of quantum correlations. For some families of correlations, this principle recovers exactly the boundary of the quantum set; for others,…