Related papers: Emergence of functional information from multivari…

Sequence alignments are used to capture patterns composed of elements representing multiple conceptual levels through the alignment of sequences that contain overlapping and variable length annotations. The alignments also determine the…

Significant work has been done on learning regular expressions from a set of data values. Depending on the domain, this approach can be very successful. However, significant time is required to learn these expressions and the resulting…

The information conveyed by a hierarchical attractor neural network is examined. The network learns sets of correlated patterns (the examples) in the lowest level of the hierarchical tree and can categorize them at the upper levels. A way…

The output of a machine learning algorithm can usually be represented by one or more multivariate functions of its input variables. Knowing the global properties of such functions can help in understanding the system that produced the data…

Predictions about people, such as their expected educational achievement or their credit risk, can be performative and shape the outcome that they aim to predict. Understanding the causal effect of these predictions on the eventual outcomes…

Many neural nets appear to represent data as linear combinations of "feature vectors." Algorithms for discovering these vectors have seen impressive recent success. However, we argue that this success is incomplete without an understanding…

Knowledge of the association information between the attributes in a data set provides insight into the underlying structure of the data and explains the relationships (independence, synergy, redundancy) between the attributes and class (if…

A central question in neuroscience is to understand how noisy firing patterns are used to transmit information. Because neural spiking is noisy, spiking patterns are often quantified via pairwise correlations, or the probability that two…

Spike patterns have been reported to encode sensory information in several brain areas. Here we assess the role of specific patterns in the neural code, by comparing the amount of information transmitted with different choices of the…

Traditional economic models typically treat private information, or signals, as generated from some underlying state. Recent work has explicated alternative models, where signals correspond to interpretations of available information. We…

The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by…

Machine learning is rapidly making its path into natural sciences, including high-energy physics. We present the first study that infers, directly from experimental data, a functional form of fragmentation functions. The latter represent a…

Written language is a complex communication signal capable of conveying information encoded in the form of ordered sequences of words. Beyond the local order ruled by grammar, semantic and thematic structures affect long-range patterns in…

A modelling language is described which is suitable for the correlation of information when the underlying functional model of the system is incomplete or uncertain and the temporal dependencies are imprecise. An efficient and incremental…

Multivariate time series analysis is extensively used in neurophysiology with the aim of studying the relationship between simultaneously recorded signals. Recently, advances on information theory and nonlinear dynamical systems theory have…

Form a pure mathematical point of view, common functional forms representing different physical phenomena can be defined. For example, rates of chemical reactions, diffusion and heat transfer are all governed by exponential-type…

We study the capacity with which a system of independent neuron-like units represents a given set of stimuli. We assume that each neuron provides a fixed amount of information, and that the information provided by different neurons has a…

Neural networks encode information through their collective spiking activity in response to external stimuli. This population response is noisy and strongly correlated, with complex interplay between correlations induced by the stimulus,…

Large language models (LLMs) are trained on enormous amounts of data and encode knowledge in their parameters. We propose a pipeline to elicit causal relationships from LLMs. Specifically, (i) we sample many documents from LLMs on a given…

Multivariate functions emerge naturally in a wide variety of data-driven models. Popular choices are expressions in the form of basis expansions or neural networks. While highly effective, the resulting functions tend to be hard to…