Related papers: Information Complexity and Estimation
As is the case of many signals produced by complex systems, language presents a statistical structure that is balanced between order and disorder. Here we review and extend recent results from quantitative characterisations of the degree of…
This paper considers a sequential estimation and sensor scheduling problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process, and makes a decision as…
High-throughput data analyses are becoming common in biology, communications, economics and sociology. The vast amounts of data are usually represented in the form of matrices and can be considered as knowledge networks. Spectra-based…
We consider the problem of distributed estimation, where local processors observe independent samples conditioned on a common random parameter of interest, map the observations to a finite number of bits, and send these bits to a remote…
We examine the influence of input data representations on learning complexity. For learning, we posit that each model implicitly uses a candidate model distribution for unexplained variations in the data, its noise model. If the model…
In this paper the complex-valued best linear unbiased estimator of an unknown constant mean of white noise was derived the ordinary least-squares estimator of an unknown constant mean of random field (arithmetic mean) charged by an…
We consider an unconstrained continuous optimization problem where, in each iteration, gradient estimates may be arbitrarily corrupted with a probability greater than 1/2. Additionally, function value estimates may exhibit heavy-tailed…
Intensity estimation for Poisson processes is a classical problem and has been extensively studied over the past few decades. Practical observations, however, often contain compositional noise, i.e. a nonlinear shift along the time axis,…
Concepts used in the scientific study of complex systems have become so widespread that their use and abuse has led to ambiguity and confusion in their meaning. In this paper we use information theory to provide abstract and concise…
In this thesis we aim to analyze and quantify the energetic and information contents that can be extracted from a dynamical system subject to the external environment. The latter is usually assumed to be deleterious for the feasibility of…
Herein we consider various concepts of entropy as measures of the complexity of phenomena and in so doing encounter a fundamental problem in physics that affects how we understand the nature of reality. In essence the difficulty has to do…
Natural language processing systems are often downstream of unreliable inputs: machine translation, optical character recognition, or speech recognition. For instance, virtual assistants can only answer your questions after understanding…
Noise-induced phase transitions are common in various complex systems, from physics to biology. In this article, we investigate the emergence of crucial events in noise-induced phase transition processes and their potential significance for…
Data informativity provides a theoretical foundation for determining whether collected data are sufficiently informative to achieve specific control objectives in data-driven control frameworks. In this study, we investigate the data…
For sensory networks, we determine the rate with which they acquire information about the changing external conditions. Comparing this rate with the thermodynamic entropy production that quantifies the cost of maintaining the network, we…
The field of information retrieval often works with limited and noisy data in an attempt to classify documents into subjective categories, e.g., relevance, sentiment and controversy. We typically quantify a notion of agreement to understand…
A modelling framework, based on the theory of signal processing, for characterising the dynamics of systems driven by the unravelling of information is outlined, and is applied to describe the process of decision making. The model input of…
The most natural way to describe an information-carrying system containing a specific noise is an additive white Gaussian-noise (AWGN) channel. In bosonic quantum systems (especially the Gaussian case), although the classical information…
Kernel random matrices have attracted a lot of interest in recent years, from both practical and theoretical standpoints. Most of the theoretical work so far has focused on the case were the data is sampled from a low-dimensional structure.…
It is not obvious what fraction of all the potential information residing in the molecules and structures of living systems is significant or meaningful to the system. Sets of random sequences or identically repeated sequences, for example,…