Related papers: Calculation of the minimum computational complexit…
The possibility to save and process information in fundamentally indistinguishable states is the quantum mechanical resource that is not encountered in classical computing. I demonstrate that, if energy constraints are imposed, this…
Characterizing the entropy of a system is a crucial, and often computationally costly, step in understanding its thermodynamics. It plays a key role in the study of phase transitions, pattern formation, protein folding and more. Current…
We illustrate how computer-aided methods can be used to investigate the fundamental limits of the caching systems, which are significantly different from the conventional analytical approach usually seen in the information theory…
We study a dynamical system with time dependent Hamiltonian by numerical experiments so as to find a relation between thermodynamics and chaotic nature of the system. Excess information loss, defined newly based on Lyapunov analysis, is…
Information theory, though originally developed for communications engineering, provides mathematical tools with broad applications across science. These tools characterize the fundamental limits of data compression and transmission in the…
Computational entropies provide a framework for quantifying uncertainty and randomness under computational constraints. They play a central role in classical cryptography, underpinning the analysis and construction of primitives such as…
The concept of Shannon entropy of random variables was generalized to measurable functions in general, and to simple functions with finite values in particular. It is shown that the information measure of a function is related to the time…
Dynamic processes on networks are fundamental to understanding modern-day phenomena such as information diffusion and opinion polarization on the internet or epidemics spreading through society. However, such processes are notoriously…
We consider the problem of succinctly encoding a static map to support approximate queries. We derive upper and lower bounds on the space requirements in terms of the error rate and the entropy of the distribution of values over keys: our…
Within the task of collaborative filtering two challenges for computing conditional probabilities exist. First, the amount of training data available is typically sparse with respect to the size of the domain. Thus, support for higher-order…
The determination of environmentally- and economically-optimal energy system designs and operations is complex. In particular, the integration of weather-dependent renewable energy technologies into energy system optimization models…
We pedagogically present the information theory as originally established, explaining its essential ideas and paying attention to the expression employed to measure the amount of information. Also we discussed relationships between…
A unified combinatorial definition of the information content and entropy of different types of patterns, compatible with the traditional concepts of information and entropy, going beyond the limitations of Shannon information interpretable…
We propose a measure of learning efficiency for non-finite state spaces. We characterize the complexity of a learning problem by the metric entropy of its state space. We then describe how learning efficiency is determined by this measure…
There are several forms of irreducibility in computing systems, ranging from undecidability to intractability to nonlinearity. This paper is an exploration of the conceptual issues that have arisen in the course of investigating speed-up…
Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension…
In the 21st century, many of the crucial scientific and technical issues facing humanity can be understood as problems associated with understanding, modelling, and ultimately controlling complex systems: systems comprised of a large number…
The relationship between entropy and information is reviewed, taking into account that information is stored in macroscopic degrees of freedom, such as the order parameter in a system exhibiting spontaneous symmetry breaking. It is shown…
We study the build up of complexity on the example of 1 kg matter in different forms. We start on the simplest example of ideal gases, and then continue with more complex chemical, biological, life and social and technical structures. We…
Current state-of-the-art solvers for mixed-integer programming (MIP) problems are designed to perform well on a wide range of problems. However, for many real-world use cases, problem instances come from a narrow distribution. This has…