相关论文: Some basic information on information-based comple…
When considering a specific event, news that accurately reflects the ground truth is deemed as real information, while news that deviates from the ground truth is classified as false information. False information often spreads fast due to…
`All models are wrong but some are useful' (George Box 1979). But, how to find those useful ones starting from an imperfect model? How to make informed data-driven decisions equipped with an imperfect model? These fundamental questions…
The complexity of a system, in general, makes it difficult to determine some or almost all matrix elements of its operators. The lack of accuracy acts as a source of randomness for the matrix elements which are also subjected to an external…
With the advent of high-performance computing, Bayesian methods are increasingly popular tools for the quantification of uncertainty throughout science and industry. Since these methods impact the making of sometimes critical decisions in…
Insightful interdisciplinary collaboration is essential to the principled governance of technology. When such efforts address the interaction between computation and society, they often focus on modeling, the process by which computer…
The paper "Is Complexity an Illusion?" (Bennett, 2024) provides a formalism for complexity, learning, inference, and generalization, and introduces a formal definition for a "policy". This reply shows that correct policies do not exist for…
Modern statisticians are often presented with hundreds or thousands of hypothesis testing problems to evaluate at the same time, generated from new scientific technologies such as microarrays, medical and satellite imaging devices, or flow…
Item Response Theory becomes an increasingly important tool when analyzing ``Big Data'' gathered from online educational venues. However, the mechanism was originally developed in traditional exam settings, and several of its assumptions…
This chapter does not deal with specific tools and techniques for managing complex systems, but proposes some basic concepts that help us to think and speak about complexity. We review classical thinking and its intrinsic drawbacks when…
The application of rough set theory in incomplete information systems is a key problem in practice since missing values almost always occur in knowledge acquisition due to the error of data measuring, the limitation of data collection, or…
Information Bottleneck (IB) is a technique to extract information about one target random variable through another relevant random variable. This technique has garnered significant interest due to its broad applications in information…
The progress of machine learning over the past decade is undeniable. In retrospect, it is both remarkable and unsettling that this progress was achievable with little to no rigorous theory to guide experimentation. Despite this fact,…
If a concept is not well defined, there are grounds for its abuse. This is particularly true of complexity, an inherently interdisciplinary concept that has penetrated very different fields of intellectual activity from physics to…
Previous theoretical results pertaining to meta-learning on sequences build on contrived assumptions and are somewhat convoluted. We introduce new information-theoretic tools that lead to an elegant and very general decomposition of error…
All natural things process and transform information. They receive environmental information as input, and transform it into appropriate output responses. Much of science is dedicated to building models of such systems -- algorithmic…
The outcome of all time series cannot be forecast, e.g. the flipping of a fair coin. Others, like the repeated {01} sequence {010101...} can be forecast exactly. Algorithmic information theory can provide a measure of forecastability that…
Circuit discovery has recently attracted attention as a potential research direction to explain the non-trivial behaviors of language models. It aims to find the computational subgraphs, also known as circuits, within the model that are…
In this position paper, I first describe a new perspective on machine learning (ML) by four basic problems (or levels), namely, "What to learn?", "How to learn?", "What to evaluate?", and "What to adjust?". The paper stresses more on the…
How best to quantify the information of an object, whether natural or artifact, is a problem of wide interest. A related problem is the computability of an object. We present practical examples of a new way to address this problem. By…
Finding the most probable explanation for observed variables in a Bayesian network is a notoriously intractable problem, particularly if there are hidden variables in the network. In this paper we examine the complexity of a related…