相关论文: The Limits of Mathematics---Tutorial Version
I'll outline the latest version of my limits of math course. The purpose of this course is to illustrate the proofs of the key information-theoretic incompleteness theorems of algorithmic information theory by means of algorithms written in…
This is an alternative version of the course notes in chao-dyn/9407003. The previous version is based on measuring the size of lisp s-expressions. This version is based on measuring the size of what I call lisp m-expressions, which are lisp…
This is yet another version of the course notes in chao-dyn/9407003. Here we change the universal Turing machine that is used to measure program-size complexity so that the constants in our information-theoretic incompleteness theorems are…
A remarkable new definition of a self-delimiting universal Turing machine is presented that is easy to program and runs very quickly. This provides a new foundation for algorithmic information theory. This new universal Turing machine is…
This is a shortened version of "The Limits of Mathematics--Course Outline & Software" (IBM Research Report RC 19324, December 1993) in which all Mathematica code has either been deleted or, if absolutely necessary, replaced by C code. The…
The proof of information inequalities and identities under linear constraints on the information measures is an important problem in information theory. For this purpose, ITIP and other variant algorithms have been developed and…
The paper elaborates an endeavor on applying the algorithmic information-theoretic computational complexity to meta-social-sciences. It is motivated by the effort on seeking the impact of the well-known incompleteness theorem to the…
This book is the final version of a course on algorithmic information theory and the epistemology of mathematics and physics. This is camera-ready copy prepared for publication as a book, but at the last minute I decided to publish it…
We consider the feasibility problem of integer linear programming (ILP). We show that solutions of any ILP instance can be naturally represented by an FO-definable class of graphs. For each solution there may be many graphs representing it.…
We consider a generalization of an important class of high-dimensional inference problems, namely spiked symmetric matrix models, often used as probabilistic models for principal component analysis. Such paradigmatic models have recently…
We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset.…
A numerical method is developed to solve linear semi-infinite programming problem (LSIP) in which the iterates produced by the algorithm are feasible for the original problem. This is achieved by constructing a sequence of standard linear…
Proving linear inequalities and identities of Shannon's information measures, possibly with linear constraints on the information measures, is an important problem in information theory. For this purpose, ITIP and other variant algorithms…
This paper establishes a theoretical foundation for understanding the fundamental limits of AI explainability through algorithmic information theory. We formalize explainability as the approximation of complex models by simpler ones,…
Formally verifying the correctness of mathematical proofs is more accessible than ever, however, the learning curve remains steep for many of the state-of-the-art interactive theorem provers (ITP). Deriving the most appropriate subsequent…
Incompleteness theorems of Godel, Turing, Chaitin, and Algorithmic Information Theory have profound epistemological implications. Incompleteness limits our ability to ever understand every observable phenomenon in the universe.…
Information theory plays a central role in establishing fundamental limits on what any learning or estimation algorithm can -- and cannot -- achieve, regardless of computational power. In this chapter, we provide an introduction to these…
We describe some "unrestricted" algorithms which are useful for the computation of elementary and special functions when the precision required is not known in advance. Several general classes of algorithms are identified and illustrated by…
Many probabilistic inference tasks involve summations over exponentially large sets. Recently, it has been shown that these problems can be reduced to solving a polynomial number of MAP inference queries for a model augmented with randomly…
Constraints on entropies are considered to be the laws of information theory. Even though the pursuit of their discovery has been a central theme of research in information theory, the algorithmic aspects of constraints on entropies remain…