相关论文: Approximation of the Two-Part MDL Code

The Minimum Description Length (MDL) principle is solidly based on a provably ideal method of inference using Kolmogorov complexity. We test how the theory behaves in practice on a general problem in model selection: that of learning the…

In the Minimum Description Length (MDL) principle, learning from the data is equivalent to an optimal coding problem. We show that the codes that achieve optimal compression in MDL are critical in a very precise sense. First, when they are…

The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description…

It is reasonable to expect the theory of quantum codes to be simplified in the case of codes of minimum distance 2; thus, it makes sense to examine such codes in the hopes that techniques that prove effective there will generalize. With…

Binary optimal codes often contain optimal or near-optimal subcodes. In this paper we show that this is true for the family of self-dual codes. One approach is to compute the optimum distance profiles (ODPs) of linear codes, which was…

We study the properties of the Minimum Description Length principle for sequence prediction, considering a two-part MDL estimator which is chosen from a countable class of models. This applies in particular to the important case of…

Is it possible to find a shortest description for a binary string? The well-known answer is "no, Kolmogorov complexity is not computable." Faced with this barrier, one might instead seek a short list of candidates which includes a laconic…

Approximate linear programming (ALP) is an efficient approach to solving large factored Markov decision processes (MDPs). The main idea of the method is to approximate the optimal value function by a set of basis functions and optimize…

The coding theorem for Kolmogorov complexity states that any string sampled from a computable distribution has a description length close to its information content. A coding theorem for resource-bounded Kolmogorov complexity is the key to…

Algebraic methods for the design of series of maximum distance separable (MDS) linear block and convolutional codes to required specifications and types are presented. Algorithms are given to design codes to required rate and required…

We leverage the Minimum Description Length (MDL) principle as a model selection technique for Bernoulli distributions and compare several types of MDL codes. We first present a simplistic crude two-part MDL code and a Normalized Maximum…

The construction of Maximum Distance Profile (MDP) convolutional codes in general requires the use of very large finite fields. In contrast convolutional codes with optimal column distances maximize the column distances for a given…

Minimum Description Length (MDL) estimators, using two-part codes for universal coding, are analyzed. For general parametric families under certain regularity conditions, we introduce a two-part code whose regret is close to the minimax…

Linear codes are widely studied in coding theory as they have nice applications in distributed storage, combinatorics, lattices, cryptography and so on. Constructing linear codes with desirable properties is an interesting research topic.…

In the signal processing and statistics literature, the minimum description length (MDL) principle is a popular tool for choosing model complexity. Successful examples include signal denoising and variable selection in linear regression,…

We study computational and statistical aspects of learning Latent Markov Decision Processes (LMDPs). In this model, the learner interacts with an MDP drawn at the beginning of each epoch from an unknown mixture of MDPs. To sidestep known…

We provide a complete characterization of the entire regularization curve of a modified two-part-code Minimum Description Length (MDL) learning rule for binary classification, based on an arbitrary prior or description language. Grunwald…

Maximum distance separable (MDS) are constructed to required specifications. The codes are explicitly given over finite fields with efficient encoding and decoding algorithms. Series of such codes over finite fields with ratio of distance…

We consider partially observable Markov decision processes (POMDPs) with a set of target states and every transition is associated with an integer cost. The optimization objective we study asks to minimize the expected total cost till the…

Locally recoverable codes are widely used in distributed and cloud storage systems. The objective of this paper is to present a construction of near MDS codes with oval polynomials and then determine the locality of the codes. It turns out…