English

Entropy-Rate Selection for Partially Observed Processes

Information Theory 2026-04-14 v1 Econometrics math.IT Probability Statistics Theory Statistics Theory

Abstract

I formulate an entropy-rate maximization problem at the observable level for stochastic processes observed through an information-reducing observation map. For a visible stationary law, the map determines an observational fiber of hidden stationary laws generating that law. In the finite-state finite-memory setting, retained visible constraints determine a feasible class of stationary (r+1)(r+1)-block laws, and the entropy maximizer is defined as the entropy-rate maximizer on this class. The paper formulates entropy-rate maximization on feasible classes induced by partial observability and develops a structural theory for the resulting maximizer. I prove existence and uniqueness of the maximizer, with uniqueness under a fixed-context-marginal hypothesis and, more generally, via a strict-concavity characterization by row proportionality. Two global characterization regimes are central: a fixed one-point marginal yields the i.i.d. maximizer, and a fixed rr-block law yields the (r1)(r-1)-step Markov extension. The gap functional equals a conditional mutual information and vanishes exactly at the maximizing completion. I also derive optimality conditions, local geometry of the maximizer, a latent random-mapping realization that leaves the visible law unchanged, and a local empirical consistency theorem, and illustrate the framework by an aliased hidden-state example.

Keywords

Cite

@article{arxiv.2604.10752,
  title  = {Entropy-Rate Selection for Partially Observed Processes},
  author = {Oleg Kiriukhin},
  journal= {arXiv preprint arXiv:2604.10752},
  year   = {2026}
}
R2 v1 2026-07-01T12:05:13.045Z