Related papers: Ranking-based rich-get-richer processes
Markov decision processes are typically used for sequential decision making under uncertainty. For many aspects however, ranging from constrained or safe specifications to various kinds of temporal (non-Markovian) dependencies in task and…
We consider $N$ counters taking integer values which are subject to the following dynamics. At every time, a pair of distinct counters is chosen uniformly at random and their states are updated according to the following rule. If the states…
We consider irreversible Markov chains on finite commutative rings randomly generated using both addition and multiplication. We restrict ourselves to the case where the addition is uniformly random and multiplication is arbitrary. We first…
Distributions over rankings are used to model data in various settings such as preference analysis and political elections. The factorial size of the space of rankings, however, typically forces one to make structural assumptions, such as…
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
One of the most used priors in Bayesian clustering is the Dirichlet prior. It can be expressed as a Chinese Restaurant Process. This process allows nonparametric estimation of the number of clusters when partitioning datasets. Its key…
Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…
Structural results impose sufficient conditions on the model parameters of a Markov decision process (MDP) so that the optimal policy is an increasing function of the underlying state. The classical assumptions for MDP structural results…
For a sequence in discrete time having stationary independent values (respectively, random walk) $X$, those random times $R$ of $X$ are characterized set-theoretically, for which the strict post-$R$ sequence (respectively, the process of…
We describe a Markov latent state space (MLSS) model, where the latent state distribution is a decaying mixture over multiple past states. We present a simple sampling algorithm that allows to approximate such high-order MLSS with fixed…
We consider the stochastic ranking process with space-time dependent unbounded jump rates for the particles. We prove that the joint empirical distribution of jump rate and scaled position converges almost surely to a deterministic…
The deterministic dynamics of randomly connected neural networks are studied, where a state of binary neurons evolves according to a discreet-time synchronous update rule. We give a theoretical support that the overlap of systems' states…
Ulam has defined a history-dependent random sequence of integers by the recursion $X_{n+1}$ $= X_{U(n)}+X_{V(n)}, n \geqslant r$ where $U(n)$ and $V(n)$ are independently and uniformly distributed on $\{1,\dots,n\}$, and the initial…
This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior. We first consider the case in which…
We consider finite horizon Markov decision processes under performance measures that involve both the mean and the variance of the cumulative reward. We show that either randomized or history-based policies can improve performance. We prove…
Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic…
The ranking problem is to order a collection of units by some unobserved parameter, based on observations from the associated distribution. This problem arises naturally in a number of contexts, such as business, where we may want to rank…
Let $\mathbf{X}(n) \in \mathbb{R}^d$ be a sequence of random vectors, where $n\in\mathbb{N}$ and $d = d(n)$. Under certain weakly dependence conditions, we prove that the distribution of the maximal component of $\mathbf{X}$ and the…
We consider the time dependent probability distribution of a coarse grained observable Y whose evolution is governed by a discrete time map. If the map is mixing, the time dependent one-step transition probabilities converge in the long…
This work investigates the almost sure stabilization of a class of regime-switching systems based on discrete-time observations of both continuous and discrete components. It develops Shao's work [SIAM J. Control Optim., 55(2017), pp.…