Related papers: Value iteration for approximate dynamic programmin…

We analyze the infinite horizon minimax average cost Markov Control Model (MCM), for a class of controlled process conditional distributions, which belong to a ball, with respect to total variation distance metric, centered at a known…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior. We first consider the case in which…

The problem of synthesizing an optimal sensor selection policy is pertinent to a variety of engineering applications ranging from event detection to autonomous navigation. We consider such a synthesis problem over an infinite time horizon…

We study a general convex optimization problem, which covers various classic problems in different areas and particularly includes many optimal transport related problems arising in recent years. To solve this problem, we revisit the…

We prove limit theorems for the number of fixed points, descents, and inversions of iterated random-to-top shuffles in two asymptotic regimes. Our proofs are analytic, and they utilize new combinatorial decompositions that represent each…

This work introduces a new abstraction technique for reducing the state space of large, discrete-time labelled Markov chains. The abstraction leverages the semantics of interval Markov decision processes and the existing notion of…

This paper considers robust Markov decision processes under parametric transition distributions. We assume that the true transition distribution is uniquely specified by some parametric distribution, and explicitly enforce that the…

We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov…

This paper presents a multinomial method for option pricing when the underlying asset follows an exponential Variance Gamma process. The continuous time Variance Gamma process is approximated by a discrete time Markov chain with the same…

Value Iteration (VI) is foundational to the theory and practice of modern reinforcement learning, and it is known to converge at a $\mathcal{O}(\gamma^k)$-rate, where $\gamma$ is the discount factor. Surprisingly, however, the optimal rate…

Distributional reinforcement learning improves performance by capturing environmental stochasticity, but a comprehensive theoretical understanding of its effectiveness remains elusive. In addition, the intractable element of the infinite…

We consider the task of estimating a structural model of dynamic decisions by a human agent based upon the observable history of implemented actions and visited states. This problem has an inherent nested structure: in the inner problem, an…

We investigate the adaptive robust control framework for portfolio optimization and loss-based hedging under drift and volatility uncertainty. Adaptive robust problems offer many advantages but require handling a double optimization problem…

In this paper, we consider the problem of invariant set computation for black-box switched linear systems using merely a finite set of observations of system trajectories. In particular, this paper focuses on polyhedral invariant sets. We…

We present the convergence rates of synchronous and asynchronous Q-learning for average-reward Markov decision processes, where the absence of contraction poses a fundamental challenge. Existing non-asymptotic results overcome this…

We consider perturbed nonlinear ill-posed equations in Hilbert spaces, with operators that are monotone on a given closed convex subset. A simple stable approach is Lavrentiev regularization, but existence of solutions of the regularized…

In this paper, the Mean value iterative process is modified with the Mann iterative process for mean nonexpansive mapping in a hyperbolic metric space that satisfy the symmetry criteria and in uniformly convex hyperbolic spaces to validate…

We present an efficient robust value iteration for \texttt{s}-rectangular robust Markov Decision Processes (MDPs) with a time complexity comparable to standard (non-robust) MDPs which is significantly faster than any existing method. We do…

Processes (MDPs) often require frequent decision making, that is, taking an action every microsecond, second, or minute. Infinite horizon discount reward formulation is still relevant for a large portion of these applications, because…