Related papers: Two Timescale Stochastic Approximation with Contro…
We consider a class of stochastic optimal control problems with partial observation, and study their approximation by discrete-time control problems. We establish a convergence result by using weak convergence technique of Kushner and…
This paper presents a novel model-free and fully data-driven policy iteration scheme for quadratic regulation of linear dynamics with state- and input-multiplicative noise. The implementation is similar to the least-squares temporal…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…
This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale…
The objective of the paper is to identify and investigate all possible types of asymptotic behavior for the maximum likelihood estimators of the unknown parameters in the second-order linear stochastic ordinary differential equation driven…
We propose a formulation for approximate constrained nonlinear output-feedback stochastic model predictive control. Starting from the ideal but intractable stochastic optimal control problem (OCP), which involves the optimization over…
Autonomous systems often have logical constraints arising, for example, from safety, operational, or regulatory requirements. Such constraints can be expressed using temporal logic specifications. The system state is often partially…
We study the estimation of the value function for continuous-time Markov diffusion processes using a single, discretely observed ergodic trajectory. Our work provides non-asymptotic statistical guarantees for the least-squares…
In this paper, we study the finite-sample statistical rates of distributional temporal difference (TD) learning with linear function approximation. The purpose of distributional TD learning is to estimate the return distribution of a…
We introduce and validate a machine-learning assisted quantum sensing protocol to classify spatial and temporal correlations of classical noise affecting two ultrastrongly coupled qubits. We consider six distinct classes of Markovian and…
Previous studies on two-timescale stochastic approximation (SA) mainly focused on bounding mean-squared errors under diminishing stepsize schemes. In this work, we investigate {\it constant} stpesize schemes through the lens of Markov…
Convergence properties of model-free two-timescale asymptotic simulations of singularly perturbed hybrid inclusions are developed. A hybrid inclusion combines constrained differential and difference inclusions to capture continuous (flow)…
This article proposes an improved trajectory optimization approach for stochastic optimal control of dynamical systems affected by measurement noise by combining optimal control with maximum likelihood techniques to improve the reduction of…
A new formulation of Stochastic Model Predictive Output Feedback Control is presented and analyzed as a translation of Stochastic Optimal Output Feedback Control into a receding horizon setting. This requires lifting the design into a…
We develop a formalism to describe the discrete-time dynamics of systems containing an arbitrary number of interacting species. The individual-based model, which forms our starting point, is described by a Markov chain, which in the limit…
We propose and analyze an alternate approach to off-policy multi-step temporal difference learning, in which off-policy returns are corrected with the current Q-function in terms of rewards, rather than with the target policy in terms of…
In unsupervised causal representation learning for sequential data with time-delayed latent causal influences, strong identifiability results for the disentanglement of causally-related latent variables have been established in stationary…
We consider spatially extended conductance based neuronal models with noise described by a stochastic reaction diffusion equation with additive noise coupled to a control variable with multiplicative noise but no diffusion. We only assume a…
In this paper we present a Learning Model Predictive Control (LMPC) strategy for linear and nonlinear time optimal control problems. Our work builds on existing LMPC methodologies and it guarantees finite time convergence properties for the…
We study the policy evaluation problem in multi-agent reinforcement learning where a group of agents, with jointly observed states and private local actions and rewards, collaborate to learn the value function of a given policy via local…