Related papers: Stopping problems with an unknown state
The halting problem is undecidable --- but can it be solved for "most" inputs? This natural question was considered in a number of papers, in different settings. We revisit their results and show that most of them can be easily proven in a…
In this work, we address the output--feedback control problem for nonlinear systems under bounded disturbances using a moving horizon approach. The controller is posed as an optimization-based problem that simultaneously estimates the state…
Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…
We study optimal stopping for diffusion processes with unknown model primitives within the continuous-time reinforcement learning (RL) framework developed by Wang et al. (2020), and present applications to option pricing and portfolio…
There are two major models of value uncertainty in the optimal stopping literature: the secretary model, which assumes no prior knowledge, and the prophet inequality model, which assumes full information about value distributions. In…
This paper addresses the synthesis of interval observers for partially unknown nonlinear systems subject to bounded noise, aiming to simultaneously estimate system states and learn a model of the unknown dynamics. Our approach leverages…
State estimation plays a key role in the transition from the passive to the active operation of distribution systems, as it allows to monitor these networks and, successively, to perform control actions. However, designing state estimators…
The problem of real-time remote tracking and reconstruction of a two-state Markov process is considered here. A transmitter sends samples from an observed information source to a remote monitor over an unreliable wireless channel. The…
We consider a class of optimal control problems, with finite or infinite horizon, for a continuous-time Markov chain with finite state space. In this case, the control process affects the transition rates. We suppose that the controlled…
The optimal zero delay coding of a finite state Markov source is considered. The existence and structure of optimal codes are studied using a stochastic control formulation. Prior results in the literature established the optimality of…
In many reinforcement learning (RL) applications, we want policies that reach desired states and then keep the controlled system within an acceptable region around the desired states over an indefinite period of time. This latter objective…
Methods for learning and planning in sequential decision problems often assume the learner is aware of all possible states and actions in advance. This assumption is sometimes untenable. In this paper, we give a method to learn factored…
This paper combines data-driven and model-driven methods for real-time misinformation detection. Our algorithm, named QuickStop, is an optimal stopping algorithm based on a probabilistic information spreading model obtained from labeled…
We present a new stochastic framework for studying ship capsize. It is a synthesis of two strands of transition state theory. The first is an extension of deterministic transition state theory to dissipative non-autonomous systems, together…
We investigate a remote estimation problem in which a transmitter observes a Markov source and chooses the power level to transmit it over a time-varying packet-drop channel. The channel is modeled as a channel with Markovian state where…
This paper investigates the state estimation problem for unknown linear systems subject to both process and measurement noise. Based on a prior input-output trajectory sampled at a higher frequency and a prior state trajectory sampled at a…
We consider the challenge of finding a deterministic policy for a Markov decision process that uniformly (in all states) maximizes one reward subject to a probabilistic constraint over a different reward. Existing solutions do not fully…
In this paper, problems of optimal control are considered where in the objective function, in addition to the control cost there is a tracking term that measures the distance to a desired stationary state. The tracking term is given by some…
We give answer to an open question by proving a sufficient optimality condition for state-linear optimal control problems with time delays in state and control variables. In the proof of our main result, we transform a delayed state-linear…
In this paper, we study an optimal stopping problem in the presence of model uncertainty and regime switching. The max-min formulation for robust control and the dynamic programming approach are adopted to establish a general theoretical…