Related papers: Multi-step Uniformization with Steady-State Detect…
Cross-sectional observations from a dynamical system can be modeled via steady-state distributions of Markov processes. The major challenge is then to determine whether the process parameters can be identified and estimated from the…
This paper, the second of a two-part series, presents a method for mean-field feedback stabilization of a swarm of agents on a finite state space whose time evolution is modeled as a continuous time Markov chain (CTMC). The resulting…
We study a decentralized variant of stochastic approximation, a data-driven approach for finding the root of an operator under noisy measurements. A network of agents, each with its own operator and data observations, cooperatively find the…
Motivated by a class of applied problems arising from physical layer based security in a digital communication system, in particular, by a secrecy sum-rate maximization problem, this paper studies a nonsmooth, difference-of-convex (dc)…
Continuous Time Markov Chains (CTMC) have been used extensively to model reliability of storage systems. While the exponentially distributed sojourn time of Markov models is widely known to be unrealistic (and it is necessary to consider…
Markov state models (MSMs) have been demonstrated to be a powerful method for computationally studying intramolecular processes such as protein folding and macromolecular conformational changes. In this article, we present a new approach to…
For a discrete time Markov chain and in line with Strotz' consistent planning we develop a framework for problems of optimal stopping that are time-inconsistent due to the consideration of a non-linear function of an expected reward. We…
Most studies in real time change-point detection either focus on the linear model or use the CUSUM method under classical assumptions on model errors. This paper considers the sequential change-point detection in a nonlinear quantile model.…
We investigate the stochastic transfer synchronization problem, which seeks to synchronize the timetables of different routes in a transit network to reduce transfer waiting times, delay times, and unnecessary in-vehicle times. We present a…
Markov chain Monte Carlo (MCMC) methods are widely used in machine learning. One of the major problems with MCMC is the question of how to design chains that mix fast over the whole state space; in particular, how to select the parameters…
This paper presents a new condition for the existence of optimal stationary policies in average-cost continuous-time Markov decision processes with unbounded cost and transition rates, arising from controlled queueing systems. This…
In this work we prove the nonlinear instability of inhomogeneous steady states solutions to the Hamiltonian Mean Field (HMF) model. We first study the linear instability of this model under a simple criterion by adapting the techniques…
I describe a new Markov chain method for sampling from the distribution of the state sequences in a non-linear state space model, given the observation sequence. This method updates all states in the sequence simultaneously using an…
In this paper, we consider queueing systems where the dynamics are non-stationary and state-dependent. For performance analysis of these systems, fluid and diffusion models have been typically used. Although they are proven to be…
A novel sequential change detection problem is proposed, in which the goal is to not only detect but also accelerate the change. Specifically, it is assumed that the sequentially collected observations are responses to treatments selected…
Long-lived quasistationary states, associated with stationary stable solutions of the Vlasov equation, are found in systems with long-range interactions. Studies of the relaxation time in a model of $N$ globally coupled particles moving on…
In this paper, we present a robust distributed model predictive control (DMPC) scheme for dynamically decoupled nonlinear systems which are subject to state constraints, coupled state constraints and input constraints. In the proposed…
Timesteppers constitute a powerful tool in modern computational science and engineering. Although they are typically used to advance the system forward in time, they can also be viewed as nonlinear mappings that implicitly encode steady…
Tandem queueing networks are widely used to model systems where services are provided in sequential stages. In this study, we assume that each station in the tandem system operates under a general renewal process. Additionally, we assume…
Self-triggered control (STC) is a resource efficient approach to determine sampling instants for Networked Control Systems (NCS). Recently, a dynamic STC strategy based on hybrid Lyapunov functions for nonlinear NCS has been proposed in…