Related papers: Fitting procedure for the two-state Batch Markov m…

We consider a novel queuing problem where the decision-maker must choose to accept or reject randomly arriving tasks into a no buffer queue which are processed by $N$ identical servers. Each task has a price, which is a positive real…

We propose using an adaptive sampling method to detect changes for a system with multiple lines. The adaptive sampling utilizes the information in responses to learn on which line is more likely to have a change thus allocating more units…

This paper proposes a unified adaptive event-triggered model predictive control (ETMPC) scheme for linear parameter-varying (LPV) systems subject to state delays, actuator saturation, and external disturbances. In existing studies, only a…

Mixed observable Markov decision processes (MOMDPs) are a modeling framework for autonomous systems described by both fully and partially observable states. In this work, we study the problem of synthesizing a control policy for MOMDPs that…

We develop a qualitative theory of Markov Decision Processes (MDPs) and Partially Observable MDPs that can be used to model sequential decision making tasks when only qualitative information is available. Our approach is based upon an…

In this paper we present a model predictive control (MPC) approach to optimize vehicle scheduling and routing in an autonomous mobility-on-demand (AMoD) system. In AMoD systems, robotic, self-driving vehicles transport customers within an…

Prediction of events such as part replacement and failure events plays a critical role in reliability engineering. Event stream data are commonly observed in manufacturing and teleservice systems. Designing predictive models for individual…

We consider a change-point detection problem for a simple class of Piecewise Deterministic Markov Processes (PDMPs). A continuous-time PDMP is observed in discrete time and through noise, and the aim is to propose a numerical method to…

We consider a class of sequential decision-making problems under uncertainty that can encompass various types of supervised learning concepts. These problems have a completely observed state process and a partially observed modulation…

Jump Markov linear systems (JMLS) are a useful class which can be used to model processes which exhibit random changes in behavior during operation. This paper presents a numerically stable method for learning the parameters of jump Markov…

In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…

This paper develops an adaptive traffic control policy inspired by Maximum Pressure (MP) while imposing coordination across intersections. The proposed Coordinated Maximum Pressure-plus-Penalty (CMPP) control policy features a local…

We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces. We introduce a property of state space partitions which we call epsilon-homogeneity. Intuitively, an epsilon-homogeneous…

Although the Poisson point process (PPP) has been widely used to model base station (BS) locations in cellular networks, it is an idealized model that neglects the spatial correlation among BSs. The present paper proposes the use of…

In this paper, we develop a more general framework of block-structured Markov processes in the queueing study of blockchain systems, which can provide analysis both for the stationary performance measures and for the sojourn times of any…

We construct a stochastic fluid process with an underlying piecewise deterministic Markov process (PDMP) akin to the one used in the construction of the rational arrival process (RAP), which we call the RAP-modulated fluid process. As…

Models of biological systems often have many unknown parameters that must be determined in order for model behavior to match experimental observations. Commonly-used methods for parameter estimation that return point estimates of the…

We introduce a framework to approximate a Markov Decision Process that stands on two pillars: state aggregation -- as the algorithmic infrastructure; and central-limit-theorem-type approximations -- as the mathematical underpinning of…

The parameters of a discrete stationary Markov model are transition probabilities between states. Traditionally, data consist in sequences of observed states for a given number of individuals over the whole observation period. In such a…

Consider a stationary discrete random process with alphabet size d, which is assumed to be the output process of an unknown stationary Hidden Markov Model (HMM). Given the joint probabilities of finite length strings of the process, we are…