Related papers: On a connection between total positivity and Berno…
The Josephus problem is a well--studied elimination problem consisting in determining the position of the survivor after repeated applications of a deterministic rule removing one person at a time from a given group. A natural probabilistic…
We study the design of optimal incentives in sequential processes. To do so, we consider a basic and fundamental model in which an agent initiates a value-creating sequential process through costly investment with random success. If…
We study the optimal multiple stopping time problem defined for each stopping time $S$ by $v(S)=\operatorname {ess}\sup_{\tau_1,...,\tau_d\geq S}E[\psi(\tau_1,...,\tau_d)|\mathcal{F}_S]$. The key point is the construction of a new reward…
We reveal an interesting convex duality relationship between two problems: (a) minimizing the probability of lifetime ruin when the rate of consumption is stochastic and when the individual can invest in a Black-Scholes financial market;…
A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance.…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
We study a sequential decision-making model where a set of items is repeatedly matched to the same set of agents over multiple rounds. The objective is to determine a sequence of matchings that either maximizes the utility of the least…
The multi-armed bandit (MAB) problem is a classic example of the exploration-exploitation dilemma. It is concerned with maximising the total rewards for a gambler by sequentially pulling an arm from a multi-armed slot machine where each arm…
The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…
We study optimal stopping for diffusion processes with unknown model primitives within the continuous-time reinforcement learning (RL) framework developed by Wang et al. (2020), and present applications to option pricing and portfolio…
We consider a singular stochastic control problem, which is called the Monotone Follower Stochastic Control Problem and give sufficient conditions for the existence and uniqueness of a local-time type optimal control. To establish this…
Consider the multiple testing problem of testing k null hypotheses, where the unknown family of distributions is assumed to satisfy a certain monotonicity assumption. Attention is restricted to procedures that control the familywise error…
We solve the problem of optimal stopping of a Brownian motion subject to the constraint that the stopping time's distribution is a given measure consisting of finitely-many atoms. In particular, we show that this problem can be converted to…
We study a class of optimal allocation problems, including the well-known Bomber Problem, with the following common probabilistic structure. An aircraft equipped with an amount~$x$ of ammunition is intercepted by enemy airplanes arriving…
The purpose of this paper is two-fold: We extend the well-known relation between optimal stopping and randomized stopping of a given stochastic process to a situation where the available information flow is a filtration with no a priori…
Given two weighted automata, we consider the problem of whether one is big-O of the other, i.e., if the weight of every finite word in the first is not greater than some constant multiple of the weight in the second. We show that the…
We derive explicit upper bounds for the $\bar{d}$-distance between a chain of infinite order and its canonical $k$-steps Markov approximation. Our proof is entirely constructive and involves a "coupling from the past" argument. The new…
Time-to-event data are often recorded on a discrete scale with multiple, competing risks as potential causes for the event. In this context, application of continuous survival analysis methods with a single risk suffer from biased…
The problem of stopping a Brownian bridge with an unknown pinning point to maximise the expected value at the stopping time is studied. A few general properties, such as continuity and various bounds of the value function, are established.…