Related papers: A complexity analysis of Policy Iteration through …
Decision-making problems in uncertain or stochastic domains are often formulated as Markov decision processes (MDPs). Policy iteration (PI) is a popular algorithm for searching over policy-space, the size of which is exponential in the…
A unique sink orientation (USO) is an orientation of the hypercube graph with the property that every face has a unique sink. A number of well-studied problems reduce in strongly polynomial time to finding the global sink of a USO; most…
The question of knowing whether the policy Iteration algorithm (PI) for solving Markov Decision Processes (MDPs) has exponential or (strongly) polynomial complexity has attracted much attention in the last 50 years. Recently, Fearnley…
Solving Markov Decision Processes (MDPs) is a recurrent task in engineering. Even though it is known that solutions for minimizing the infinite horizon expected reward can be found in polynomial time using Linear Programming techniques,…
A unique sink orientation (USO) is an orientation of the edges of a hypercube such that each face has a unique sink. Many optimization problems like linear programs reduce to USOs, in the sense that each vertex corresponds to a possible…
Policy iteration is a family of algorithms that are used to find an optimal policy for a given Markov Decision Problem (MDP). Simple Policy iteration (SPI) is a type of policy iteration where the strategy is to change the policy at exactly…
Policy Iteration (PI) is a classical family of algorithms to compute an optimal policy for any given Markov Decision Problem (MDP). The basic idea in PI is to begin with some initial policy and to repeatedly update the policy to one from an…
The Matou\v{s}ek LP-type problems were used by Matou\v{s}ek to show that the Sharir-Welzl algorithm may require at least subexponential time. Later, G\"artner translated this result into the language of Unique Sink Orientations (USOs) and…
We consider quantum algorithms for the unique sink orientation problem on cubes. This problem is widely considered to be of intermediate computational complexity. This is because there no known polynomial algorithm (classical or quantum)…
We develop value iteration-based algorithms to solve in a unified manner different classes of combinatorial zero-sum games with mean-payoff type rewards. These algorithms rely on an oracle, evaluating the dynamic programming operator up to…
A unique sink orientation (USO) is an orientation of the $n$-dimensional hypercube graph such that every non-empty face contains a unique sink. Schurr showed that given any $n$-dimensional USO and any dimension $i$, the set of edges $E_i$…
Algorithms for finding the sink in Unique Sink Orientations (USOs) of the hypercube can be used to solve many algebraic and geometric problems, most importantly including the P-Matrix Linear Complementarity Problem and Linear Programming.…
Unique Sink Orientations (USOs) of cubes can be used to capture the combinatorial structure of many essential algebraic and geometric problems. For various structural and algorithmic questions, including enumeration of USOs and algorithm…
Given a Markov Decision Process (MDP) with $n$ states and a totalnumber $m$ of actions, we study the number of iterations needed byPolicy Iteration (PI) algorithms to converge to the optimal$\gamma$-discounted policy. We consider two…
Policy Iteration (PI) is a widely used family of algorithms to compute optimal policies for Markov Decision Problems (MDPs). We derive upper bounds on the running time of PI on Deterministic MDPs (DMDPs): the class of MDPs in which every…
In order to address the imprecision often introduced by widening operators in static analysis, policy iteration based on min-computations amounts to considering the characterization of reachable value set of a program as an iterative…
An orientation of a grid is called unique sink orientation (USO) if each of its nonempty subgrids has a unique sink. Particularly, the original grid itself has a unique global sink. In this work we investigate the problem of how to find the…
A basic model in sequential decision making is the Markov decision process (MDP), which is extended to Robust MDPs (RMDPs) by allowing uncertainty in transition probabilities and optimizing against the worst-case transition probabilities…
In this paper, we demonstrate that policy iteration, introduced in the context of HJB equations in [Forsyth & Labahn, 2007], is an extremely simple generic algorithm for solving linear complementarity problems resulting from the finite…
Relational Markov Decision Processes are a useful abstraction for complex reinforcement learning problems and stochastic planning problems. Recent work developed representation schemes and algorithms for planning in such problems using the…