相关论文: Markovian Testing Equivalence and Exponentially Ti…

Several Markovian process calculi have been proposed in the literature, which differ from each other for various aspects. With regard to the action representation, we distinguish between integrated-time Markovian process calculi, in which…

It is common, when dealing with quantum processes involving a subsystem of a much larger composite closed system, to treat them as effectively memory-less (Markovian). While open systems theory tells us that non-Markovian processes should…

We derive a necessary and sufficient condition for a quantum process to be Markovian which coincides with the classical one in the relevant limit. Our condition unifies all previously known definitions for quantum Markov processes by…

The theory of ``Markov-up'' processes is being developed. This is a new class of stochastic processes with ``partial'' markovian features; it could also be called ``one-sided Markov''. Such a behavior may be found in the real world and in…

Several application domains require formal but flexible approaches to the comparison problem. Different process models that cannot be related by behavioral equivalences should be compared via a quantitative notion of similarity, which is…

We define a testing equivalence in the spirit of De Nicola and Hennessy for reactive probabilistic processes, i.e. for processes where the internal nondeterminism is due to random behaviour. We characterize the testing equivalence in terms…

The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term 'risk-sensitive' refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk.…

Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…

Semi-Markov processes are Markovian processes in which the firing time of the transitions is modelled by probabilistic distributions over positive reals interpreted as the probability of firing a transition at a certain moment in time. In…

We have recently defined a weak Markovian bisimulation equivalence in an integrated-time setting, which reduces sequences of exponentially timed internal actions to individual exponentially timed internal actions having the same average…

The notion of a successful coupling of Markov processes, based on the idea that both components of the coupled system ``intersect'' in finite time with probability one, is extended to cover situations when the coupling is unnecessarily…

Comparison results are given for time-inhomogeneous Markov processes with respect to function classes induced stochastic orderings. The main result states comparison of two processes, provided that the comparability of their infinitesimal…

The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume…

Memoryless processes are ubiquitous in nature, in contrast with the mathematics of open systems theory, which states that non-Markovian processes should be the norm. This discrepancy is usually addressed by subjectively making the…

We develop an approach to time-consistent risk evaluation of continuous-time processes in Markov systems. Our analysis is based on dual representation of coherent risk measures, differentiability concepts for multivalued mappings, and a…

A new weak bisimulation semantics is defined for Markov automata that, in addition to abstracting from internal actions, sums up the expected values of consecutive exponentially distributed delays possibly intertwined with internal actions.…

Generic non-Markovian quantum processes have infinitely long memory, implying an exact description that grows exponentially in complexity with observation time. Here, we present a finite memory ansatz that approximates (or recovers) the…

In the classical domain, it is well-known that divisibility does not imply that a stochastic process is Markovian. However, for quantum processes, divisibility is often considered to be synonymous with Markovianity. We show that completely…

In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…

This paper is devoted to solving a time-inconsistent risk-sensitive control problem with parameter $\e$ and its limit case ($\e\rightarrow0^+$) for countable-stated Markov decision processes (MDPs for short). Since the cost functional is…