Discrete-Time Semi-Markov Random Evolutions and their Applications

Abstract

In this paper we introduce discrete-time semi-Markov random evolutions (DTSMREs) and study asymptotic properties, namely, averaging, diffusion approximation, and diffusion approximation with equilibrium by the martingale weak convergence method. The controlled DTSMREs are introduced and Hamilton–Jacobi–Bellman equations are derived for them. The applications here concern the additive functionals (AFs), geometric Markov renewal chains (GMRCs), and dynamical systems (DSs) in discrete time. The rates of convergence in the limit theorems for DTSMREs and AFs, GMRCs, and DSs are also presented.