APPLICATION OF SINGLE STEP EULER AND TRAPEZOID METHODS OF SOLVING TRANSIENT DISTRIBUTION IN MARKOV CHAIN

Abstract

The computation of state probability distributions at an arbitrary point in time, which in the case of a discrete-time Markov chain means finding the distribution at some arbitrary time step n denoted π^((n) ), a row vector whose i^th component is the probability that the Markov chain is in state i at time step n, and this is the iterative solution methods for transient distribution in Markov chain. In this study, the solutions of transient distribution in Markov chain using Euler and trapezoid methods have been investigated, in order to provide some insight into the solutions of transient distribution in Markov chain, which produce a significantly more accurate response in less time for some types of situations and also tries to get to the end result as quickly as possible with the solution conform with a specified number of well-defined stages is computed for large space Markov chain. Matrices operations, such as the product and matrix inversion are performed and Markov chain laws, theorems, formulas with MATLAB software are utilized. For illustrative examples, the transient distribution vector's π_((i+1) ),i=0,1,2,…, ; is computed for both Euler and Trapezoid methods and their corresponding error when compared with different method. The effect of decreasing the step size using the same infinitesimal generator is examined, also, by taking π_((0) )=(■(1&0&0)) and the length of the interval of integration to be τ = 1, it is observed that the accuracy achieved with the explicit Euler method has a direct relationship with the step size. It is also concluded that the much more accurate results are obtained with the trapezoid method.