Graphs and Markov chains

Abstract

In its simplest form the theory of Markov chains states that, if the (i,j) element of matrix M is the probability pij of a one-step transition from state i to state j, then the (ij) element of the matrix Mk gives the probability of transition from state i to state j in k steps. Since the probabilities of transition to the different states add to 1, the sum of the elements in any row of M is necessarily equal to 1. (Readers are warned that in some elementary treatments the alternative convention is adopted of writing pij in the (j,i) position, so that the column sums are 1, in order that the probability vectors should be column matrices, which are more familiar in school mathematics.)