Surprise Probabilities in Markov Chains

Abstract

In a Markov chain started at a statex, thehitting timeτ(y) is the first time that the chain reaches another statey. We study the probability$\mathbb{P}_x(\tau(y) = t)$that the first visit toyoccurs precisely at a given timet. Informally speaking, the event that a new state is visited at a large timetmay be considered a ‘surprise’. We prove the following three bounds.•In any Markov chain withnstates,$\mathbb{P}_x(\tau(y) = t) \le {n}/{t}$.•In a reversible chain withnstates,$\mathbb{P}_x(\tau(y) = t) \le {\sqrt{2n}}/{t}$ for $t \ge 4n + 4$.•For random walk on a simple graph withn≥ 2 vertices,$\mathbb{P}_x(\tau(y) = t) \le 4e \log(n)/t$.We construct examples showing that these bounds are close to optimal. The main feature of our bounds is that they require very little knowledge of the structure of the Markov chain.To prove the bound for random walk on graphs, we establish the following estimate conjectured by Aldous, Ding and Oveis-Gharan (private communication): for random walk on ann-vertex graph, for every initial vertexx,$$ \sum_y \biggl( \sup_{t \ge 0} p^t(x, y) \biggr) = O(\log n). $$