Markov Chain

Def Period The period of state , is the largest common divisor of the set:We say that state is aperiodic if . If a Markov chain is periodic then there are parts of the state space that the chain can visit only at regular intervals. Prop implies that .

Def Irreducibility A Markov chain is said to be -irreducible if for every ,Irreducibility means that the chain can move from anywhere to anywhere else in a finite number of steps.

Thrm Limit Theorem of Markov Chain If is a -irreducible Markov chain with transition kernel and invariant distribution , and is a real valued function with , thenIf the chain is also aperiodic, then there is convergence in total variation: That is the Markov chain sample path mimics a random sample from , and the distribution of the realization converges to .

Corollary For an aperiodic and -irreducible Markov chain, , is also the limiting distribution of successive chain iterates, regardless of the starting values of the chain.