Steady-state Analysis and Limiting Distributions
If the Markov chain is a time-homogeneous Markov chain, so that the process is described by a single, time-independent matrix, then the vector is called a stationary distribution (or invariant measure) if its entries are non-negative and sum to 1 and if it satisfies
An irreducible chain has a stationary distribution if and only if all of its states are positive recurrent. In that case, π is unique and is related to the expected return time:
where is the normalizing constant. Further, if the chain is both irreducible and aperiodic, then for any i and j,
Note that there is no assumption on the starting distribution; the chain converges to the stationary distribution regardless of where it begins. Such π is called the equilibrium distribution of the chain.
If a chain has more than one closed communicating class, its stationary distributions will not be unique (consider any closed communicating class in the chain; each one will have its own unique stationary distribution . Extending these distributions to the overall chain, setting all values to zero outside the communication class, yields that the set of invariant measures of the original chain is the set of all convex combinations of the 's). However, if a state j is aperiodic, then
and for any other state i, let fij be the probability that the chain ever visits state j if it starts at i,
If a state i is periodic with period k > 1 then the limit
does not exist, although the limit
does exist for every integer r.
Read more about this topic: Markov Chain, Markov Chains
Famous quotes containing the words analysis and/or limiting:
“Whatever else American thinkers do, they psychologize, often brilliantly. The trouble is that psychology only takes us so far. The new interest in families has its merits, but it will have done us all a disservice if it turns us away from public issues to private matters. A vision of things that has no room for the inner life is bankrupt, but a psychology without social analysis or politics is both powerless and very lonely.”
—Joseph Featherstone (20th century)
“O wily painter, limiting the scene
From a cacophony of dusty forms
To the one convulsion,”
—Thom Gunn (b. 1929)