# What is Markov chain?

• (noun): A Markov process for which the parameter is discrete time values.
Synonyms: Markoff chain

## Markov Chain

A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process usually characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property. Markov chains have many applications as statistical models of real-world processes.

### Some articles on Markov chain:

Balance Equation - Global Balance
... always be solved to give the equilibrium distribution of a Markov chain (when such a distribution exists) ... For a Markov chain with state space S, transition rate from state i to j given by qij and equilibrium distribution given by, the global balance equations are ... For a discrete time Markov chain with transition matrix P and equilibrium distribution the global balance equation is ...
Nonlinear Dimensionality Reduction - Manifold Learning Algorithms - Diffusion Maps
... maps leverages the relationship between heat diffusion and a random walk (Markov Chain) an analogy is drawn between the diffusion operator on a manifold and a Markov transition matrix operating ... It is easy to see here that from the tuple {X,k} one can construct a reversible Markov Chain ... If has to faithfully represent a Markov matrix, then it has to be normalized by the corresponding degree matrix now represents a Markov chain ...
Markov Chain - History
... Andrey Markov produced the first results (1906) for these processes, purely theoretically ... Markov chains are related to Brownian motion and the ergodic hypothesis, two topics in physics which were important in the early years of the twentieth ... Seneta provides an account of Markov's motivations and the theory's early development ...
Multiple-try Metropolis
... In Markov chain Monte Carlo, the Metropolis–Hastings algorithm (MH) can be used to sample from a probability distribution which is difficult to sample from directly ... almost all steps will be accepted, and the Markov chain will be similar to a random walk through the probability space ... In this event, the Markov Chain will not fully explore the probability space in any reasonable amount of time ...
Prediction Suffix Tree
... The concept of the Markov chain of order L, which we essentially owe to the Russian mathematician Andrej Andreevic Markov (1907), has two drawbacks ... the model grows exponentially with the order L of the chain ... in compression data (Weinberger - 1992, Willems - 1995) was the Variable Length Markov chain (Buhlmann - 1999) ...

### Famous quotes containing the word chain:

How have I been able to live so long outside Nature without identifying myself with it? Everything lives, moves, everything corresponds; the magnetic rays, emanating either from myself or from others, cross the limitless chain of created things unimpeded; it is a transparent network that covers the world, and its slender threads communicate themselves by degrees to the planets and stars. Captive now upon earth, I commune with the chorus of the stars who share in my joys and sorrows.
Gérard De Nerval (1808–1855)