Mathematical Definitions
A Markov process, like a Markov chain, can be thought of as a directed graph of states of the system. The difference is that, rather than transitioning to a new (possibly the same) state at each time step, the system will remain in the current state for some random (in particular, exponentially distributed) amount of time and then transition to a different state. The process is characterized by "transition rates" between states i and j. Let X(t) be the random variable describing the state of the process at time t, and assume that the process is in a state i at time t. (for ) measures how quickly that transition happens. Precisely, after a tiny amount of time h, the probability the state is now j is given by
where o(h) represents a quantity that goes to zero faster than h goes to zero (see the article on order notation). Hence, over a sufficiently small interval of time, the probability of a particular transition (between different states) is roughly proportional to the duration of that interval. The are called transition rates because if we have a large ensemble of n systems in state i, they will switch over to state j at an average rate of until n decreases appreciably.
The transition rates are typically given as the ij-th elements of the transition rate matrix Q (also known as an intensity matrix). As the transition rate matrix contains rates, the rate of departing from one state to arrive at another should be positive, and the rate that the system remains in a state should be negative. The rates for a given state should sum to zero, yielding the diagonal elements to be
With this notation, and letting, the evolution of a continuous-time Markov process is given by the first-order differential equation
The probability that no transition happens in some time r is
That is, the probability distribution of the waiting time until the first transition is an exponential distribution with rate parameter, and continuous-time Markov processes are thus memoryless processes.
A time dependent (time heterogeneous) Markov process is a Markov process as above, but with the q-rate a function of time, denoted qij(t).
Read more about this topic: Continuous-time Markov Process
Famous quotes containing the words mathematical and/or definitions:
“What he loved so much in the plant morphological structure of the tree was that given a fixed mathematical basis, the final evolution was so incalculable.”
—D.H. (David Herbert)
“What I do not like about our definitions of genius is that there is in them nothing of the day of judgment, nothing of resounding through eternity and nothing of the footsteps of the Almighty.”
—G.C. (Georg Christoph)