A Partially Observable Markov Decision Process (POMDP) is a generalization of a Markov Decision Process. A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot directly observe the underlying state. Instead, it must maintain a probability distribution over the set of possible states, based on a set of observations and observation probabilities, and the underlying MDP.
The POMDP framework is general enough to model a variety of real-world sequential decision processes. Applications include robot navigation problems, machine maintenance, and planning under uncertainty in general. The framework originated in the Operations Research community, and was later taken over by the Artificial Intelligence and Automated Planning communities.
An exact solution to a POMDP yields the optimal action for each possible belief over the world states. The optimal action maximizes (or minimizes) the expected reward (or cost) of the agent over a possibly infinite horizon. The sequence of optimal actions is known as the optimal policy of the agent for interacting with its environment.
Read more about Partially Observable Markov Decision Process: Belief Update, Belief MDP, Approximate POMDP Solutions, POMDP Uses
Famous quotes containing the words partially, observable, decision and/or process:
“I remember once dreaming of pushing a canoe up the rivers of Maine, and that, when I had got so high that the channels were dry, I kept on through the ravines and gorges, nearly as well as before, by pushing a little harder, and now it seemed to me that my dream was partially realized.”
—Henry David Thoreau (1817–1862)
“To develop an empiricist account of science is to depict it as involving a search for truth only about the empirical world, about what is actual and observable.... It must involve throughout a resolute rejection of the demand for an explanation of the regularities in the observable course of nature, by means of truths concerning a reality beyond what is actual and observable, as a demand which plays no role in the scientific enterprise.”
—Bas Van Fraassen (b. 1941)
“Concision in style, precision in thought, decision in life.”
—Victor Hugo (1802–1885)
“By Modernism I mean the positive rejection of the past and the blind belief in the process of change, in novelty for its own sake, in the idea that progress through time equates with cultural progress; in the cult of individuality, originality and self-expression.”
—Dan Cruickshank (b. 1949)