A Partially Observable Markov Decision Process (POMDP) is a generalization of a Markov Decision Process. A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot directly observe the underlying state. Instead, it must maintain a probability distribution over the set of possible states, based on a set of observations and observation probabilities, and the underlying MDP.
The POMDP framework is general enough to model a variety of real-world sequential decision processes. Applications include robot navigation problems, machine maintenance, and planning under uncertainty in general. The framework originated in the Operations Research community, and was later taken over by the Artificial Intelligence and Automated Planning communities.
An exact solution to a POMDP yields the optimal action for each possible belief over the world states. The optimal action maximizes (or minimizes) the expected reward (or cost) of the agent over a possibly infinite horizon. The sequence of optimal actions is known as the optimal policy of the agent for interacting with its environment.
Read more about Partially Observable Markov Decision Process: Belief Update, Belief MDP, Approximate POMDP Solutions, POMDP Uses
Famous quotes containing the words partially, observable, decision and/or process:
“Let us consider that we are all partially insane. It will explain us to each other; it will unriddle many riddles; it will make clear and simple many things which are involved in haunting and harassing difficulties and obscurities now.”
—Mark Twain [Samuel Langhorne Clemens] (1835–1910)
“Every living language, like the perspiring bodies of living creatures, is in perpetual motion and alteration; some words go off, and become obsolete; others are taken in, and by degrees grow into common use; or the same word is inverted to a new sense or notion, which in tract of time makes an observable change in the air and features of a language, as age makes in the lines and mien of a face.”
—Richard Bentley (1662–1742)
“Moral choices do not depend on personal preference and private decision but on right reason and, I would add, divine order.”
—Basil Hume (b. 1923)
“The invention of photography provided a radically new picture-making process—a process based not on synthesis but on selection. The difference was a basic one. Paintings were made—constructed from a storehouse of traditional schemes and skills and attitudes—but photographs, as the man on the street put, were taken.”
—Jean Szarkowski (b. 1925)