A Partially Observable Markov Decision Process (POMDP) is a generalization of a Markov Decision Process. A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot directly observe the underlying state. Instead, it must maintain a probability distribution over the set of possible states, based on a set of observations and observation probabilities, and the underlying MDP.
The POMDP framework is general enough to model a variety of real-world sequential decision processes. Applications include robot navigation problems, machine maintenance, and planning under uncertainty in general. The framework originated in the Operations Research community, and was later taken over by the Artificial Intelligence and Automated Planning communities.
An exact solution to a POMDP yields the optimal action for each possible belief over the world states. The optimal action maximizes (or minimizes) the expected reward (or cost) of the agent over a possibly infinite horizon. The sequence of optimal actions is known as the optimal policy of the agent for interacting with its environment.
Read more about Partially Observable Markov Decision Process: Belief Update, Belief MDP, Approximate POMDP Solutions, POMDP Uses
Famous quotes containing the words partially, observable, decision and/or process:
“There was an Old Man who supposed,
That the street door was partially closed;”
—Edward Lear (1812–1888)
“Every living language, like the perspiring bodies of living creatures, is in perpetual motion and alteration; some words go off, and become obsolete; others are taken in, and by degrees grow into common use; or the same word is inverted to a new sense or notion, which in tract of time makes an observable change in the air and features of a language, as age makes in the lines and mien of a face.”
—Richard Bentley (1662–1742)
“The women of my mother’s generation had, in the main, only one decision to make about their lives: who they would marry. From that, so much else followed: where they would live, in what sort of conditions, whether they would be happy or sad or, so often, a bit of both. There were roles and there were rules.”
—Anna Quindlen (20th century)
“The toddler’s wish to please ... is a powerful aid in helping the child to develop a social awareness and, eventually, a moral conscience. The child’s love for the parent is so strong that it causes him to change his behavior: to refrain from hitting and biting, to share toys with a peer, to become toilet trained. This wish for approval is the parent’s most reliable ally in the process of socializing the child.”
—Alicia F. Lieberman (20th century)