Partially Observable Markov Decision Process - Belief Update

Belief Update

An agent needs to update its belief upon taking the action and observing . Since the state is Markovian, maintaining a belief over the states solely requires knowledge of the previous belief state, the action taken, and the current observation. The operation is denoted . Below we describe how this belief update is computed.

In, the agent observes with probability . Let be a probability distribution over the state space : denotes the probability that the environment is in state . Given, then after taking action and observing ,

b'(s') = \eta \Omega(o\mid s',a) \sum_{s\in S} T(s'\mid s,a)b(s)

where is a normalizing constant with .

Read more about this topic:  Partially Observable Markov Decision Process

Famous quotes containing the word belief:

    “Those of us who were brought up as Christians and have lost our faith have retained the sense of sin without the saving belief in redemption. This poisons our thought and so paralyses us in action.”
    —Cyril Connolly (1903–1974)