Memory-prediction Framework - The Basic Theory: Recognition and Prediction in Bi-directional Hierarchies

The Basic Theory: Recognition and Prediction in Bi-directional Hierarchies

The central concept of the memory-prediction framework is that bottom-up inputs are matched in a hierarchy of recognition, and evoke a series of top-down expectations encoded as potentiations. These expectations interact with the bottom-up signals to both analyse those inputs and generate predictions of subsequent expected inputs. Each hierarchy level remembers frequently observed temporal sequences of input patterns and generates labels or 'names' for these sequences. When an input sequence matches a memorized sequence at a given layer of the hierarchy, a label or 'name' is propagated up the hierarchy - thus eliminating details at higher levels and enabling them to learn higher-order sequences. This process produces increased invariance at higher levels. Higher levels predict future input by matching partial sequences and projecting their expectations to the lower levels. However, when a mismatch between input and memorized/predicted sequences occurs, a more complete representation propagates upwards. This causes alternative 'interpretations' to be activated at higher levels, which in turn generates other predictions at lower levels.

Consider, for example, the process of vision. Bottom-up information starts as low-level retinal signals (indicating the presence of simple visual elements and contrasts). At higher levels of the hierarchy, increasingly meaningful information is extracted, regarding the presence of lines, regions, motions, etc. Even further up the hierarchy, activity corresponds to the presence of specific objects - and then to behaviours of these objects. Top-down information fills in details about the recognized objects, and also about their expected behaviour as time progresses.

The sensory hierarchy induces a number of differences between the various layers. As one moves up the hierarchy, representations have increased:

• Extent - for example, larger areas of the visual field, or more extensive tactile regions.
• Temporal stability - lower-level entities change quickly, whereas, higher-level percepts tend to be more stable.
• Abstraction - through the process of successive extraction of invariant features, increasingly abstract entities are recognized.

The relationship between sensory and motor processing is an important aspect of the basic theory. It is proposed that the motor areas of cortex consist of a behavioural hierarchy similar to the sensory hierarchy, with the lowest levels consisting of explicit motor commands to musculature and the highest levels corresponding to abstract prescriptions (e.g. 'resize the browser'). The sensory and motor hierarchies are tightly coupled, with behaviour giving rise to sensory expectations and sensory perceptions driving motor processes.

Finally, it is important to note that all the memories in the cortical hierarchy have to be learnt - this information is not pre-wired in the brain. Hence, the process of extracting this representation from the flow of inputs and behaviours is theorized as a process that happens continually during cognition.