Overview of The Leabra Algorithm
The pseudocode for Leabra is given here, showing exactly how the pieces of the algorithm described in more detail in the subsequent sections fit together.
Iterate over minus and plus phases of settling for each event. o At start of settling, for all units: - Initialize all state variables (activation, v_m, etc). - Apply external patterns (clamp input in minus, input & output in plus). - Compute net input scaling terms (constants, computed here so network can be dynamically altered). - Optimization: compute net input once from all static activations (e.g., hard-clamped external inputs). o During each cycle of settling, for all non-clamped units: - Compute excitatory netinput (g_e(t), aka eta_j or net) -- sender-based optimization by ignoring inactives. - Compute kWTA inhibition for each layer, based on g_i^Q: * Sort units into two groups based on g_i^Q: top k and remaining k+1 -> n. * If basic, find k and k+1th highest If avg-based, compute avg of 1 -> k & k+1 -> n. * Set inhibitory conductance g_i from g^Q_k and g^Q_k+1 - Compute point-neuron activation combining excitatory input and inhibition o After settling, for all units, record final settling activations as either minus or plus phase (y^-_j or y^+_j).
After both phases update the weights (based on linear current weight values), for all connections: o Compute error-driven weight changes with CHL with soft weight bounding o Compute Hebbian weight changes with CPCA from plus-phase activations o Compute net weight change as weighted sum of error-driven and Hebbian o Increment the weights according to net weight change.
Read more about this topic: Leabra