Hopfield Network - Structure

Structure

The units in Hopfield nets are binary threshold units, i.e. the units only take on two different values for their states and the value is determined by whether or not the units' input exceeds their threshold. Hopfield nets can either have units that take on values of 1 or -1, or units that take on values of 1 or 0. So, the two possible definitions for unit i's activation, are:

(1) a_i \leftarrow \left\{\begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ -1 & \mbox {otherwise.}\end{matrix}\right.

(2) a_i \leftarrow \left\{\begin{matrix} 1 & \mbox {if }\sum_{j}{w_{ij}s_j}>\theta_i, \\ 0 & \mbox {otherwise.}\end{matrix}\right.

Where:

  • is the strength of the connection weight from unit j to unit i (the weight of the connection).
  • is the state of unit j.
  • is the threshold of unit i.

The connections in a Hopfield net typically have the following restrictions:

  • (no unit has a connection with itself)
  • (connections are symmetric)

The requirement that weights be symmetric is typically used, as it will guarantee that the energy function decreases monotonically while following the activation rules, and the network may exhibit some periodic or chaotic behaviour if non-symmetric weights are used. However, Hopfield found that this chaotic behaviour is confined to relatively small parts of the phase space, and does not impair the network's ability to act as a content-addressable associative memory system.

Hopfield nets have a scalar value associated with each state of the network referred to as the "energy", E, of the network, where:

This value is called the "energy" because the definition ensures that if units are randomly chosen to update their activations the network will converge to states which are local minima in the energy function (which is considered to be a Lyapunov function). Thus, if a state is a local minimum in the energy function it is a stable state for the network. Note that this energy function belongs to a general class of models in physics, under the name of Ising models; these in turn are a special case of Markov networks, since the associated probability measure, the Gibbs measure, has the Markov property.

Read more about this topic:  Hopfield Network

Famous quotes containing the word structure:

    “Vashtar: So it’s finished. A structure to house one man and the greatest treasure of all time.
    Senta: And a structure that will last for all time.
    Vashtar: Only history will tell that.
    Senta: Sire, will he not be remembered?
    Vashtar: Yes, he’ll be remembered. The pyramid’ll keep his memory alive. In that he built better than he knew.”
    —William Faulkner (1897–1962)

    “I really do inhabit a system in which words are capable of shaking the entire structure of government, where words can prove mightier than ten military divisions.”
    —Václav Havel (b. 1936)

    “Why does philosophy use concepts and why does faith use symbols if both try to express the same ultimate? The answer, of course, is that the relation to the ultimate is not the same in each case. The philosophical relation is in principle a detached description of the basic structure in which the ultimate manifests itself. The relation of faith is in principle an involved expression of concern about the meaning of the ultimate for the faithful.”
    —Paul Tillich (1886–1965)