Structural Information Theory - Structural Versus Algorithmic Information Theory

Structural Versus Algorithmic Information Theory

Since the 1960s, SIT (in psychology) and AIT (in computer science) evolved independently as viable alternatives for Shannon's classical information theory which had been developed in communication theory. In Shannon's approach, things are assigned codes with lengths based on their probability in terms of frequencies of occurrence (as, e.g., in the Morse code). In many domains, including perception, such probabilities are hardly quantifiable if at all, however. Both SIT and AIT circumvent this problem by turning to descriptive complexities of individual things.

Although SIT and AIT share many starting points and objectives, there are also several relevant differences:

• First, SIT makes the perceptually relevant distinction between structural and metrical information, whereas AIT does not;
• Second, SIT encodes for a restricted set of perceptually relevant kinds of regularities, whereas AIT encodes for any imaginable regularity;
• Third, in SIT, the relevant outcome of an encoding is a hierarchical organization, whereas in AIT, it is only a complexity value.