Computational Creativity - A Formal Theory of Curiosity and Selective Attention

A Formal Theory of Curiosity and Selective Attention

The Formal Theory of Creativity is based on a simple computational principle published by Jürgen Schmidhuber in 1991. The theory postulates that creativity and curiosity and selective attention in general are by-products of a simple algorithmic principle for measuring and optimizing learning progress. Consider a human or non-human agent able to manipulate its environment and thus its own sensory inputs. The agent can use a black box optimization method such as reinforcement learning or evolutionary computation to learn (through informed trial and error) sequences of actions that maximize the expected sum of its future reward signals. Traditional "extrinsic" reward signals include those for solving externally given problems, such as finding and consuming food when hungry. But Schmidhuber's objective function to be maximized also includes an additional, non-standard, "intrinsic" term to model learning progress. This non-standard term motivates purely creative behavior of the agent even when there are no external goals. Learning progress is formally defined as follows. As the agent is creating and observing and encoding the continually growing history of actions and sensory inputs, it keeps improving the encoder of this data. (The encoder can be implemented as a self-learning, predictive, artificial neural network or some other machine learning device that can exploit regularities in the data to become a better predictor or encoder over time.) The encoder's improvements can be measured precisely, by computing the difference in computational costs (storage size, number of neurons, errors, time) needed to encode new data before and after learning. This difference depends on the encoder's present subjective knowledge, which changes over time, but the theory takes this into account. It measures the present learning progress, which becomes proportional to the present reward signal for the action selector. The objective function thus motivates the action optimizer to create action sequences causing more learning progress.

Only some yet unknown regularity in the data may permit learning progress, while irregular, random data (or noise) is boring by nature (no intrinsic reward). Already known and predictable regularities also become boring. This provokes continual, open-ended, active, creative exploration of more and more, initially unknown, novel, regular patterns.

Schmidhuber argues that his objective function explains the basic activities of scientists and artists and comedians. For example, a "physicist gets intrinsic reward for creating an experiment leading to observations obeying a previously unpublished physical law that allows for better compressing the data". A composer gets intrinsic reward for "creating a new but non-random, non-arbitrary melody with novel, unexpected but regular harmonies that also permit learning progress of the adaptive data encoder." A comedian gets intrinsic reward for "inventing a novel joke with an unexpected punch line, related to the beginning of the story in an initially unexpected but quickly learnable way that also allows for better compression of the perceived data." Schmidhuber argues that various simple implementations of the basic principle since 1990 can already be viewed as rudimentary artificial scientists and artists, and that ongoing computer hardware advances will greatly scale them up. He used the theory to create an attractive human face and other types of low-complexity art.