Empirical Motivation
The multi-armed bandit problem models an agent that simultaneously attempts to acquire new knowledge and to optimize its decisions based on existing knowledge. There are many practical applications:
- clinical trials investigating the effects of different experimental treatments while minimizing patient losses, and
- adaptive routing efforts for minimizing delays in a network.
In these practical examples, the problem requires balancing reward maximization based on the knowledge already acquired with attempting new actions to further increase knowledge. This is known as the exploitation vs. exploration tradeoff in reinforcement learning.
The model can also be used to control dynamic allocation of resources to different projects, answering the question "which project should I work on" given uncertainty about the difficulty and payoff of each possibility.
Originally considered by Allied scientists in World War II, it proved so intractable that, according to Peter Whittle, it was proposed the problem be dropped over Germany so that German scientists could also waste their time on it.
The version of the problem now commonly analyzed was formulated by Herbert Robbins in 1952.
Read more about this topic: Multi-armed Bandit
Famous quotes containing the words empirical and/or motivation:
“To develop an empiricist account of science is to depict it as involving a search for truth only about the empirical world, about what is actual and observable.... It must involve throughout a resolute rejection of the demand for an explanation of the regularities in the observable course of nature, by means of truths concerning a reality beyond what is actual and observable, as a demand which plays no role in the scientific enterprise.”
—Bas Van Fraassen (b. 1941)
“Self-determination has to mean that the leader is your individual gut, and heart, and mind or we’re talking about power, again, and its rather well-known impurities. Who is really going to care whether you live or die and who is going to know the most intimate motivation for your laughter and your tears is the only person to be trusted to speak for you and to decide what you will or will not do.”
—June Jordan (b. 1939)