Principle of Indifference - History of The Principle of Indifference

History of The Principle of Indifference

The original writers on probability, primarily Jacob Bernoulli and Pierre Simon Laplace, considered the principle of indifference to be intuitively obvious and did not even bother to give it a name. Laplace wrote:

The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

These earlier writers, Laplace in particular, naively generalized the principle of indifference to the case of continuous parameters, giving the so-called "uniform prior probability distribution", a function which is constant over all real numbers. He used this function to express a complete lack of knowledge as to the value of a parameter. According to Stigler (page 135), Laplace's assumption of uniform prior probabilities was not a meta-physical assumption. It was an implicit assumption made for the ease of analysis.

The principle of insufficient reason was its first name, given to it by later writers, possibly as a play on Leibniz's principle of sufficient reason. These later writers (George Boole, John Venn, and others) objected to the use of the uniform prior for two reasons. The first reason is that the constant function is not normalizable, and thus is not a proper probability distribution. The second reason is its inapplicability to continuous variables, as described above. (However, these paradoxical issues can be resolved. In the first case, a constant, or any more general finite polynomial, is normalizable within any finite range: the range is all that matters here. Alternatively, modify the function to be zero outside that range. See uniform distribution. In the second case, there is no ambiguity provided the problem is "well-posed", so that no unwarranted assumptions can be made, or have to be made, thereby fixing the appropriate prior probability density function or prior moment generating function (with variables fixed appropriately) to be used for the probability itself. See the Bertrand paradox (probability) for an analogous case.)

The "Principle of insufficient reason" was renamed the "Principle of Indifference" by the economist John Maynard Keynes (1921), who was careful to note that it applies only when there is no knowledge indicating unequal probabilities.

Attempts to put the notion on firmer philosophical ground have generally begun with the concept of equipossibility and progressed from it to equiprobability.

The principle of indifference can be given a deeper logical justification by noting that equivalent states of knowledge should be assigned equivalent epistemic probabilities. This argument was propounded by E.T. Jaynes: it leads to two generalizations, namely the principle of transformation groups as in the Jeffreys prior, and the principle of maximum entropy.

More generally, one speaks of non-informative priors.