Laplace's Method - General Theory of Laplace's Method

General Theory of Laplace's Method

To state and motivate the method, we need several assumptions. We will assume that x₀ is not an endpoint of the interval of integration, that the values ƒ(x) cannot be very close to ƒ(x₀) unless x is close to x₀, and that the second derivative .

We can expand ƒ(x) around x₀ by Taylor's theorem,

where

Since ƒ has a global maximum at x₀, and since x₀ is not an endpoint, it is a stationary point, so the derivative of ƒ vanishes at x₀. Therefore, the function ƒ(x) may be approximated to quadratic order

for x close to x₀ (recall that the second derivative is negative at the global maximum ƒ(x₀)). The assumptions made ensure the accuracy of the approximation

(see the picture on the right). This latter integral is a Gaussian integral if the limits of integration go from −∞ to +∞ (which can be assumed because the exponential decays very fast away from x₀), and thus it can be calculated. We find

A generalization of this method and extension to arbitrary precision is provided by Fog (2008).

Formal statement and proof:

Assume that is a twice differentiable function on with the unique point such that . Assume additionally that .

Then,

$\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) =1$

Proof:
Lower bound: Let . Then by the continuity of there exists such that if then . By Taylor's Theorem, for any, . Then we have the following lower bound: $\int_a^b e^{n f(x) } dx \ge \int_{x_0 - \delta}^{x_0 + \delta} e^{n f(x)} dx \ge e^{n f(x_0)} \int_{x_0 - \delta}^{x_0 + \delta} e^{\frac{n}{2} (f''(x_0) - \epsilon)(x-x_0)^2} dx = e^{n f(x_0)} \sqrt{\frac{1}{n (-f''(x_0) + \epsilon)}} \int_{-\delta \sqrt{n (-f''(x_0) + \epsilon)} }^{\delta \sqrt{n (-f''(x_0) + \epsilon)} } e^{-\frac{1}{2}y^2} dy$ where the last equality was obtained by a change of variables . Remember that so that is why we can take the square root of its negation. If we divide both sides of the above inequality by and take the limit we get: $\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \ge \lim_{n \to +\infty} \frac{1}{\sqrt{2 \pi}} \int_{-\delta\sqrt{n (-f''(x_0) + \epsilon)} }^{\delta \sqrt{n (-f''(x_0) + \epsilon)} } e^{-\frac{1}{2}y^2} dy \sqrt{\frac{-f''(x_0)}{-f''(x_0) + \epsilon}} = \sqrt{\frac{-f''(x_0)}{-f''(x_0) + \epsilon}}$ since this is true for arbitrary we get the lower bound: $\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \ge 1$ Upper bound: The proof of the upper bound is similar to the proof of the lower bound but there are a few inconveniences. Again we start by picking an but in order for the proof to work we need small enough so that . Then, as above, by continuity of and Taylor's Theorem we can find so that if, then . Lastly, by our assumptions there exists an such that if, then . Then we can calculate the following upper bound: $\int_a^b e^{n f(x) } dx \le \int_a^{x_0-\delta} e^{n f(x) } dx + \int_{x_0-\delta}^{x_0 + \delta} e^{n f(x) } dx + \int_{x_0 + \delta}^b e^{n f(x) } dx \le (a-b)e^{n (f(x_0) - \eta)} + \int_{x_0-\delta}^{x_0 + \delta} e^{n f(x) } dx$ $\le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \int_{x_0-\delta}^{x_0 + \delta} e^{\frac{1}{2} (f''(x_0)+\epsilon)(x-x_0)^2} dx \le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \int_{-\infty}^{+\infty} e^{\frac{1}{2} (f''(x_0)+\epsilon)(x-x_0)^2} dx$ $\le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \sqrt{\frac{2 \pi}{n (-f''(x_0) - \epsilon)}}$ If we divide both sides of the above inequality by and take the limit we get: $\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \le \lim_{n \to +\infty} \left( (a-b) e^{-\eta n} \sqrt{\frac{n (-f''(x_0))}{2 \pi}} + \sqrt{\frac{-f''(x_0)}{-f''(x_0) - \epsilon}} \right) = \sqrt{\frac{-f''(x_0)}{-f''(x_0) - \epsilon}}$ Since is arbitrary we get the upper bound: $\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \le 1$ And combining this with the lower bound gives the result.

Proof:

Lower bound:

Let . Then by the continuity of there exists such that if then . By Taylor's Theorem, for any, .

Then we have the following lower bound:

$\int_a^b e^{n f(x) } dx \ge \int_{x_0 - \delta}^{x_0 + \delta} e^{n f(x)} dx \ge e^{n f(x_0)} \int_{x_0 - \delta}^{x_0 + \delta} e^{\frac{n}{2} (f''(x_0) - \epsilon)(x-x_0)^2} dx = e^{n f(x_0)} \sqrt{\frac{1}{n (-f''(x_0) + \epsilon)}} \int_{-\delta \sqrt{n (-f''(x_0) + \epsilon)} }^{\delta \sqrt{n (-f''(x_0) + \epsilon)} } e^{-\frac{1}{2}y^2} dy$

where the last equality was obtained by a change of variables . Remember that so that is why we can take the square root of its negation.

If we divide both sides of the above inequality by and take the limit we get:

$\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \ge \lim_{n \to +\infty} \frac{1}{\sqrt{2 \pi}} \int_{-\delta\sqrt{n (-f''(x_0) + \epsilon)} }^{\delta \sqrt{n (-f''(x_0) + \epsilon)} } e^{-\frac{1}{2}y^2} dy \sqrt{\frac{-f''(x_0)}{-f''(x_0) + \epsilon}} = \sqrt{\frac{-f''(x_0)}{-f''(x_0) + \epsilon}}$

since this is true for arbitrary we get the lower bound:

$\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \ge 1$

Upper bound:

The proof of the upper bound is similar to the proof of the lower bound but there are a few inconveniences. Again we start by picking an but in order for the proof to work we need small enough so that . Then, as above, by continuity of and Taylor's Theorem we can find so that if, then . Lastly, by our assumptions there exists an such that if, then .

Then we can calculate the following upper bound:

$\int_a^b e^{n f(x) } dx \le \int_a^{x_0-\delta} e^{n f(x) } dx + \int_{x_0-\delta}^{x_0 + \delta} e^{n f(x) } dx + \int_{x_0 + \delta}^b e^{n f(x) } dx \le (a-b)e^{n (f(x_0) - \eta)} + \int_{x_0-\delta}^{x_0 + \delta} e^{n f(x) } dx$

$\le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \int_{x_0-\delta}^{x_0 + \delta} e^{\frac{1}{2} (f''(x_0)+\epsilon)(x-x_0)^2} dx \le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \int_{-\infty}^{+\infty} e^{\frac{1}{2} (f''(x_0)+\epsilon)(x-x_0)^2} dx$

$\le (a-b)e^{n (f(x_0) - \eta)} + e^{n f(x_0)} \sqrt{\frac{2 \pi}{n (-f''(x_0) - \epsilon)}}$

If we divide both sides of the above inequality by and take the limit we get:

$\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \le \lim_{n \to +\infty} \left( (a-b) e^{-\eta n} \sqrt{\frac{n (-f''(x_0))}{2 \pi}} + \sqrt{\frac{-f''(x_0)}{-f''(x_0) - \epsilon}} \right) = \sqrt{\frac{-f''(x_0)}{-f''(x_0) - \epsilon}}$

Since is arbitrary we get the upper bound:

$\lim_{n \to +\infty} \left( \frac{\int_a^b e^{nf(x)} dx}{\left( e^{nf(x_0)}\sqrt{\frac{2 \pi}{n (-f''(x_0))}} \right)} \right) \le 1$

And combining this with the lower bound gives the result.

Read more about this topic: Laplace's Method

Famous quotes containing the words general, theory, laplace and/or method:

“Through the particular, in wartime, I felt the high-voltage current of the general pass.”
—Elizabeth Bowen (1899–1973)

“The human species, according to the best theory I can form of it, is composed of two distinct races, the men who borrow and the men who lend.”
—Charles Lamb (1775–1834)

“Given for one instant an intelligence which could comprehend all the forces by which nature is animated and the respective positions of the beings which compose it, if moreover this intelligence were vast enough to submit these data to analysis, it would embrace in the same formula both the movements of the largest bodies in the universe and those of the lightest atom; to it nothing would be uncertain, and the future as the past would be present to its eyes.”
—Pierre Simon De Laplace (1749–1827)

“The insidiousness of science lies in its claim to be not a subject, but a method. You could ignore a subject; no subject is all-inclusive. But a method can plausibly be applied to anything within the field of consciousness.”
—Katharine Fullerton Gerould (1879–1944)