Normal-gamma Distribution - Posterior Distribution of The Parameters

Posterior Distribution of The Parameters

Assume that x is distributed according to a normal distribution with unknown mean and precision .

and that the prior distribution on and, has a normal-gamma distribution

$(\mu,\tau) \sim \text{NormalGamma}(\mu_0,\lambda_0,\alpha_0,\beta_0) ,$

for which the density π satisfies

$\pi(\mu,\tau) \propto \tau^{\alpha_0-\frac{1}{2}}\,\exp\,\exp.$

Given a dataset, consisting of independent and identically distributed random_variables (i.i.d), the posterior distribution of and given this dataset can be analytically determined by Bayes' theorem. Explicitly,

where is the likelihood of the data given the parameters.

Since the data are i.i.d, the likelihood of the entire dataset is equal to the product of the likelihoods of the individual data samples:

$\mathbf{L}(\mathbf{X} | \tau, \mu) = \prod_{i=1}^n \mathbf{L}(x_i | \tau, \mu) .$

This expression can be simplified as follows:

$\begin{align} \mathbf{L}(\mathbf{X} | \tau, \mu) & \propto \prod_{i=1}^n \tau^{1/2} \exp \\ & \propto \tau^{n/2} \exp \\ & \propto \tau^{n/2} \exp \\ & \propto \tau^{n/2} \exp \\ & \propto \tau^{n/2} \exp , \end{align}$

where, the mean of the data samples, and, the sample variance.

The posterior distribution of the parameters is proportional to the prior times the likelihood.

$\begin{align} \mathbf{P}(\tau, \mu | \mathbf{X}) &\propto \mathbf{L}(\mathbf{X} | \tau,\mu) \pi(\tau,\mu) \\ &\propto \tau^{n/2} \exp \tau^{\alpha_0-\frac{1}{2}}\,\exp\,\exp \\ &\propto \tau^{\frac{n}{2} + \alpha_0 - \frac{1}{2}}\exp \exp\left \\ \end{align}$

The final exponential term is simplified by completing the square.

$\begin{align} \lambda_0(\mu-\mu_0)^2 + n(\bar{x} -\mu)^2&=\lambda_0 \mu^2 - 2 \lambda_0 \mu \mu_0 + \lambda_0 \mu_0^2 + n \mu^2 - 2 n \bar{x} \mu + n \bar{x}^2 \\ &= (\lambda_0 + n) \mu^2 - 2(\lambda_0 \mu_0 + n \bar{x}) \mu + \lambda_0 \mu_0^2 +n \bar{x}^2 \\ &= (\lambda_0 + n)( \mu^2 - 2 \frac{\lambda_0 \mu_0 + n \bar{x}}{\lambda_0 + n} \mu ) + \lambda_0 \mu_0^2 +n \bar{x}^2 \\ &= (\lambda_0 + n)\left(\mu - \frac{\lambda_0 \mu_0 + n \bar{x}}{\lambda_0 + n} \right) ^2 + \lambda_0 \mu_0^2 +n \bar{x}^2 - \left( \frac{\lambda_0 \mu_0 +n \bar{x}}{\lambda_0 + n} \right)^2 \\ &= (\lambda_0 + n)\left(\mu - \frac{\lambda_0 \mu_0 + n \bar{x}}{\lambda_0 + n} \right) ^2 + \frac{\lambda_0 n (\bar{x} - \mu_0 )^2}{\lambda_0 +n} \end{align}$

On inserting this back into the expression above,

$\begin{align} \mathbf{P}(\tau, \mu | \mathbf{X}) & \propto \tau^{\frac{n}{2} + \alpha_0 - \frac{1}{2}} \exp \left \exp \left\\ & \propto \tau^{\frac{n}{2} + \alpha_0 - \frac{1}{2}} \exp \left \exp \left \end{align}$

This final expression is in exactly the same form as a Normal-Gamma distribution, i.e.,

$\mathbf{P}(\tau, \mu | \mathbf{X}) = \text{NormalGamma}\left(\frac{\lambda_0 \mu_0 + n \bar{x}}{\lambda_0 + n}, \lambda_0 + n, \alpha_0+\frac{n}{2}, \beta_0+ \frac{1}{2}\left(n s + \frac{\lambda_0 n (\bar{x} - \mu_0 )^2}{\lambda_0 +n} \right) \right)$

Read more about this topic: Normal-gamma Distribution

Famous quotes containing the words distribution and/or parameters:

“The man who pretends that the distribution of income in this country reflects the distribution of ability or character is an ignoramus. The man who says that it could by any possible political device be made to do so is an unpractical visionary. But the man who says that it ought to do so is something worse than an ignoramous and more disastrous than a visionary: he is, in the profoundest Scriptural sense of the word, a fool.”
—George Bernard Shaw (1856–1950)

“Men have defined the parameters of every subject. All feminist arguments, however radical in intent or consequence, are with or against assertions or premises implicit in the male system, which is made credible or authentic by the power of men to name.”
—Andrea Dworkin (b. 1946)