Information Bottleneck Method - Defining Decision Contours

Defining Decision Contours

To categorize a new sample external to the training set, apply the previous distance metric to find the transition probabilities between and all samples in, with a normalisation. Secondly apply the last two lines of the 3-line algorithm to get cluster, and conditional category probabilities.

\begin{align}
& \tilde p(c_i ) = p(c_i | x' ) = \sum_j p(c_i | x_j)p(x_j | x') =\sum_j p(c_i | x_j) \tilde p(x_j)\\
& p(y_i | c_j) = \sum_k p(y_i | x_k) p(c_j | x_k)p(x_k | x') / p(c_j | x' )
= \sum_k p(y_i | x_k) p(c_j | x_k) \tilde p(x_k) / \tilde p(c_j) \\
\end{align}

Finally we have

Parameter must be kept under close supervision since, as it is increased from zero, increasing numbers of features, in the category probability space, snap into focus at certain critical thresholds.

Read more about this topic:  Information Bottleneck Method

Famous quotes containing the words defining, decision and/or contours:

    Art, if one employs this term in the broad sense that includes poetry within its realm, is an art of creation laden with ideals, located at the very core of the life of a people, defining the spiritual and moral shape of that life.
    Ivan Sergeevich Turgenev (1818–1883)

    Concision in style, precision in thought, decision in life.
    Victor Hugo (1802–1885)

    Pregnant women! They had that weird frisson, an aura of magic that combined awkwardly with an earthy sense of duty. Mundane, because they were nothing unique on the suburban streets; ethereal because their attention was ever somewhere else. Whatever you said was trivial. And they had that preciousness which they imposed wherever they went, compelling attention, constantly reminding you that they carried the future inside, its contours already drawn, but veiled, private, an inner secret.
    Ruth Morgan (1920–1978)