LPBoost - Base Learners

Base Learners

LPBoost is an ensemble learning method and thus does not dictate the choice of base learners, the space of hypotheses . Demiriz et al. showed that under mild assumptions, any base learner can be used. If the base learners are particularly simple, they are often referred to as decision stumps.

The number of base learners commonly used with Boosting in the literature is large. For example, if, a base learner could be a linear soft margin support vector machine. Or even more simple, a simple stump of the form

$h(\boldsymbol{x} ; \omega \in \{1,-1\}, p \in \{1,\dots,n\}, t \in {\mathbb R}) := \left\{\begin{array}{cl} \omega & \textrm{if~} \boldsymbol{x}_p \leq t\\ -\omega & \textrm{otherwise}\end{array}\right..$

The above decision stumps looks only along a single dimension of the input space and simply thresholds the respective column of the sample using a constant threshold . Then, it can decide in either direction, depending on for a positive or negative class.

Given weights for the training samples, constructing the optimal decision stump of the above form simply involves searching along all sample columns and determining, and in order to optimize the gain function.

Famous quotes containing the word base:

“What I often forget about students, especially undergraduates, is that surface appearances are misleading. Most of them are at base as conventional as Presbyterian deacons.”
—Muriel Beadle (b. 1915)