The Decomposition Algorithm

As in the classification algorithm proposed by Joachims [5], which was based on a idea from Osuna et al [11], our regression algorithm is subdivided into the following four steps, which are explained afterward in the following subsections:

1.

Select q variables $\alpha_i$ or $\alpha_i^{\star}$ as the new working set, called $\mathcal{S}$.

2.

Fix the other variables $\mathcal{F}$ to their current values and solve the problem (3) with respect to $\mathcal{S}$.

3.

Search for variables whose values have been at 0 or C for a long time and that will probably not change anymore. This optional step is the shrinking phase, as these variables are removed from the problem.

4.

Test whether the optimization is finished; if not, return to the first step.

Many other decomposition algorithms for regression have been published recently, and a comparison is given later in section 2.6.