next up previous
Next: Acknowledgements Up: Lagrangian Support Vector Machines Previous: 5. Numerical Implementation and


6. Conclusion

A fast and extremely simple algorithm, LSVM, considerably easier to code than SVM$^{light}$ [11], SMO [24], SOR [17] and ASVM [18], capable of classifying datasets with millions of points has been proposed and implemented in a few lines of MATLAB code. For a linear kernel, LSVM is an iterative method which requires nothing more complex than the inversion of a single matrix of the order of the input space plus one, and thus has the ability to handle massive problems. For a positive semidefinite nonlinear kernel, a single matrix inversion is required in the space of dimension equal to the number of points classified. Hence, for such nonlinear classifiers LSVM can handle only intermediate size problems.

There is room for future work in reducing the number of support vectors in the solutions yielded by LSVM. One method would be to augment the quadratic term in $y$ in the objective function of (7) by a 1-norm in $y$. This should decrease the number of support vectors as in work by [2] where the 1-norm was used to effectively suppress features. Additionally, the iterative algorithm itself could be modified so that dual variables with values smaller than a tolerance would be automatically set to zero.

Further future work includes extensions to parallel processing of the data and handling very large datasets directly from disk as well as extending nonlinear kernel classification to very large datasets.


next up previous
Next: Acknowledgements Up: Lagrangian Support Vector Machines Previous: 5. Numerical Implementation and
Journal of Machine Learning Research