Hybrid MPI/OpenMP Parallel Linear Support Vector Machine Training

Technical Report ERGO-09-001

Kristian Woodsend and J. Gondzio

Support Vector Machines are a powerful machine learning technology, but the training process involves a dense quadratic optimization problem and is computationally challenging. A parallel implementation of Support Vector Machine training has been developed, using a combination of MPI and OpenMP. Using an interior point method for the optimization and a reformulation that avoids the dense Hessian matrix, the structure of the augmented system matrix is exploited to partition data and computations amongst parallel processors efficiently. The new implementation has been applied to solve problems from the PASCAL Challenge on Large Scale Learning. We show that our approach is competitive, and is able to solve problems in the Challenge many times faster than other parallel approaches. We also demonstrate that the hybrid version performs more efficiently than the version using pure MPI.

Key words: Parallel support vector machines, Interior point method, Separable quadratic program, MPI/OpenMP Implementation.

PDF ERGO-09-001.pdf.

Written: January 6, 2009, revised May 22, 2009.
Appeared in: Journal of Machine Learning Research 10 (2009) pp. 1937--1953.

Related Software:
OOPS Object-Oriented Parallel Solver.
SVM-OOPS Extention of OOPS for SVM problems.