Kristian Woodsend and J. Gondzio
Abstract
Support Vector Machines are a powerful machine learning technology,
but the training process involves a dense quadratic optimization
problem and is computationally challenging. A parallel implementation
of Support Vector Machine training has been developed, using a
combination of MPI and OpenMP. Using an interior point method for
the optimization and a reformulation that avoids the dense Hessian matrix,
the structure of the augmented system matrix is exploited to partition
data and computations amongst parallel processors efficiently.
The new implementation has been applied to solve problems from the
PASCAL Challenge on Large Scale Learning. We show that our approach
is competitive, and is able to solve problems in the Challenge
many times faster than other parallel approaches. We also demonstrate
that the hybrid version performs more efficiently than the version using
pure MPI.
Key words: Parallel support vector machines, Interior point method, Separable quadratic program, MPI/OpenMP Implementation.