Scalable Semi-Supervised SVM via Triply Stochastic Gradients
Abstract
Semi-supervised learning (SSL) plays an increasingly important role in the big data era because a large number of unlabeled samples can be used effectively to improve the performance of the classifier. Semi-supervised support vector machine (SVM) is one of the most appealing methods for SSL, but scaling up SVM for kernel learning is still an open problem. Recently, a doubly stochastic gradient (DSG) algorithm has been proposed to achieve efficient and scalable training for kernel methods. However, the algorithm and theoretical analysis of DSG are developed based on the convexity assumption which makes them incompetent for non-convex problems such as SVM. To address this problem, in this paper, we propose a triply stochastic gradient algorithm for SVM, called TSGSVM. Specifically, to handle two types of data instances involved in SVM, TSGSVM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient. We use the approximated gradient to update the solution. More importantly, we establish new theoretic analysis for TSGSVM which guarantees that TSGSVM can converge to a stationary point. Extensive experimental results on a variety of datasets demonstrate that TSGSVM is much more efficient and scalable than existing SVM algorithms.
Cite
@article{arxiv.1907.11584,
title = {Scalable Semi-Supervised SVM via Triply Stochastic Gradients},
author = {Xiang Geng and Bin Gu and Xiang Li and Wanli Shi and Guansheng Zheng and Heng Huang},
journal= {arXiv preprint arXiv:1907.11584},
year = {2019}
}