IEEE Transactions on Parallel and Distributed Systems · 2021 · 40 citations · 57 references
ComputingComputer EngineeringComputer ScienceMachine LearningParallel AlgorithmsParallel ComputingData ParallelismGraphics Processing Unit ComputingSecure Multi-party ComputationComputational SecurityData ProcessingGpu SecurityInter-node Communication OptimizationInter-process CommunicationIntra-node Communication OptimizationParallel Machine LearningSecure Machine Learning
Machine learning is widely used in our daily lives. Large amounts of data have been continuously produced and transmitted to the cloud for model training and data processing, which raises a problem: how to preserve the security of the data. Recently, a secure machine learning system named SecureML has been proposed to solve this issue using two-party computation. However, due to the excessive computation expenses of two-party computation, the secure machine learning is about 2× slower than the original machine learning methods. Previous work on secure machine learning mostly focused on novel protocols or improving accuracy, while the performance metric has been ignored. In this article, we propose a GPU-based framework ParSecureML to improve the performance of secure machine learning algorithms based on two-party computation. The main challenges of developing ParSecureML lie in the complex computation patterns, frequent intra-node data transmission between CPU and GPU, and complicated inter-node data dependence. To handle these challenges, we propose a series of novel solutions, including profiling-guided adaptive GPU utilization, fine-grained double pipeline for intra-node CPU-GPU cooperation, and compressed transmission for inter-node communication. Moreover, we integrate architecture specific optimizations, such as Tensor Cores, into ParSecureML. As far as we know, this is the first GPU-based secure machine learning framework. Compared to the state-of-the-art framework, ParSecureML achieves an average of 33.8× speedup. ParSecureML can also be applied to inferences, which achieves 31.7× speedup on average.
57
Pattern Recognition and Machine Learning
Journal of Electronic Imaging · 2007 · 22.1K citations
Yangqing Jia, Evan Shelhamer, Jeff Donahue et al. · 2014 · 11.2K citations
Least Squares Support Vector Machine Classifiers
Johan A. K. Suykens, Joos Vandewalle · Neural Processing Letters · 1999 · 9.4K citations · Full text
TensorFlow: A system for large-scale machine learning
Martı́n Abadi, Paul Barham, Jianmin Chen et al. · arXiv (Cornell University) · 2016 · 8.8K citations · Full text
Pattern Recognition and Machine Learning
Kybernetes · 2007 · 8.4K citations