Proceedings of the VLDB Endowment · 2022 · 13 citations · 10 references
ComputingComputer ScienceComputer SystemsEdge ComputingHeterogeneous ComputingMachine LearningEdge Computing InfrastructureEdge InfrastructureEdge-cloud ArchitectureEdge-cloud IntegrationHigh-throughput ComputingOptimization For Machine LearningAutomlAutoml For Edge DevicesCompressionCost ModelsHeterogeneous EnvironmentsMachine Learning WorkloadsModel CompressionModel-based Compression
With the advent of ubiquitous deployment of smart devices and the Internet of Things, data sources for machine learning inference have increasingly moved to the edge of the network. Existing machine learning inference platforms typically assume a homogeneous infrastructure and do not take into account the more complex and tiered computing infrastructure that includes edge devices, local hubs, edge datacenters, and cloud datacenters. On the other hand, recent AutoML efforts have provided viable solutions for model compression, pruning and quantization for heterogeneous environments; for a machine learning model, now we may easily find or even generate a series of model variants with different tradeoffs between accuracy and efficiency. We design and implement JellyBean, a system for serving and optimizing machine learning inference workflows on heterogeneous infrastructures. Given service-level objectives (e.g., throughput, accuracy), JellyBean picks the most cost-efficient models that meet the accuracy target and decides how to deploy them across different tiers of infrastructures. Evaluations show that JellyBean reduces the total serving cost of visual question answering by up to 58% and vehicle tracking from the NVIDIA AI City Challenge by up to 36%, compared with state-of-the-art model selection and worker assignment solutions. JellyBean also outperforms prior ML serving systems (e.g., Spark on the cloud) up to 5x in serving costs.
10
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
1993 · 14.3K citations
Transformers: State-of-the-Art Natural Language Processing
Thomas Wolf, Lysandre Debut, Victor Sanh et al. · 2020 · 8K citations · Full text
Knowledge Distillation: A Survey
Jianping Gou, Baosheng Yu, Stephen J. Maybank et al. · International Journal of Computer Vision · 2021 · 3.5K citations · Full text
Computing, Chemical Engineering, Knowledge Representation +2
Yacc : Yet Another Compiler Compiler
S. C. Johnson, Murray Hill · 1978 · 1.2K citations
MLlib: Machine Learning in Apache Spark
Xiangrui Meng, Joseph K. Bradley, Burak Yavuz et al. · arXiv (Cornell University) · 2015 · 961 citations · Full text