IEEE Internet of Things Journal · 2023 · 16 citations · 25 references
ComputingComputer ScienceComputer SystemsDistributed SystemsEdge ComputingServerless ComputingContainerization At The EdgeEdge OrchestrationEdge Resource ManagementEdge Serverless ComputingEdge-cloud ArchitectureLatency OptimizationEdge CloudEdge Cloud OrchestrationEdge Computing NetworksMarkov Decision ModelsResource Consumption AnalysisServerless Edge ComputingSplitting
Distributed serverless edge clouds and split computing are promising technologies to reduce the inference latency of large-scale deep neural networks (DNNs). In this article, we propose a dynamic split computing framework (DSCF) in distributed serverless edge clouds. In DSCF, the edge cloud orchestrator dynamically determines 1) splitting point and 2) warm status maintenance of container instances (i.e., whether or not to maintain each container instance in a warm status). For optimal decisions, we formulate a constrained Markov decision process (CMDP) problem to minimize the inference latency while maintaining the average resource consumption of distributed edge clouds below a certain level. The optimal stochastic policy can be obtained by converting the CMDP model into a linear programming (LP) model. The evaluation results demonstrate that DSCF can achieve less than half the inference latency compared to the local computing scheme while maintaining sufficient low resource consumption of distributed edge clouds.
25
Efficient Processing of Deep Neural Networks: A Tutorial and Survey
Vivienne Sze, Yu‐Hsin Chen, Tien-Ju Yang et al. · Proceedings of the IEEE · 2017 · 4K citations
Serverless Computing: Design, Implementation, and Performance
Garrett McGrath, Paul Brenner · 2017 · 360 citations
Stefanos Laskaridis, Stylianos I. Venieris, Mário Almeida et al. · 2020 · 282 citations · Full text