IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · 2022 · 12 citations · 14 references
ComputingComputer ScienceMachine LearningOptimizationWireless CommunicationsExploration-exploitation TradeoffReinforcement LearningBandit AlgorithmsBandwidthKullback-leibler DivergenceMulti-armed Bandit OptimizationThroughputThroughput ControlWireless Data TransmissionWireless Transmission
Interactive panoramic scene delivery not only consumes 4∼6× more bandwidth than traditional video streaming of the same resolution but also requires timely displaying the delivered content to ensure smooth interaction. Since users can only see roughly 20% of the entire scene at a time (called the viewport), it is sufficient to deliver the relevant portion of the panoramic scene if we can accurately predict the user’s motion. It is customary to deliver a portion larger than the viewport to tolerate inaccurate predictions. Intuitively, the larger the delivered portion, the higher the prediction accuracy and lower the wireless transmission success probability. The goal is to select an appropriate delivery portion to maximize system throughput. We formulate this problem as a multi-armed bandit problem and use the classical Kullback-Leibler Upper Confidence Bound (KL-UCB) algorithm for the portion selection. We further develop a novel variant of the KL-UCB algorithm that effectively leverages two-level feedback (i.e., both prediction and transmission outcomes) after each decision on the selected portion and show its asymptotical optimality, which may be of independent interest by itself. We demonstrate the superior performance of our proposed algorithms over existing heuristic methods using both synthetic simulations and real experimental evaluations.
14
Asymptotically efficient adaptive allocation rules
Tze-Leung Lai, Herbert Robbins · Advances in Applied Mathematics · 1985 · 2.5K citations
Feng Qian, Bo Han, Qingyang Xiao et al. · 2018 · 284 citations · Full text