International Journal of Applied Earth Observation and Geoinformation · 2023 · 15 citations · 45 references
ComputingComputer ScienceKnowledge TransferMachine LearningRemote SensingDeep LearningEnsemble LearningEnsemble MethodsNeural NetworksRepresentation LearningStudent ModelingDeep Feature FusionDeep Learning FusionDeep Learning-based Data FusionFeature Map FusionKnowledge DistillationMixture Of Experts FusionRoad Feature DetectionTeacher Networks
Knowledge distillation is one effective approach to compress deep learning models. However, the current distillation methods are relatively monotonous. There are still rare studies about the combination of distillation strategies using multiple types of knowledge and employing multiple teacher models. Besides, how to optimize the weights among different teacher models is still an open problem. To address these issues, this paper proposes a novel approach for knowledge distillation, which effectively enhances the robustness of the distilled student model by a weights adaptive multi-teacher collaborative distillation. Moreover, the proposed method utilizes feature knowledge exchange guidance between teacher networks to transfer more comprehensive feature knowledge to the student model, which further improves the learning capability of hidden layers’ details. The extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance on Massachusetts Roads Dataset, LRSNY Roads Dataset, and WHU Building Dataset. Specifically, under the guidance of the first ensemble of teacher networks, we obtained IoU scores of 47.33%, 78.15%, and 80.71%, respectively. Under the guidance of the second ensemble of teacher networks, we obtained IoU scores of 48.56%, 79.51%, and 81.35%, respectively.
45
Feature Pyramid Networks for Object Detection
Tsung-Yi Lin, Piotr Dollár, Ross Girshick et al. · 2017 · 28.7K citations
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2017 · 22.1K citations
Focal Loss for Dense Object Detection
Tsung-Yi Lin, Priya Goyal, Ross Girshick et al. · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2018 · 9.6K citations
China and India lead in greening of the world through land-use management
Chi Chen, Taejin Park, Xuhui Wang et al. · Nature Sustainability · 2019 · 2.9K citations · Full text