IEEE Transactions on Circuits and Systems for Video Technology · 2023 · 20 citations · 60 references
ComputingComputer ScienceEducational TechnologyArtificial Intelligence In EducationComputer VisionMachine LearningImage ClassificationMultimodal InstructionMultimodal InteractionRemote Sensing Image AnalysisRemote Sensing ImageryClassification AccuracyFeature FusionHistogram EqualizationImage Feature FusionMultimodal Data FusionMultimodal Feature FusionMultimodal Feature RepresentationMultimodal Fusion
A growing number of earth observation satellites are able to simultaneously gather multimodal images of the same area due to the expanding availability and resolution of satellite remote sensing data. This paper proposes a novel multimodal balanced self-learning interaction network (MBSI-Net) for the classification task. It involves a dual-branch teacher-student network that enables knowledge interaction and transfer between the multimodalities. Firstly, in order to introduce statistical information in addition to local and global structural information, a texture feature equalization module (TFE-Module) is proposed. This can enhance the texture information of features through histogram equalization and further improve the representation ability of features. Secondly, to enable the student network to provide timely feedback questions, the paper proposes a feature fusion module (F <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -Module) that models and enhances teacher features through the student network. This helps to raise the classification’s accuracy by incorporating information from multimodal images. Finally, the paper proposes a loss function based on structural similarity analysis to ensure balanced self-learning between the student and the teacher networks. Taking the multispectral (MS) and the panchromatic (PAN) images of the same scene as examples, through experimental verification, the proposed method can achieve good results on multiple datasets compared with other methods. Therefore, it offers an effective method for classifying and fusing multimodal data.
60
DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · 2023 · 76.3K citations · Full text
Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals · arXiv (Cornell University) · 2015 · 14K citations · Full text
Momentum Contrast for Unsupervised Visual Representation Learning
Kaiming He, Haoqi Fan, Yuxin Wu et al. · 2020 · 12K citations
SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers
Danfeng Hong, Zhu Han, Jing Yao et al. · arXiv (Cornell University) · 2021 · 1.2K citations · Full text