CoinNet: Copy Initialization Network for Multispectral Imagery Semantic Segmentation

Abstract

Remote sensing imagery semantic segmentation refers to assigning a label to every pixel. Recently, deep convolutional neural networks (CNNs)-based methods have presented an impressive performance in this task. Due to the lack of sufficient labeled remote sensing images, researchers usually utilized transfer learning (TL) strategies to fine tune networks which were pretrained in huge RGB-scene data sets. Unfortunately, this manner may not work if the target images are multispectral/hyperspectral. The basic assumption of TL is that the low-level features extracted by the former layers are similar in most data sets, hence users only require to train the parameters in the last layers that are specific to different tasks. However, if one should use a pretrained deep model in RGB data for multispectral /hyperspectral imagery semantic segmentation, the structure of the input layer has to be adjusted. In this case, the first convolutional layer has to be trained using the multispectral /hyperspectral data sets which are much smaller. Apparently, the feature representation ability of the first convolutional layer will decrease and it may further harm the following layers. In this letter, we propose a new deep learning model, COpy INitialization Network (CoinNet), for multispectral imagery semantic segmentation. The major advantage of CoinNet is that it can make full use of the initial parameters in the pretrained network's first convolutional layer. Comparison experiments on a challenging multispectral data set have demonstrated the effectiveness of the proposed improvement. The demo and a trained network will be published in our homepage.