A multi-task framework with feature passing module for skin lesion classification and segmentation

Abstract

Skin lesion classification and segmentation are highly correlated tasks. However, their relationship is not fully utilized in previous methods. In this paper, we propose a multi-task deep convolutional neural network architecture to solve the skin lesion classification and segmentation problem simultaneously. To take full advantage of features from different tasks and thus get richer knowledge about the sample, we design a feature passing module to pass messages between segmentation branch and classification branch. Since feature passing module is not always helpful and can be related with individual samples, gate functions are used for controlling messages transmission. Therefore, features from one task are learned and selectively passed to the other task, and vice versa, which effectively improves the performance of both tasks. We have evaluated the proposed method on ISBI-2017 challenge dataset, and the experimental results demonstrate the superiority and effectiveness of the proposed method, compared to our base model and other state-of-art methods.