A deep transfer learning based convolution neural network framework for air temperature classification using human clothing images

Abstract

Weather recognition is crucial due to its significant impact on various aspects of daily life, such as weather prediction, environmental monitoring, tourism, and energy production. Several studies have already conducted research on image-based weather recognition. However, previous studies have addressed few types of weather phenomena recognition from images with insufficient accuracy. In this paper, we propose a transfer learning CNN framework for classifying air temperature levels from human clothing images. The framework incorporates various deep transfer learning approaches, including DeepLabV3 Plus for semantic segmentation and others for classification such as BigTransfer (BiT), Vision Transformer (ViT), ResNet101, VGG16, VGG19, and DenseNet121. Meanwhile, we have collected a dataset called the Human Clothing Image Dataset (HCID), consisting of 10,000 images with two categories (High and Low air temperature). All the models were evaluated using various classification metrics, such as the confusion matrix, loss, precision, F1-score, recall, accuracy, and AUC-ROC. Additionally, we applied Gradient-weighted Class Activation Mapping (Grad-CAM) to emphasize significant features and regions identified by models during the classification process. The results show that DenseNet121 outperformed other models with an accuracy of 98.13%. Promising experimental results highlight the potential benefits of the proposed framework for detecting air temperature levels, aiding in weather prediction and environmental monitoring.