IEEE Transactions on Multimedia · 2020 · 27 citations · 56 references
MedicineComputer ScienceVision ScienceComputer VisionVisual AttentionVisual PerceptionObject-based AttentionVisibility DeterminationField Of ViewSaliency DetectionSaliency EstimationSaliency MappingVisual Attention GuidanceVisual Attention In 360-Degree MediaVisual SaliencyVisual Saliency Detection
360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> media allows observers to explore the scene in all directions. The consequence is that the human visual attention is guided by not only the perceived area in the viewport but also the overall content in 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> . In this paper, we propose a method to estimate the 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> saliency map which extracts salient features from the entire 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> image in each viewport in three different Field of Views (FoVs). Our model is first pretrained with a large-scale 2D image dataset to enable the interpretation of semantic contents, then fine-tuned with a relative small 360 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> image dataset. A novel weighting loss function attached with stretch weighted maps is introduced to adaptively weight the losses of three evaluation metrics and attenuate the impact of stretched regions in equirectangular projection during training process. Experimental results demonstrate that our model achieves better performance with the integration of three FoVs and its diverse viewport images. Results also show that the adaptive weighting losses and stretch weighted maps effectively enhance the evaluation scores compared to the fixed weighting losses solutions. Comparing to other state of the art models, our method surpasses them on three different datasets and ranks the top using 5 performance evaluation metrics on the Salient360! benchmark set. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/FannyChao/MV-SalGAN360</uri> .
56
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Learning to predict where humans look
Tilke Judd, Krista A. Ehinger, Frédo Durand et al. · 2009 · 2K citations · Full text
The Secrets of Salient Object Segmentation
Yin Li, Xiaodi Hou, Christof Koch et al. · 2014 · 1.4K citations · Full text