2024 · 13 citations · 38 references
MedicineComputer ScienceComputer VisionVisual AttentionVisual PerceptionAttention NetworksAttentional ControlEye-movement ControlReinforcement LearningSpatial AttentionUser ModelingAttentionDeep Reinforcement LearningGaze-driven InteractionGraphical User InterfacesPersonalized User InterfacesReinforcement Learning For VisionTransformer TheoryViewer Attention
From a visual-perception perspective, modern graphical user interfaces (GUIs) comprise a complex graphics-rich two-dimensional visuospatial arrangement of text, images, and interactive objects such as buttons and menus. While existing models can accurately predict regions and objects that are likely to attract attention “on average”, no scanpath model has been capable of predicting scanpaths for an individual. To close this gap, we introduce EyeFormer, which utilizes a Transformer architecture as a policy network to guide a deep reinforcement learning algorithm that predicts gaze locations. Our model offers the unique capability of producing personalized predictions when given a few user scanpath samples. It can predict full scanpath information, including fixation positions and durations, across individuals and various stimulus types. Additionally, we demonstrate applications in GUI layout optimization driven by our model.
38
Reinforcement Learning: An Introduction
IEEE Transactions on Neural Networks · 2005 · 25.8K citations
Ming Jiang, Shengsheng Huang, Juanyong Duan et al. · 2015 · 712 citations
Predicting human gaze beyond pixels
Jinhong Xu, Ming Jiang, Shuo Wang et al. · Journal of Vision · 2014 · 356 citations · Full text