Learning to Predict Perceptual Distributions of Haptic Adjectives

TLDR

Human touch allows immediate description of objects using haptic adjectives, but perception is subjective and noisy, and prior robotic work has focused only on binary adjectives, ignoring intensity and variability. This study develops a machine‑learning approach that, from a single robot interaction, predicts a probability distribution over ordinal haptic adjectives by leveraging partial label‑distribution knowledge. The method trains on multi‑modal tactile features extracted from repeated robot touches of 60 objects, and evaluates feature importance to assess each sensor modality’s influence on adjective prediction. Results show that modeling both intensity and perceptual variation of haptic adjectives is feasible, addressing previously neglected aspects of human haptic perception.

Abstract

When humans touch an object with their fingertips, they can immediately describe its tactile properties using haptic adjectives, such as hardness and roughness; however, human perception is subjective and noisy, with significant variation across individuals and interactions. Recent research has worked to provide robots with similar haptic intelligence but was focused on identifying binary haptic adjectives, ignoring both attribute intensity and perceptual variability. Combining ordinal haptic adjective labels gathered from human subjects for a set of 60 objects with features automatically extracted from raw multi-modal tactile data collected by a robot repeatedly touching the same objects, we designed a machine-learning method that incorporates partial knowledge of the distribution of object labels into training; then, from a single interaction, it predicts a probability distribution over the set of ordinal labels. In addition to analyzing the collected labels (10 basic haptic adjectives) and demonstrating the quality of our method's predictions, we hold out specific features to determine the influence of individual sensor modalities on the predictive performance for each adjective. Our results demonstrate the feasibility of modeling both the intensity and the variation of haptic perception, two crucial yet previously neglected components of human haptic perception.