arXiv (Cornell University) · 2017 · 111 citations · 31 references
EngineeringComputer ScienceRobotics EngineeringGraspingRoboticsSensor FusionGrasp DetectionManipulation PerceptionMulti-fingered GraspingTactile PerceptionTactile SensingTactile Sensing For GraspingGraspGrasp Success PredictionHigh-resolution Tactile SensingMultimodal SensingMultimodal Tactile SensingRobotic GraspingTactile SensorsTouch Sensing
Robotic grasp success hinges on precise contact force balance, and inferring success from vision alone is difficult, making direct touch sensing an attractive yet challenging approach. This study examines whether adding touch sensing to vision improves grasp outcome prediction accuracy in a multimodal framework. We performed over 9,000 trials with a two‑finger gripper equipped with GelSight tactile sensors and trained deep neural networks to predict grasp outcomes from vision alone, touch alone, or both. Results show that incorporating tactile readings substantially improves grasping performance.
A successful grasp requires careful balancing of the contact forces. Deducing whether a particular grasp will be successful from indirect measurements, such as vision, is therefore quite challenging, and direct sensing of contacts through touch sensing provides an appealing avenue toward more successful and consistent robotic grasping. However, in order to fully evaluate the value of touch sensing for grasp outcome prediction, we must understand how touch sensing can influence outcome prediction accuracy when combined with other modalities. Doing so using conventional model-based techniques is exceptionally difficult. In this work, we investigate the question of whether touch sensing aids in predicting grasp outcomes within a multimodal sensing framework that combines vision and touch. To that end, we collected more than 9,000 grasping trials using a two-finger gripper equipped with GelSight high-resolution tactile sensors on each finger, and evaluated visuo-tactile deep neural network models to directly predict grasp outcomes from either modality individually, and from both modalities together. Our experimental results indicate that incorporating tactile readings substantially improve grasping performance.
31
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Adam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba · UvA-DARE (University of Amsterdam) · 2014 · 84.8K citations · Full text
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher et al. · 2009 IEEE Conference on Computer Vision and Pattern Recognition · 2009 · 61.8K citations
Jiquan Ngiam, Aditya Khosla, Mingyu Kim et al. · 2011 · 2.3K citations