FuseBot: mechanical search of rigid and deformable objects via multi-modal perception
-
Published:2023-09-23
Issue:8
Volume:47
Page:1137-1154
-
ISSN:0929-5593
-
Container-title:Autonomous Robots
-
language:en
-
Short-container-title:Auton Robot
Author:
Boroushaki Tara,Dodds Laura,Naeem Nazish,Adib Fadel
Abstract
AbstractMechanical search is a robotic problem where a robot needs to retrieve a target item that is partially or fully-occluded from its camera. State-of-the-art approaches for mechanical search either require an expensive search process to find the target item, or they require the item to be tagged with a radio frequency identification tag (e.g., RFID), making their approach beneficial only to tagged items in the environment. We present FuseBot, the first robotic system for RF-Visual mechanical search that enables efficient retrieval of both RF-tagged and untagged items in a pile. Rather than requiring all target items in a pile to be RF-tagged, FuseBot leverages the mere existence of an RF-tagged item in the pile to benefit both tagged and untagged items. Our design introduces two key innovations. The first is RF-Visual Mapping, a technique that identifies and locates RF-tagged items in a pile and uses this information to construct an RF-Visual occupancy distribution map. The second is RF-Visual Extraction, a policy formulated as an optimization problem that minimizes the number of actions required to extract the target object by accounting for the probabilistic occupancy distribution, the expected grasp quality, and the expected information gain from future actions. We built a real-time end-to-end prototype of our system on a UR5e robotic arm with in-hand vision and RF perception modules. We conducted over 200 real-world experimental trials to evaluate FuseBot and compare its performance to a state-of-the-art vision-based system named X-Ray (Danielczuk et al., in: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), IEEE, 2020). Our experimental results demonstrate that FuseBot outperforms X-Ray’s efficiency by more than 40% in terms of the number of actions required for successful mechanical search. Furthermore, in comparison to X-Ray’s success rate of 84%, FuseBot achieves a success rate of 95% in retrieving untagged items, demonstrating for the first time that the benefits of RF perception extend beyond tagged objects in the mechanical search problem.
Funder
Massachusetts Institute of Technology
Publisher
Springer Science and Business Media LLC
Subject
Artificial Intelligence
Reference37 articles.
1. Avigal, Y., Satish, V., Tam, Z., Huang, H., Zhang, H., Danielczuk, M., Ichnowski, J., & Goldberg, K. (2021). AVPLUG: Approach vector planning for unicontact grasping amid clutter. In 2021 IEEE 17th international conference on automation science and engineering (CASE) (pp. 1140–1147). IEEE. 2. Aydemir, A., Sjöö, K., Folkesson, J., Pronobis, A., & Jensfelt, P. (2011). Search in the real world: Active visual object search based on spatial relations. In 2011 IEEE international conference on robotics and automation (pp. 2818–2824). IEEE. 3. Bajcsy, R. (1988). Active perception. Proceedings of the IEEE, 76(8), 966–1005. 4. Bohg, J., Hausman, K., Sankaran, B., Brock, O., Kragic, D., Schaal, S., & Sukhatme, G. S. (2017). Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics, 33(6), 1273–1291. 5. Boroushaki, T., Leng, J., Clester, I., Rodriguez, A., & Adib, F. (2021a). Robotic grasping of fully-occluded objects using rf perception. In 2021 international conference on robotics and automation (ICRA). IEEE.
|
|