Visual quality and safety monitoring system for human-robot cooperation

Author:

Kozamernik Nejc,Zaletelj Janez,Košir Andrej,Suligoj Filip,Bracun Drago1ORCID

Affiliation:

1. University of Ljubljana Faculty of Mechanical engineering

Abstract

Abstract Efficient workspace awareness is critical for improved interaction in cooperative and collaborative robotics applications. In addition to safety and control aspects, quality-related tasks such as the monitoring of manual activities and the final quality assessment of the results are also required. In this context, a visual quality and safety monitoring system is developed and evaluated. The system integrates close-up observation of manual activities and posture monitoring. A compact single-camera stereo vision system and a time-of-flight depth camera are used to minimize the interference of the sensors with the operator and the workplace. Data processing is based on a deep learning to detect classes related to quality and safety aspects. The operation of the system is evaluated while monitoring a human-robot manual assembly task. The results show that the proposed system ensures a high level of safety, provides reliable visual feedback to the operator on errors in the assembly process, and inspects the finished assembly with a low critical error rate.

Publisher

Research Square Platform LLC

Reference57 articles.

1. Arash Ajoudani and Andrea Maria Zanchettin and Serena Ivaldi and Alin Albu-Sch äffer and Kazuhiro Kosuge and Oussama Khatib (2018) Progress and prospects of the human –robot collaboration. Autonomous Robots 42 https://doi.org/10.1007/s10514-017-9677-2, 5, 15737527, Recent technological advances in hardware design of the robotic platforms enabled the implementation of various control modalities for improved interactions with humans and unstructured environments. An important application area for the integration of robots with such advanced interaction capabilities is human –robot collaboration. This aspect represents high socio-economic impacts and maintains the sense of purpose of the involved people, as the robots do not completely replace the humans from the work process. The research community ’s recent surge of interest in this area has been devoted to the implementation of various methodologies to achieve intuitive and seamless human –robot-environment interactions by incorporating the collaborative partners ’ superior capabilities, e.g. human ’s cognitive and robot ’s physical power generation capacity. In fact, the main purpose of this paper is to review the state-of-the-art on intermediate human –robot interfaces (bi-directional), robot control modalities, system stability, benchmarking and relevant use cases, and to extend views on the required future developments in the realm of human –robot collaboration.

2. S. V. Amarasinghe and H. S. Hewawasam and W. B.D.K. Fernando and J. V. Wijayakulasooriya and G. M.R.I. Godaliyadda and M. P.B. Ekanayake (2015) Vision based obstacle detection and map generation for reconnaissance. Institute of Electrical and Electronics Engineers Inc., 2, Aerial map generation,Depth calculation,Obstacle detection,Reconnaissance,Stereo vision, 9th International Conference on Industrial and Information Systems, ICIIS 2014, 9781479964994, 10.1109/ICIINFS.2014.7036570, Obstacle detection and map generation is an essential tool for site reconnaissance applications. Further it enables optimal and efficient path planning for mobile agents to navigate in unknown environments. This paper proposes a solution to this problem through a stereo vision-based obstacle detection and depth measurement method for reconnaissance. The proposed approach employs a boundary tracing algorithm to identify the objects in the close vicinity to the vision system. Next, it generates a depth map that contains the obstacles and the distance to each obstacle with respect to the position of the mobile agent. An error model was developed to further improve the accuracy of the depth estimation. Depth maps from multiple angles for a given set of identified obstacles are superimposed in the same coordinate system to generate the aerial map. For objects with sharp edges, the aerial map was generated based on corners identified using a corner detection algorithm. A prototype of the system was implemented and tested in a controlled environment. The results show that percentage estimation error was significantly reduced after the depth calculation was refined using the proposed error model.

3. Simone Antonelli and Danilo Avola and Luigi Cinque and Donato Crisostomi and Gian Luca Foresti and Fabio Galasso and Marco Raoul Marini and Alessio Mecca and Daniele Pannone (2022) Few-Shot Object Detection: A Survey. ACM Computing Surveys https://doi.org/10.1145/3519022, 0360-0300, Deep Learning approaches have recently raised the bar in many fields, from Natural Language Processing to Computer Vision, by leveraging large amounts of data. However, they could fail when the retrieved information is not enough to fit the vast number of parameters, frequently resulting in overfitting and, therefore, in poor generalizability. Few-Shot Learning aims at designing models which can effectively operate in a scarce data regime, yielding learning strategies that only need few supervised examples to be trained. These procedures are of both practical and theoretical importance, as they are crucial for many real-life scenarios in which data is either costly or even impossible to retrieve. Moreover, they bridge the distance between current data-hungry models and human-like generalization capability. Computer Vision offers various tasks which can be few-shot inherent, such as person re-identification. This survey, which to the best of our knowledge is the first tackling this problem, is focused on Few-Shot Object Detection, which has received far less attention compared to Few-Shot Classification due to the intrinsic challenge level. In this regard, this review presents an extensive description of the approaches that have been tested in the current literature, discussing their pros and cons, and classifying them according to a rigorous taxonomy.

4. Janis Arents and Valters Abolins and Janis Judvaitis and Oskars Vismanis and Aly Oraby and Kaspars Ozols (2021) Human –robot collaboration trends and safety aspects: A systematic review. Journal of Sensor and Actuator Networks 10 https://doi.org/10.3390/jsan10030048, 3, 22242708, Smart manufacturing and smart factories depend on automation and robotics, whereas human –robot collaboration (HRC) contributes to increasing the effectiveness and productivity of today ’s and future factories. Industrial robots especially in HRC settings can be hazardous if safety is not addressed properly. In this review, we look at the collaboration levels of HRC and what safety actions have been used to address safety. One hundred and ninety-three articles were identified from which, after screening and eligibility stages, 46 articles were used for the extraction stage. Predefined parameters such as: devices, algorithms, collaboration level, safety action, and standards used for HRC were extracted. Despite close human and robot collaboration, 25% of all reviewed studies did not use any safety actions, and more than 50% did not use any standard to address safety issues. This review shows HRC trends and what kind of functionalities are lacking in today ’s HRC systems. HRC systems can be a tremendously complex process; therefore, proper safety mechanisms must be addressed at an early stage of development.

5. Yang Bai and Qujiang Lei and Hongda Zhang and Yang Yang and Yue He and Zhongxing Duan (2019) An Investigation of Security Approaches in Industrial Robots. 2019 5th International Conference on Control, Automation and Robotics, ICCAR 2019 : 103-110 https://doi.org/10.1109/ICCAR.2019.8813393, Institute of Electrical and Electronics Engineers Inc., 4, collision detection,human-robot collaboration,industrial robot,obstacle avoidance,security approaches, 9781728133263, Human-robot collaboration is an important direction for the development of the robot industry. In order to ensure that robot can share their workspace with human, the safety mechanism of the robot needs to be considered. Not only to ensure that the robot will not harm the operator, but also to protect the robot itself. We review several related works in the field of human-robot coexistence security methods, including obstacle avoidance and collision detection. Then concerning works of implementation of security approaches in industrial scenario are discussed in detail. Finally, we summarize the problems that have not yet been solved, and hope to inspire new ideals in research of human-robot coexistence security approaches in the future.

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3