Generation of realistic synthetic cable images to train Deep Learning segmentation models

Author:

Fresnillo Pablo Malvido1,Mohammed Wael M.1,Vasudevan Saigopal1,Garcia Jose A. Perez2,Lastra Jose L. Martinez1

Affiliation:

1. Tampere University

2. Universidade de Vigo

Abstract

Abstract One of the main contemporary challenges in robotics is the automation of tasks that involve handling Deformable Linear Objects (DLOs), such as cables or ropes. Due to their changing shape, constant perception is required, which makes computer vision, and in particular, image segmentation, fundamental. Even though image segmentation is a very studied problem, which has been addressed with high accuracy by many different Deep Learning models, they need to be properly trained to segment the objects of interest (DLOs in this case). To do this, it is necessary to have a large and diverse dataset. However, such a dataset hasn’t been created for cable images, and doing it manually would be complicated and extremely time-consuming. This paper addresses this issue with a novel methodology, which automatically generates synthetic cable datasets to train image segmentation models. This methodology utilizes Blender to create photo-realistic scenes and a Python pipeline to interact with them. To ensure the diversity of the dataset, before the generation of each image, the pipeline performs random variations on the elements of the scene. After this, the scene is animated, dropping the cables from a certain height and letting them deform naturally after landing, which results in realistic arrangements of the cables. The effectiveness of the methodology was demonstrated by training six popular segmentation models with synthetic datasets and using them to segment real cable images with great results (IoU over 70% and Dice coefficient over 80% for all the models).

Publisher

Research Square Platform LLC

Reference55 articles.

1. Coleman, G.B. and Andrews, H.C. (1979) Image segmentation by clustering. Proceedings of the IEEE 67(5): 773--785 https://doi.org/10.1109/PROC.1979.11327, IEEE Xplore Abstract Record:C\:\\Users\\rmpama\\Zotero\\storage\\XZVP4LYG\\1455596.html:text/html;IEEE Xplore Full Text PDF:C\:\\Users\\rmpama\\Zotero\\storage\\K2ZIHG9C\\Coleman and Andrews - 1979 - Image segmentation by clustering.pdf:application/pdf, Conference Name: Proceedings of the IEEE, May, 2023-12-26, This paper describes a procedure for segmenting imagery using digital methods and is based on a mathematical-pattern recognition model. The technique does not require training prototypes but operates in an "unsupervised" mode. The features most useful for the given image to be segmented are retained by the algorithm without human interaction, by rejecting those attributes which do not contribute to homogeneous clustering in N-dimensional vector space. The basic procedure is a K-means clustering algorithm which converges to a local minimum in the average squared intercluster distance for a specified number of clusters. The algorithm iterates on the number of clusters, evaluating the clustering based on a parameter of clustering quality. The parameter proposed is a product of between and within cluster scatter measures, which achieves a maximum value that is postulated to represent an intrinsic number of clusters in the data. At this value, feature rejection is implemented via a Bhattacharyya measure to make the image segments more homogeneous (thereby removing "noisy" features); and reclustering is performed. The resulting parameter of clustering fidelity is maximized with segmented imagery resulting in psychovisually pleasing and culturally logical image segments., 1558-2256

2. Degraded {Image} {Semantic} {Segmentation} {With} {Dense}-{Gram} {Networks} {\textbar} {IEEE} {Journals} & {Magazine} {\textbar} {IEEE} {Xplore}. Degraded Image Semantic Segmentation With Dense-Gram Networks | IEEE Journals & Magazine | IEEE Xplore:C\:\\Users\\rmpama\\Zotero\\storage\\VTKI96EK\\8812903.html:text/html, 2023-12-26

3. Yarram, Sudhir and Yuan, Junsong and Yang, Ming (2022) {Adversarial structured prediction for domain-adaptive semantic segmentation}. Machine Vision and Applications 33(5): 1--13 https://doi.org/10.1007/s00138-022-01308-8, Springer Berlin Heidelberg, 1432-1769, September

4. Kirillov, Alexander and He, Kaiming and Girshick, Ross and Rother, Carsten and Dollar, Piotr (2019) Panoptic {Segmentation}. Full Text PDF:C\:\\Users\\rmpama\\Zotero\\storage\\G7KFNWLZ\\Kirillov et al. - 2019 - Panoptic Segmentation.pdf:application/pdf, 9404--9413, 2023-12-26

5. Yi, Jingru and Wu, Pengxiang and Jiang, Menglin and Huang, Qiaoying and Hoeppner, Daniel J. and Metaxas, Dimitris N. (2019) Attentive neural cell instance segmentation. Medical Image Analysis 55: 228--240 https://doi.org/10.1016/j.media.2019.05.004, ScienceDirect Snapshot:C\:\\Users\\rmpama\\Zotero\\storage\\7ZM3ZDBB\\S1361841518308442.html:text/html, Cell detection, Cell segmentation, Instance segmentation, Neural cell, July, 2023-12-26, Neural cell instance segmentation, which aims at joint detection and segmentation of every neural cell in a microscopic image, is essential to many neuroscience applications. The challenge of this task involves cell adhesion, cell distortion, unclear cell contours, low-contrast cell protrusion structures, and background impurities. Consequently, current instance segmentation methods generally fall short of precision. In this paper, we propose an attentive instance segmentation method that accurately predicts the bounding box of each cell as well as its segmentation mask simultaneously. In particular, our method builds on a joint network that combines a single shot multi-box detector (SSD) and a U-net. Furthermore, we employ the attention mechanism in both detection and segmentation modules to focus the model on the useful features. The proposed method is validated on a dataset of neural cell microscopic images. Experimental results demonstrate that our approach can accurately detect and segment neural cell instances at a fast speed, comparing favorably with the state-of-the-art methods. Our code is released on GitHub. The link is https://github.com/yijingru/ANCIS-Pytorch., 1361-8415

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3