Integrating computer vision and a strain sensor for human–machine interfaces with enhanced versatility and scalability

Abstract

Abstract Soft strain sensors play a major role in emerging human–machine interfaces. Most advanced soft strain sensors rely on nanotechnologies including microfabrication techniques. However, the low reproducibility of these sensors due to their highly specialized fabrication techniques, as well as their vulnerability to environmental noise and short lifetime are remaining challenges to limit their application under real-world conditions. Here, we propose a novel approach of integrating computer vision with streamlined microfabrication techniques to solve the aforementioned problems, which may be challenging to resolve using only nanotechnology. We developed a computer vision-based optical strain (CVOS) sensor system comprising an easily fabricated soft silicone substrate with micro-markers and a tiny camera as a highly sensitive marker detector. We then embedded an artificial intelligence (AI) model with an automated response correction algorithm for tracking markers and detecting the sensor state. The findings in this study confirmed that proposed CVOS sensor is a promising approach that facilitates the development of highly sensitive and versatile human–machine interfaces for long-term operation under real-world conditions.