This wristband makes you a robot puppeteer
Popular Science...
Robots are already better than humans at quite a few tasks. Try to outplay an AI-enabled system at chess or outlast a robot worker operating in rooms filled with radiation and you’d likely fall short. But even with all that advancement, machines still struggle to complete many seemingly basic tasks, especially those involving delicate hand movements. Something as simple as peeling a banana without squishing it is still considered a challenge for robotic systems, in large part because researchers haven’t figured out a way of accurately capturing the complexity of our own human hands. A new wrist wearable may change that.
This week, a team of engineers from the Massachusetts Institute of Technology (MIT) showcased a wristband that uses ultrasound imaging to constantly monitor the interior of a person’s wrist. It takes ultrasound images that produce a constant series of images showing how a wearer’s wrist muscles, tendons, and ligaments interact with each other to produce hand gestures. That stream of images are then linked with an AI algorithm that can interpret the images in real time and transmit them to a robotic hand.
Using that system, the robotic hand mirrors the wristband wearer’s minute gestures in real time. Volunteers wearing the device could direct the robotic hand to grab tennis balls, make hand signs, and even play notes on a piano. The engineers behind the wristband believe it may be the most advanced tool yet for training robots to use their hands more like humans. That same technique can also be applied to digital environments, which means future wearers could control a phone screen without ever touching it, or interact with virtual reality in ways that feel more immersive. The team published their findings in the journal Nature Electronics.
“We think this work has immediate impact in potentially replacing hand tracking techniques with wearable ultrasound bands in virtual and augmented reality,” MIT mechanical engineering professor and study co-author Xuanhe Zhao said in a statement. “It could also provide huge amounts of training data for dexterous humanoid robots.”
Fingers are like a puppet on a string
The wearable uses a smartwatch-sized “ultrasound sticker” placed on a person’s wrist, which essentially takes a peek beneath the surface. By imaging the micro-movements of the muscles and tendons below, the engineers could infer how those movements correspond to specific finger positions. In their words, the tendons and muscles are like strings pulling on a puppet. In this case, the puppet is the wearer’s fingers and thumb. Every image taken of those “strings” corresponds to a particular hand state. Using that approach, the team was able to identify muscle and tendon movements that correlate to all 22 degrees of freedom the human hand is capable of.
But matching each of those images to a hand position manually simply isn’t feasible. To solve that, the team developed an AI algorithm trained on ultrasound images carefully labeled by humans. That AI can rapidly analyze incoming images and determine what hand position they represent. Using that method, engineers can take the gestures decoded by the AI and apply them to robots and digital environments. The system proved precise enough to accurately differentiate all 26 letters in American Sign Language. It could also interpret the pinching motion a wearer made and translate that gesture into a zoom command on a screen.
“We think these wearable ultrasound bands can provide intuitive and versatile controls for virtual reality and robotic hands,” Zhao added.
Looking ahead, the team wants to reduce the size of the device. Though they call it a wrist wearable, the accompanying electronics currently make it look more like a cyberpunk gauntlet than an Apple Watch. They also plan to train their AI on even more gestures from a wider variety of volunteers with differing hand sizes and shapes. Eventually, they hope to finalize a wearable that just about anyone can use to remotely control robots.
The post This wristband makes you a robot puppeteer appeared first on Popular Science.