What is haptic feedback? It is the sensation of touch, one of our five senses. We experience it when using smartphones today. When you put your alert on vibrate, you can feel that someone is messaging you. And many smartphone applications provide haptic feedback as a feature.
But going from haptic feedback in a two-dimensional object to one that is three-dimensional and virtual is a “dimensional” leap. You can achieve this in several ways. One is aerohaptics, an air-based haptic feedback system to create a sense of touch without wearing any enabling apparatus such as a headset. Another is to use focused ultrasound to achieve a similar result.
The leap motion controller used by Professor Dahiya comes from Ultraleap. Ultraleap has a competing haptic feedback solution that combines small ultrasound speakers emitting precisely directed waves sequenced by computer algorithms to arrive at the same point in space and time to create tactile effects that a hand can detect in mid-air. In other words, virtual touch.
Aerohaptics does it differently by involving the pairing of 3D-display technology with directed airflow to create virtual touch. The University of Glasgow’s Bendable Electronics and Sensing Technologies (BEST) research group came up with the idea.
Their demonstration is a holographic bouncing basketball hovering in space that anyone can interact with. The aerohaptic feedback lets participants feel the virtual ball as it rolls off their fingertips. They can bounce it, slap it against their palms, and apply varying force and direction to where and how it goes. An illustration of the developed system can be seen at the top of this article.
Using airflow and varying its intensity is a novel approach to creating a virtual sense of touch. As you attempt to bounce the basketball using harder and faster movements the haptic feedback increases. More subtle movements cause the airflow to diminish and the ball to move appropriately.
So which is better? Ultrasound haptics or aerohaptics?
The advantage goes to the latter over the former for two reasons.
Professor Dahiya sees the technology as a precursor to the Trekkian holodeck. He notes that aerohaptics creates “a convincing sensation of physical interaction on users’ hands at a relatively low cost….adding temperature control to their airflow to deepen the sensation of interacting with hot or cool objects” would be the next step and is on the drawing board. But he doesn’t have a solution yet for his holodeck virtual objects displaying sentience and following plot lines.
The future uses he does see “could form the basis for many new applications…such as creating convincing, interactive 3D renderings of real people for teleconferences,” or “it could help teach surgeons to perform tricky procedures in virtual spaces during their training or even allow them to command robots to do the surgeries for real.”
]]>The communicator was a wireless device that bears a strong resemblance to the first flip phones.
In Star Trek the ship’s computer and crew interacted not just through keyboards but also by voice. Today we can converse with the digital world through Alexa, Google Home and Siri.
There were replicators that not only manufactured food, hot or cold, and the containers and cutlery with which to eat. We are seeing in the latest 3D printing technologies a future when the replicator will become a reality.
Some NASA scientists even believe that warp drive, a technology that alters a contained area of space around a starship to move it at speeds beyond light may be attainable.
And then there is the holodeck, a technology that first appeared in Star Trek’s Next Generation, in which crew members could immerse themselves and interact within a virtual reality without wearing goggles, or putting on a special sensor-laden suit.
How close are we to this type of immersive holodeck experience? In postings to this site in the past, I’ve written about Disney’s Holobench, and the University of Tokyo’s Haptoclone, but there is much more out there moving closer to turning the fiction of Star Trek into a reality.
Judge for yourself.
SandboxVR, a Hong Kong-based virtual reality gaming company, describes its technology as “bleeding edge” uncompromising immersion, as close to the holodeck as the world can get today. They use full-body motion capture technology in which you and others can experience VR together.
Steve Zhao, the founder, created his first VR game back in 2016. But in 2017 he decided to go for broke in building a VR interface as close to the holodeck experience as possible. SandboxVR opened its doors to the public that year but only a trickle of visitors came to experience immersive VR, that is, until a Facebook video about one visitor’s experience went viral. Zhao, in a piece he wrote for Medium, describes SandboxVR’s mission, “to create a reality where you can be what you want to be and go wherever you want to go.” Today a visitor to a SandboxVR can immerse themselves in three different virtual worlds: Curse of Davy Jones, where you can hunt for lost treasure, Amber Sky 2088, where you can experience a space elevator and deliver a package to Earth to save it from invaders, and Deadwood Mansion, where you get to fight zombies. Current locations include Hong Kong, Jakarta, Macau, San Francisco, Singapore, and Vancouver, with Austin, Chicago, Dallas, Los Angeles, New York, and San Diego, all coming soon.
In pursuit of the Star Trek holodeck, Microsoft is developing what it calls a mixed reality capture studio with locations in San Francisco, London, and Redmond, Washington, that not only have technology in place to create life-like holograms but even can get the VR representation to speak in other languages simulating your voice. To do this Microsoft employs its HoloLens, mixed reality, and neural text-to-speech. The company has recently demonstrated this capability which you can watch here.
And then there is Mixed Place, an Israeli company, that is developing mixed reality AR cloud infrastructure to stream location-specific experiences around the world. According to the company’s founder and CEO, Alon Melchner, the environment it is creating is midway between immersive VR and augmented reality (AR), superimposing 3D objects on the real world through a mobile phone, camera, VR/AR headsets, and VR contact lenses. Melchner’s invention allows virtual objects to blend into the real world and interact with it seamlessly.
In a recent article, he talks about what this could mean for retail: “We can create a virtual salesperson that knows when you’re getting closer to the store and can guide you in. Through your phone or smart glasses, you’ll see shoes personalized for you on the empty table. You can rotate them, change the colors and then make your order.”
Mixed Place technology doesn’t give you the ability to feel objects like the holodeck experience, but it creatives a powerful immersive blend of VR and AR. Imagine touring Pompeii and seeing what the city actually looked like before Vesuvius exploded. Or experiencing a game in a real physical location, not a studio, where you interact with virtual objects. As Melchner states, Mixed Place is “where content and the real environment become one.”
]]>
A product of the Shinoda-Makino Lab at University of Tokyo’s Graduate School of Frontier Sciences, HaptoClone creates telehaptic experiences using ultrasound energy which produces a virtual sensation of touch when interacting with cloned objects.
What is telehaptic? It is a technology that allows for physical interaction between two separated things or people through a feedback system mimicking the real sensation of touch.
The first telehaptic projects were attempted in the mid-1980s. One of them was called the Transatlantic Telephonic Arm Wrestling system. It connected two people by modem and used two electro-mechanical wrestling arms to allow a participant in Canada to virtually arm wrestle with an individual located in Paris. Since then telehaptic systems have allowed for everything from virtual handshakes to virtual kisses. There is even a virtual Hug Shirt.
So what makes what the Shinoda-Makino Lab’s technology different?
What the researchers here have done is produce the next step in virtual reality. It goes beyond allowing two users to interact. Now a user can interact with a cloned virtual object.
How does it work?
The system features two work spaces set apart from each other. The two are virtually superimposed optically and haptically. The light field and force field of each is cloned and forwarded to the other so when a user in one work space sees a clone image of an object existing in the other work space they can touch and manipulate what they see. The system uses concentrated ultrasound to mimic the sensation of touch exerting pressure on the virtual cloned object. This represents the first telehaptic technology allowing for virtual manipulation of cloned objects and it truly is worth watching.
]]>