Dijam is co-founder and COO of GridRaster, a company well-versed in developing augmented, virtual and mixed realities for industrial and commercial applications. GridRaster provides cloud-based augmented and virtual reality (AR/VR) platforms and experiences for mobile devices and computing environments.
The world of digital twinning is no longer in its infancy and immersive technologies that use AR/VR headsets are making aerospace, defence, transportation and manufacturing companies integrate mixed realities into their planning, product development and production processes.
In this posting, Dijam introduces Apple’s new Vision Pro headset and how it will help manufacturers and engineers leverage the Metaverse. The Metaverse is considered by many to be the next evolution of the Internet. So please enjoy and feel free to comment.
Manufacturing is embracing new technologies. While the Internet and mobile technology have played a major role in this evolution, Apple’s release of the Vision Pro headset combined with the Metaverse represents a promising and elevating experience for manufacturers.
Equipped with two high‑resolution main cameras, six world‑facing tracking cameras, four eye‑tracking cameras and a TrueDepth camera, plus multiple sensors, a LiDAR scanner and more, the Vision Pro provides innovative features to distinguish it from other AR/VR headsets and gives manufacturers an effective immersive tool.
The Vision Pro uses finger, hand gestures and voice commands. It is equipped with two displays, one for each eye with a combined total of 23 million pixels. The visual experience is nothing short of extraordinary. With a custom 3D lens, the user interface remains consistently visible, complemented by features such as high dynamic range (HDR) depth, colour and contrast to enhance image quality.
Does that make Vision Pro just a better virtual gaming tool, or has it been designed to revolutionize manufacturing?
Manufacturers today can leverage the Metaverse and build digital twins to create greater efficiencies in their operations. Vision Pro can make this even better.
The Metaverse is evolving. The virtual shared digital universe where users interact with each other in a computer-generated environment has taken off and is in use to create virtual retail spaces, warehouses, and manufacturing facilities.
Digital twins inhabit the Metaverse. They are digital representations or replicas of real-world entities or systems. Digital twins include physical objects, processes, and people. Using data from sensors, Internet-of-Things (IoT) devices, and other sources, digital twins simulate reality
Vision Pro features are a perfect companion for the Metaverse. The spatial computing capability of the technology lets workers overlay digital twin information, such as instructions, diagrams, or 3D models, onto reality. This makes for more efficient operational processes providing visual cues, step-by-step guidance, and real-time feedback. Maintenance tasks are streamlined. Technicians access digital manuals, diagnostics, and remote expert support hands-free. Overall productivity improves and downtime is reduced.
Digital twins provide real-time visibility into inventory, production, and distribution. They identify bottlenecks, optimize logistics, reduce new operational startup time, enhance virtual prototyping, and accelerate product testing and design. The result is faster time-to-market at reduced cost.
In the retail space, the Metaverse using Vision Pro becomes a world of virtual stores and showrooms enhancing the customer’s experience, allowing for exploration and interaction with products in the virtual space and driving online sales.
Manufacturers can use Vision Pro in the Metaverse for employee training simulations especially where complex machinery is involved. This can improve employee skills and enhance safety. It also enables opportunities to create virtual workspaces across a wide geography, promoting collaboration and communication among even remote team members. Manufacturers can also leverage digital twins when physical presence is challenging.
One of the key requirements for mixed reality applications in the Metaverse is precisely overlaying an object in the physical world. This provides a visual presentation and work instructions for assembly and training. In the case of manufacturing overlaying objects with a digital twin can catch any errors or defects. Users can track objects and adjust renderings as work progresses.
Most on-device object tracking systems use 2D images or marker-based tracking. This severely limits overlay accuracy in 3D because 2D tracking cannot estimate depth with high accuracy, and consequently the scale, and the pose. This means even though users can get what looks like a good match when looking from one angle or position, the overlay loses alignment as the user moves around in 6DOF.
Not familiar with the term 6DOF. Machines or motion systems on the drawing board are designed to move a certain way but may move differently in the real world. These six types of motion are linear, horizontal, vertical, pitch, yaw and roll. Only 3D digital simulation captures all of them. When you add deep learning-based 3D artificial intelligence (AI), users can identify objects of arbitrary shape and size in various orientations in the virtual space and compare them with real-world counterparts.
Technologies like AR/VR have been in use for several years. Many companies have deployed virtual solutions where all the data is stored locally. This severely limits performance and scale for today’s virtual designs. It also limits the ability to share knowledge between organizations which can prove critical in designing new products and understanding the best way for virtual buildouts.
To overcome these limitations companies are turning to cloud-based (or remote server-based) AR/VR platforms powered by distributed architecture and 3D vision-based AI. These cloud platforms provide the desired performance and scalability to drive innovation in the industry at speed and scale.
Vision Pro and its successors are among the technologies shaping the Metaverse in the present and future. New extended reality devices will continue to emerge such as brain-computer interfaces. The cloud will continue to grow. We are still at the very beginning of the age of computing barely three-quarters of a century old.
The human digital experience, therefore, remains at an early stage as we enter an alternate to the physical Universe, the Metaverse of our creation. So prepare to have your mind blown.
]]>
Before delving into this topic it is important to differentiate between a digital twin and a simulation. A digital twin is s digital model of something real or intended to exist in the physical real world. A digital twin is indistinguishable from what exists in reality. A digital twin serves as a stable duplication of something real that can be tested. A digital twin can provide feedback to its real-world equivalent.
What a digital twin is not is a simulation. The latter is limited to the imagination of the designer. A change occurring in a digital twin, if tried on a real-world system or object, produces the same results. Simulations can have multiple results and follow many different paths.
I am more familiar with simulations and simulators. I remember years ago playing with Microsoft Flight Simulator. I could learn to fly and remember playing pilot. I was okay with takeoffs, but not so much with landings.
Then, a few years ago while on a road trip that took us through Ohio, my wife and I, along with our daughter and son-in-law, dropped in on the hometown of Neil Armstrong and the Armstong Air & Space Museum, in Wapakoneta. We had passed the museum when on previous road trips but this time decided to visit. That led to my second experience using a simulator. I tried to land the Lunar Excursion Module (LEM) on the Moon. My son-in-law mastered landing the LEM after a few tries. I crashed and crashed, and crashed. Thankfully, what I was playing with was not a digital twin with a feedback loop.
The promise of digital twin technologies is increasingly being seen as critical to national policy in the United States and elsewhere. Why? Because it is seen as a means to accelerate scientific discovery and revolutionize health care, manufacturing, supply chain, and even our understanding of the interactions of the systems that govern our planet.
The American National Academy of Science (NAS) recently produced a pre-publication online edition of a consensus study and report on digital twins. The highlighted text provides you with a link.
One of its authors is Karen Willcox, Professor of Aerospace Engineering and Engineering Mechanics at the University of Texas-Austin. She describes the great promise that digital twins can bring to science and technology noting that there are serious research questions to tackle by using the mimicry, structure, context and behaviour of a physical system through a digital representation.
She notes that a digital twin can do what a simulation cannot. A digital twin can produce a bidirectional interaction between the virtual and physical through a feedback loop. This allows the digital twin to learn from its physical real-world counterpart and update itself. A digital twin can have predictive capability and inform decision-making for the use of the system or object to which it is twinned. These interactions could be fully automated, or feed recommendations to the system or object’s human operators to make adjustments as learned in the feedback.
The growing complexity of our world and the technologies we rely on are making digital twinning vital. Digital twins can help solve seemingly intractable problems and challenges. Introducing digital twins at the outset of projects from design to deployment can lead to “improved decision-making in biomedical settings, enhanced capabilities for making weather forecasts and simulating climate variability and change, and more efficient operations and production strategies,” states Willcox.
This type of digital twin represents parts of a system or product. The automotive parts example comes to mind. A digital twin can be something as small as a screw or as big as an engine. Component twins can be used to model parts that during use may be subject to stresses or heat. Designers can see how well these parts perform and make improvements.
These twins are usually of an entire product rather than a component or part. They may be composed of component twins to test how an entire product operates in its digital version subject to real-world conditions. A nuclear power plant is a good digital twin-use example. A single wind turbine assembly undergoing digital performance tests is another.
This type of digital twin is exactly what it says, a digital model of an entire system looking at the performance of all the various components within it. System twins contain assets and components and look at how all of these relate and perform. A good example of use would be the operation of a power plant and supporting an electrical grid.
These types of digital twins represent multiple systems working together. Whereas a system twin may look at the line of production in a factory, a process twin would model the entire facility including the human resources operating the machinery on the factory floor.
Where would digital twins help us? A digital twin can be fit for purpose to meet something very small or something much larger. It can be used to analyze a wide range of data inputs, use advanced virtual models that have greater fidelity to their physical counterparts, and harvest and analyze vast amounts of computational information and data.
Here are a few examples:
There are considerations to keep in mind when developing or using digital twins. In building a digital twin the aggregated information could be deemed sensitive. Think about using digital twins in healthcare, for example, where patient privacy is a concern.
The NAS report identifies the security risk entailed in the large-scale deployment of digital twins for the defence industry and the manufacturing of key technologies. Think about the integrated workings of an aircraft or automobile as just one example where a digital twin with a feedback loop could be hacked leading to real-world disasters. Imagine, if the digital twin of an autonomous vehicle system were to communicate with its real-world equivalent after being compromised by a cybercrook.
The major players developing digital twin technology include Microsoft Azure, IBM, Amazon Web Services, General Electric, Siemens, ANSYS, Dassault Systèmes, PTC, Robert Bosch and 15 other companies. In 2022, revenue from all 25 generated US $10.1 billion. By 2028, that number is expected to grow tenfold to more than $110 billion.