Nexco-East Leverages Microsoft’s Metaverse
Learn how the HoloLens innovator provides leading technology firms with immersive workplace solutions
Microsoft, a trailblazer in the technology industry, has been collaborating with an illustrious roster of clients to deliver its cutting-edge immersive solutions tailored to various enterprise needs. This array of partnerships comes as Microsoft gears up for the launch of its industrial Metaverse next year.
One of Microsoft’s distinguished XR clients is Nexco-East Engineering, responsible for managing approximately 4,000 kilometers of expressways in Japan. Recently, the company introduced an immersive training application for ETC equipment and tunnel emergency facilities, harnessing the power of Mixed Reality (MR) and digital twin technologies.
Currently, the region is grappling with a pressing social issue stemming from over 30 years of public use of Japan’s 10,000 kilometers of expressways, resulting in extensive maintenance requirements for road surfaces, bridges, and tunnels, as reported by Nexco-East.
At their Technical Training Center (TTC) in Takasaki, Gunma, Nexco-East’s trainees have access to a state-of-the-art Microsoft-powered training service. The company emphasizes that this training equips every engineer with the essential knowledge and skills to execute repair and maintenance tasks effectively.
Tetsuo Hideshima, the Managing Director and General Planning Manager of Nexco-East Engineering, underlines their commitment to ensuring the safe use of expressways by providing inspections and troubleshooting services round the clock, year-round. He stresses the importance of comprehensive checks and diagnoses aided by Information and Communication Technologies (ICTs) for the continual maintenance and management of expressways.
Masayuki Namiki, the General Manager of TTC, echoes this sentiment, noting that over 1,000 engineers participate in training programs annually. These programs are designed to enhance skills through an internal skill check system, transforming engineers into experts known as ‘highway doctors.’
Solving Pain Points with XR Solutions
As expressways become increasingly intricate in their designs, Nexco-East trainees require more sophisticated and relevant immersive training scenarios that leverage 3D visualizations. To address this need, Nexco-East turned to MR to visualize mission-critical training data effectively.
Nexco-East encountered challenges during traditional hands-on learning experiences, especially when dealing with complex building materials. This necessitated a deep digital standardization of training through Microsoft’s immersive services.
Microsoft and Nexco-East’s immersive training solution places a strong emphasis on translating the practical, real-world use of ETC equipment within a safe and repeatable virtual training environment.
Yoshitaka Nakamura, Nexco-East Engineering’s Manager of Facilities Construction Work Division, elucidates that ETC equipment comprises multiple devices, requiring comprehensive training for workers. He emphasizes that training sessions must include detailed materials such as radio communication flow charts, PowerPoint slides, and instructor-led explanations, followed by practical testing with actual ETC devices.
However, Nakamura points out that understanding the associations among these training materials can be time-consuming. Radio communication and control signal flows are challenging to visualize, often relying on trainees’ imaginations, resulting in gaps in understanding.
Working with Microsoft’s Immersive Portfolio
Nexco-East harnessed Microsoft’s Mixed Reality (MR) and digital twin solutions to bridge knowledge gaps and optimize learning outcomes through its internal “MR for ETC equipment training” project. This project leveraged digital twin imagery to simulate ETC equipment alongside integrated systems such as infrared sensors, radio communication, and data communication routes.
Additionally, Nexco-East collaborated with DataMesh to leverage MR experiences, digital twins, and XR services. Kensho Kashimada, the Director of Marketing and Sales at DataMesh Japan Co., emphasizes the uniqueness of their endeavor, with no similar attempts in Japan at the time. He highlights the challenge of creating MR materials that are easily understandable for every trainee, where there was no clear blueprint for success.
Kashimada further elaborates on their meticulous development process, involving repeated feedback sessions with the project’s leading department and TTC instructors who would be using the tool in practice. He notes that traditional training methods, which rely on oral explanations and 2D materials, often fall short in helping trainees grasp complex ETC mechanisms.
To address these challenges, Nexco-East dedicated six months to refining its training procedures by integrating MR solutions with traditional teaching methods. Additionally, the company utilizes Microsoft’s HoloLens 2 headset along with tablet devices to facilitate the training process.
Leveraging Technology for Real-world Impact
Kashimada highlights the importance of SLAM (Simultaneous Localization and Mapping) accuracy in MR superimposition, usually dependent on hardware performance. However, for Nexco-East’s MR for ETC equipment training, DataMesh’s proprietary digital twin platform (FactVerse) played a crucial role in ensuring consistent SLAM accuracy across multiple devices.
Following the successful launch of this project, Nexco-East is now gearing up to introduce another MR learning initiative called “MR for tunnel emergency facilities training.”
The Power of Digital Twins in Immersive Training
The XR industry is abuzz with technological trends and buzzwords, similar to other emerging sectors. However, the advent of the industrial Metaverse and its practical applications has brought legitimacy to certain integrated technologies, with digital twins at the forefront.
Digital twins have become a widely adopted solution for numerous enterprise use cases, including Architecture, Engineering, and Construction (AEC), product design, and immersive training. While they share similarities with Computer-Aided Design (CAD) files, digital twins distinguish themselves through their ability to replicate objects with remarkable detail, including small components and faults. Furthermore, they can create 3D representations of objects in motion or in use, making them invaluable for training purposes.
For Nexco-East, collaboration with project partners enabled the recreation of equipment movements. DataMesh, for instance, created visualizations of internal structures to develop MR-based ETC equipment training scenarios tailored to real-world situations.
By harnessing immersive technology, Nexco-East can offer unique training experiences that are unattainable through conventional methods. This includes features such as adjusting water transparency in an XR digital twin animation, providing a precise simulation of water flow inside a pipeline. Additionally, the company integrates other simulation methods, such as showcasing the impact of fire on equipment, allowing trainees to gain realistic insights into complex scenarios.
The Latest Developments in Microsoft’s Industrial Metaverse
Notably, Microsoft’s HoloLens faced uncertainties in the past, with internal controversies and reorganizations causing doubts about its future. However, behind the scenes, Microsoft was diligently optimizing its XR product portfolio, primarily targeting enterprise users. This strategic move coincided with the emergence of MR headsets from competitors like Apple and Meta.
Microsoft’s XR portfolio, it seems, is squarely focused on serving the needs of enterprise users, with no evident ambitions in the consumer market. This approach positions Microsoft uniquely in the marketplace.
At the Inspire 2023 showcase event, Microsoft unveiled insights into its immersive roadmap, aligning with the company’s genAI objectives. Lili Cheng, Corporate Vice President of Business Applications and Platforms at Microsoft, shed light on Microsoft’s plans to launch its industrial Metaverse and MR product roadmap, leveraging recent AI innovations.
Cheng emphasizes the symbiotic relationship between AI and XR, enhancing outcomes across various enterprise use cases. Microsoft’s investments in generative AI-powered service CoPilot are expected to enhance MR dynamics guide services for XR headsets.
Furthermore, Cheng underscores the pivotal role played by integrated technologies such as digital twins. Real-time 3D (RT3D) visualization and data empower frontline workers with a deeper understanding of machinery and processes. AI, when integrated with XR, amplifies the capabilities of these immersive technologies.
Cheng elaborates:
“Mixed reality is the eyes and ears of AI. Delving into operation nuances for personalized, in-depth learning becomes easier. When integrated, mixed reality and AI accelerate worker training, shortening steps and supplying users with the working knowledge they need for the task at hand.”
Microsoft’s Industrial Metaverse Roadmap
Microsoft’s industrial Metaverse roadmap includes a meticulously curated learning path for end-users, providing resources to introduce workplace XR technology. This comprehensive plan encompasses a range of first-party technologies, including Microsoft Co-Pilot, Azure OpenAI, AI, ML, Azure Databricks, Synapse Analytics (2024), Microsoft Fabric (2024), Microsoft Purview, Power Platform, D365 Field Service, D365 Guides, D365 Remote Assist, HoloLens 2, Azure Maps, Azure HPC products, and Azure Digital Twins.
Microsoft launched this industrial Metaverse roadmap as part of its AI Cloud Partner Program, bringing together significant partners to drive innovation in enterprise-grade AI. The program aims to enhance the application of AI, Cloud, and Metaverse solutions to boost productivity, resilience, and sustainability across various industries.
The investments in AI are instrumental in optimizing Microsoft’s XR solutions for frontline operations. Microsoft’s services, such as Dynamics 365 Guides, create a spatial environment where frontline workers can access collaboration tools like Teams.
Much like the success story of Nexco-East, integrated AI features empower XR-ready frontline workers to complete repair procedures efficiently, overcoming traditional obstacles. This connectivity extends workers beyond their physical limits, further enhancing their capabilities.
