In this guest blog, Greg Demchak, Senior Director, Digital Innovation Lab, Bentley Systems, covers the increased adoption of digital twins for day-to-day use, and how the virtual environment can also reflect the actual conditions in the field. Which can be made available in interactive 3D experiences.

For construction professionals, architects, and engineers, it has never been more challenging to design, build, and maintain infrastructure.

Organisations face new global regulations, standards, and sustainability requirements, all while managing worker shortages and supply chain issues. In this stressful project delivery environment, where time is short and mistakes are costly, teams leaning heavily on virtual collaboration using 3D digital twins and real-time data benefit from working together to identify and solve problems, as well as make better decisions.

Advancements in gaming technology, along with platforms such as Bentley’s iTwin Platform, have accelerated how to bring all this information together, aligning 3D models to get information to the right person at the right time. These advances can help organisations build better infrastructure in a virtual environment first—to prototype solutions before it gets built.

And now, as we increasingly see the adoption of digital twins for day-to-day use, the virtual environment can also reflect the actual conditions in the field and make this available in interactive 3D experiences.

Envisioning Opportunity

Matthew Ball, an expert on the metaverse and all things artificial reality (AR) and virtual reality (VR), envisions the metaverse as, “a massively scaled and interoperable network of real-time rendered 3D virtual worlds and environments, which can be experienced synchronously and persistently by an effectively unlimited number of users with an individual sense of presence and with continuity of data, such as identity, history, entitlements, objects, and payment communications.”

At Bentley Systems, when I think about how construction teams, designers, operators, and engineers can leverage the metaverse, I see continuity of data as the key to success in all aspects of collaboration.

Continuity is Key

While the ability to render 3D models as virtual worlds in real time has been a gamechanger, we can now anticipate multiple people experiencing those 3D models together. To do this, we need a continuously updated version of the truth using 3D models and multiple forms of reality capture—from point clouds to CCTV camera feeds.

This experience of continuity of data is something that we are developing, and when you render that into the metaverse, it becomes interesting. The metaverse for the construction industry is not just about one identity, one avatar, one game running in a virtual space, but rather about a continuity of digital information that moves dynamically through a design-construct-operations lifecycle.

We can now take construction models, engineering models, design models, and reality capture and use our new technology to stream these models directly into game engines, such as Unreal Engine. This is a live connection to the latest models—not a file export-import process.

This distinction is important because a game engine becomes the base platform that enables tackling the challenging requirements for the metaverse laid out by Ball.

For construction, the metaverse can be used to explore different moments at different stages. You could conduct a complete walkthrough of a job looking for safety hazards and risks and do that from the safety of a VR headset or desktop application with your team.

The construction metaverse will be accessed from a wide range of devices depending on the use-case. Image courtesy of Bentley Systems.

Multiple Methods for “Multiplayer” Collaboration

As the metaverse will inherently enable multiplayer immersive experiences, it also enables multi device inputs. Therefore, from the context of Unreal Engine, we can deploy any experience, whether it is for VR, mixed reality, a desktop experience, or even a tablet.

With a game engine platform, you can get immersed in a 3D model that will stream the whole VR experience. You can have an avatar to navigate in the model using a game controller or VR headset to conduct a design review, for example. Then, you and your colleagues can discuss an object in that design. The game-like navigation methods are an intuitive way to move through these 3D models.

What is unique about the VR experience is the multimodal or multi-sensory aspect. Using digital twin technology and real-time data, this experience gives you a one-to-one sense of the actual scale of a 3D model. In a virtual training simulation, this can lead to improvements in health and safety because you are getting trained on actual look and feel, so when you go out into the field, you are not surprised. It is intrinsically valuable for training.

One of the latest projects at Bentley’s Innovation Lab is a VR training simulation for a high-risk area. These real-life scenarios involve workers going out in a boat to an offshore wind turbine where they deploy a drone to do the remote inspection.

We can replicate the entire environment, including learning how to fly the drone, and then deploy that drone in the field. Applying metaverse concepts, this application can include multiplayer, with multiple people immersed together in the same experience, and each person could be participating from a different part of the world. The simulation could even have an instructor there helping to guide and teach operators how to use the drone. Digital rehearsals also will become standard in the metaverse. Right now, we can take a two-week look ahead of the construction schedule—a 4D look ahead—using solutions like SYNCHRO. With multi-player, game engine enabled capabilities, the whole team can look at the same 4D content at the same time, synchronously and seamlessly.

AI and the Future of the Infrastructure Metaverse

There has been much talk about artificial intelligence (AI) in recent weeks. How does AI play a role here in construction?

According to a report by Reports and Data, and referenced in Global Banking & Finance Review, global AI in the construction market is forecast to reach USD 4.51 billion. Using available data, AI can assist in risk mitigation, safety, and predicting time and cost consumption requirements. As we get comfortable using text prompts with ChatGPT to request and receive information, we are a step away from that interaction being done with natural spoken language.

Imagine being in these 3D environments and directing questions to a 3D avatar of ChatGPT.  Maybe you ask, “Where am I most likely to fall behind based on the current progress and planned schedule?” And it could possibly respond, “Based on available information, you should consider replanning the ductwork delivery due to manufacturer delays reported in the supply chain.” Then it may ask, “Should we issue a change order?” I could also ask, “Will the weather conditions be favourable in two weeks for that delivery?” And it might say, “No, you might want to wait an extra week to avoid a potential problem out in the field.”

The idea is that the assistant becomes an agent and can be rendered as such. This is one aspect of the metaverse that experts like Ball should be adding to the collaboration equation.