Introduction
Virtual Worlds are undergoing a structural transformation. What initially emerged as isolated immersive environments for visualization or simulation is evolving into a new class of persistent, intelligent, and operational digital environments tightly coupled with physical systems. This transformation is driven by the convergence of Artificial Intelligence (AI), the Internet of Things (IoT), and their integrated form, Artificial Intelligence of Things (AIoT). Together, these technologies enable Virtual Worlds not only to represent reality, but to sense, reason, predict, and influence it.
Within the European research and innovation landscape, this convergence is not accidental. It reflects a deliberate shift toward human-centric, trustworthy, and sovereign digital infrastructures, aligned with Industry 5.0 principles. Initiatives such as OPENVERSE, VW4ALL, SPIRIT, XR5.0, MOTIVATE XR, CARTIF, and Smart4Fit collectively illustrate how Virtual Worlds are becoming an enabling layer for intelligent workplaces, adaptive training, collaborative robotics, and real-time decision-making.
This report develops a coherent narrative across these initiatives, examining how AI and AIoT reshape the architecture, function, and impact of Virtual Worlds. Each section builds on the previous one, progressively moving from foundational technologies to application domains, market impact, and governance implications.
1. Artificial Intelligence as the Cognitive Layer of Virtual Worlds
The transformation of Virtual Worlds begins with the introduction of AI as a cognitive layer. Early XR systems were largely deterministic, relying on predefined scripts and static simulations. The integration of AI fundamentally alters this model by enabling perception, inference, learning, and adaptation within immersive environments.
Across the examined projects, AI is applied through complementary paradigms. Perception-oriented AI enables tracking, recognition, and interpretation of human actions, machines, and environments, as demonstrated in human–robot collaboration frameworks such as CARTIFactory. Predictive and physics-informed AI, exemplified by SimZero within the SPIRIT/SITE project, combines machine learning with scientific simulations to generate real-time, high-fidelity predictions that can be visualized and acted upon in XR environments. Generative AI, prominently used in MOTIVATE XR, transforms technical documentation and expert knowledge into structured, XR-ready content, lowering barriers to content creation and reuse. Explainable and neuro-symbolic AI, central to XR5.0, addresses the need for transparency and trust by making AI reasoning accessible to human operators through immersive visual explanations.
These approaches collectively reposition AI from an external optimization tool to an embedded reasoning component of Virtual Worlds. Rather than automating decisions, AI augments human cognition by contextualizing information and supporting situational awareness.
Critical reflection. While AI substantially enhances the adaptability and usefulness of Virtual Worlds, it also introduces dependencies on data quality, model validity, and interpretability. The diversity of AI paradigms observed across projects highlights the absence of a single dominant approach, suggesting that future Virtual Worlds will rely on hybrid AI stacks. This observation motivates the need to ground AI-driven cognition in reliable data flows, which leads directly to the role of IoT and AIoT.
2. IoT and AIoT as the Physical–Digital Coupling Mechanism
2. IoT and AIoT as the Physical–Digital Coupling Mechanism
If AI provides cognition, IoT provides grounding. Virtual Worlds become operationally relevant only when they are continuously informed by the physical world. IoT infrastructures supply this connection by streaming real-time data from sensors, machines, wearables, and environments into digital representations.
The solutions developed by centres and projects analysed demonstrate that raw connectivity alone is insufficient. AIoT architectures, where intelligence is distributed across edge and cloud components, are increasingly adopted to meet latency, scalability, and robustness requirements. In Smart4Fit, wearable IMU sensors capture biomechanical signals that are locally processed and interpreted by AI agents to adapt AR-based training in real time. In SPIRIT/SITE, robotic platforms, environmental sensors, and 5G connectivity enable physics-informed XR digital twins that support remote operation under hazardous conditions. Industrial pilots in XR5.0 and MOTIVATE XR similarly rely on connected assets to synchronize physical operations with immersive guidance and analytics.
Through AIoT, Virtual Worlds evolve into closed-loop systems in which sensing, simulation, prediction, visualization, and action form a continuous cycle. This loop transforms XR from a passive interface into an active operational layer.
Critical reflection. The tight coupling enabled by AIoT increases system responsiveness and realism, but also raises challenges related to interoperability, security, and system complexity. Heterogeneous devices and platforms must operate coherently within shared Virtual Worlds. Addressing these challenges requires a shift from isolated applications to structured environments that support human work at scale, motivating the concept of the augmented workplace.
3. Virtual Worlds as Intelligent Augmented Workplaces
Building on AI-driven cognition and AIoT-enabled coupling, Virtual Worlds increasingly function as intelligent augmented workplaces. Rather than replacing human labor, these environments enhance human capabilities by embedding guidance, analysis, and collaboration directly into work contexts.
VW4ALL and the augmented workplace framework developed by CCG/ZGDV emphasize context-aware interaction as a core principle. AI continuously evaluates task progress, environmental conditions, and user state to deliver adaptive guidance that reduces cognitive load. Similarly, CARTIF demonstrates how Virtual Worlds serve as shared coordination spaces for human–robot collaboration, allowing operators to train, supervise, and intervene safely through XR interfaces. XR5.0 extends this concept by integrating explainable AI into immersive environments, ensuring that workers understand not only what actions are recommended, but why.
In this configuration, Virtual Worlds act as mediating systems between humans, machines, and AI agents. Collaboration becomes spatial, intuitive, and traceable, aligning with Industry 5.0 goals of human-centricity and resilience.
Critical reflection. While augmented workplaces improve efficiency and safety, they also redefine skill requirements and responsibility boundaries. Effective deployment depends on careful co-design with end users and transparent AI behavior. These considerations naturally extend toward the underlying representations that support such workplaces, notably digital twins.
4. Digital Twins and Physics-Based Virtual Worlds
Before addressing digital twins at scale, it is necessary to consider a concrete class of XR-enabled systems where AI-driven design, simulation, and human monitoring already converge on interactive cockpit and control-environment design. This class of applications provides a technically mature illustration of how Virtual Worlds integrate AI cognition, physical simulation, and human-centered evaluation.
Digital twins provide the structural backbone for many AI-enabled Virtual Worlds. Moving beyond static replicas, current initiatives emphasize operational and predictive twins that evolve in real time and support decision-making.
The SPIRIT/SITE project illustrates this evolution through the integration of high-fidelity CFD simulations, machine learning acceleration, and XR visualization. These physics-informed digital twins allow users to explore scenarios, assess risks, and control remote systems within immersive environments. MOTIVATE XR addresses complementary challenges by democratizing digital twin creation through AI-assisted scanning, no-code modelling, and automated knowledge extraction, enabling wider adoption across industrial contexts.
Digital twins thus become not only mirrors of physical systems, but interactive experimentation spaces embedded within Virtual Worlds.
A representative example is XR-based cockpit and control-room design, as developed within VW4ALL-related research activities. Here, Virtual Worlds enable the parametrized design of cockpit geometries—such as rectangular or elliptical layouts integrating infotainment and control elements—followed by immersive simulation under static and dynamic conditions, including parked, transitional, and operational scenarios. AI-assisted interaction allows designers to iteratively modify layouts using natural language or declarative constraints, while generative models support rapid prototyping of interface variants.
Crucially, these environments integrate human-state monitoring during simulation. Eye tracking, gaze patterns, heart rate, and other physiological or biometric signals are captured to assess cognitive load, attention distribution, and fatigue. AI models correlate these signals with spatial layouts and interaction patterns, enabling evidence-based design decisions that go beyond subjective evaluation. Techniques such as neural splatting and point-based reconstruction further enhance realism by embedding high-fidelity representations of real-world imagery directly into immersive simulations, improving perceptual continuity between physical and virtual domains.
These interactive design loops demonstrate the technical maturity of AI-enabled Virtual Worlds, but they also expose dependencies on sensing accuracy, calibration, and hardware ergonomics. While software architectures increasingly support such complexity, hardware limitations remain a constraining factor, motivating broader consideration of device evolution and accessibility in immersive systems. Furthermore, the increasing sophistication of digital twins improves predictive power and usability, but also demands standardization, validation, and lifecycle management. Without these, scalability across sectors remains limited. This observation leads to an examination of how such systems translate into tangible impact across markets.
5. Market Impact Across Key Application Domains
The preceding sections have established the technical foundations of AI-enabled Virtual Worlds, from cognition and physical coupling to interactive digital twins. These foundations translate into differentiated impacts across application domains, shaped by domain-specific constraints, risk profiles, and adoption dynamics.
The convergence of AI, AIoT, and Virtual Worlds manifests differently across market segments, reflecting domain-specific constraints and opportunities.
In industrial manufacturing and maintenance, projects such as MOTIVATE XR, XR5.0, CARTIF, and VW4ALL demonstrate reductions in training time, error rates, and safety risks through immersive, AI-assisted guidance. In training and education, Virtual Worlds enable experiential learning and personalized skill development, supporting continuous reskilling. In sports and health, Smart4Fit shows how AI-driven AR environments can deliver hyper-personalized training while minimizing injury risk, with applicability extending from elite athletes to recreational users. In infrastructure and emergency response, physics-informed XR digital twins developed in SPIRIT/SITE enhance situational awareness and enable safer remote operations.
While benefits are evident, market uptake depends on cost, interoperability, and trust. The diversity of application domains underscores the need for shared frameworks and governance mechanisms to ensure sustainable deployment.
6. Governance, Trust, and European Digital Sovereignty
As Virtual Worlds transition from experimental systems to operational infrastructures, governance becomes a central concern. XR5.0 explicitly addresses trustworthiness through explainable AI and human-in-the-loop design, while OPENVERSE provides a broader architectural and policy-oriented framework encompassing interoperability, ethics, and data governance.
These efforts reflect a strategic European objective: ensuring that Virtual Worlds remain aligned with societal values and reduce dependency on non-European platforms. Governance is therefore not an external constraint, but an enabling condition for long-term adoption.
Embedding governance into technical architectures remains challenging, particularly as systems scale and integrate generative and agentic AI. Addressing this challenge is essential for sustaining innovation without compromising trust.
7. Conclusions – Final Perspective
The convergence of AI, AIoT, and IoT is redefining Virtual Worlds as intelligent, human-centric, and operational environments. Across the initiatives examined, this transformation is shown to be neither speculative nor confined to experimental prototypes. Instead, it is already materializing through deployable systems that measurably affect industrial productivity, workforce training, health and wellbeing, infrastructure management, and safety-critical operations. Virtual Worlds are increasingly embedded in real processes, shaping how decisions are made, how skills are acquired, and how humans interact with complex socio-technical systems.
The coherence observed across projects points to an emerging architectural pattern that is both technically robust and conceptually stable. Artificial Intelligence acts as the cognitive layer, enabling perception, reasoning, prediction, and explanation. AIoT provides grounding by tightly coupling physical assets, sensors, and actuators with digital representations through low-latency, adaptive feedback loops. In this scope, the linkage with Virtual Worlds is seamless. The latter serves as the exploitation layer, closer to the end user and where the interaction and orchestration occur. Where humans, machines, and AI systems converge through immersive, spatial, and collaborative interfaces, and with the most important, trust in the technology is enabled through governance frameworks, encompassing explainability, ethics, interoperability, and data protection, that complete the picture providing a stabilizing layer that sustains, in addition, long-term viability.
This layered pattern is significant because the progress in Virtual Worlds is no longer driven by isolated technological advances, but by the alignment of multiple building blocks into coherent systems. Advances in generative AI, physics-informed simulation, human-centric XR, or edge intelligence only translate into impact when they are integrated within architectures that support interoperability, scalability, and accountability. As demonstrated by the European projects discussed, such integration is achievable when technical development is guided by explicit human-centric and societal objectives.
Looking forward, the primary challenge is not the absence of enabling technologies, but the consolidation of these architectures across domains and scales. Standardization of interfaces, shared data models, and common design principles will be required to prevent fragmentation and to allow Virtual Worlds to interoperate across sectors. Equally important is the continued integration of human-in-the-loop mechanisms, ensuring that increasing levels of autonomy and intelligence do not erode human agency, responsibility, or understanding.
Virtual Worlds should be understood not as end-user applications or isolated platforms, but as a foundational digital layer that mediates between physical reality, artificial intelligence, and human activity. When designed and governed appropriately, they provide a means to operationalize Europe’s vision of Industry 5.0: systems that are productive yet resilient, intelligent yet transparent, and technologically advanced while remaining firmly aligned with human and societal values.