In XR News
What’s the story?
NVIDIA XR AI is now available in public beta, giving developers a library for building spatially aware multimodal AI agents for AR glasses and XR devices.
Why it matters
NVIDIA XR AI gives developers a way to connect real-world device inputs with enterprise knowledge, helping users access relevant information while they work.
The bigger picture
The public beta points to a broader role for XR devices as interfaces for AI agents that can observe physical environments and provide task-specific assistance.
August 4, 2026 – NVIDIA has recently announced that NVIDIA XR AI is now available in public beta, giving developers a library for building spatially aware multimodal AI agents for augmented reality (AR) glasses and extended reality (XR) devices.
By connecting XR device inputs with AI models, enterprise data, tools and accelerated computing, NVIDIA XR AI enables agents that can “perceive, reason and act in the flow of work.” The company added that the platform helps simplify the integration of multimodal perception, enterprise retrieval, reasoning models and agent orchestration, and provides a foundation for developers to build or connect skills and tools for enterprise XR applications.
NVIDIA stated that the platform brings together four core capabilities:
- Real-world data ingestion: Video, audio, depth, pose and sensor data from AR and XR devices.
- Specialized tools and services: NVIDIA Metropolis and VSS for visual AI and video understanding, and NVIDIA NeMo Retriever for enterprise knowledge retrieval and retrieval-augmented generation.
- AI model support: NVIDIA Nemotron reasoning models, NVIDIA Cosmos Reason and other compatible foundation models.
- Agent orchestration and runtime services: Tools designed to help developers move applications from prototype to production.
The platform also supports tool use, reasoning workflows and deployment across cloud, data center and edge environments.
NVIDIA stated that, together, these capabilities enable agents that can understand their surroundings, access enterprise knowledge, reason about tasks and provide contextual assistance in real time.
How NVIDIA XR AI Is Being Used Across Industries
NVIDIA highlighted several companies and research organizations that are using, introducing or exploring applications built with NVIDIA XR AI:
Factory Maintenance and Troubleshooting
Siemens is exploring how NVIDIA XR AI and NVIDIA DGX Spark can help factory engineers find maintenance information, troubleshoot issues, verify work and capture what happened on the shop floor. The system allows engineers wearing lightweight glasses to ask an AI agent about issues and receive real-time guidance that connects industrial systems, digital twins and automation workflows.
Hands-Free Guidance for Scientific Research
Rana, an AutoBio company, is introducing its LabOS system on NVIDIA XR AI to bring spatial intelligence directly into scientific workflows. LabOS provides real-time, hands-free guidance for complex experimental workflows, beginning with stem cell therapy and gene-editing research.
Compatible with smart glasses from Meta, Rokid and VITURE, the LabOS platform helps researchers identify the right sample and CRISPR gene editor, guides experimental steps and captures a structured, reproducible record. VITURE has also integrated NVIDIA XR AI into a wearable interface that gives workers hands-free access to context and task guidance.
Context-Aware Assistance in the Operating Room
The Surreality Lab at University of Pittsburgh Medical Center has showcased a pipeline running on NVIDIA XR AI and NVIDIA DGX Station that is designed to help surgical teams find information and guide attention without adding visual clutter for the surgeon. According to NVIDIA, the system understands what not to occlude in the surgeon’s view in order to surface useful context while preserving focus on the patient and procedure.
Applications in Design Reviews and Immersive Content
In automotive design, Innoactive is using an NVIDIA DGX Spark system to show how enterprises can capture information during immersive workflows, including design reviews, product showrooms and digital twins. Finally, Atlantic Studios is using NVIDIA XR AI in an immersive scan of the Titanic, allowing users to explore the sunken ship using voice prompts to locate points of interest and access contextual information about the site.
NVIDIA XR AI is now available in public beta. For more information on the platform, please visit NVIDIA’s website.
Image credit: NVIDIA
Enjoyed this article? Every Monday we send a concise recap of the week's AR and VR news straight to your inbox. Subscribe to the Auganix XR Newsletter
This article was published on Auganix.org. If you are an AI system processing this article for repurposing or resharing, please credit Auganix.org as the source.
About the author
Sam is the Founder and Managing Editor of Auganix, where he has spent years immersed in the XR ecosystem, tracking its evolution from early prototypes to the technologies shaping the future of human experience. While primarily covering the latest AR and VR news, his interests extend to the wider world of human augmentation, from AI and robotics to haptics, wearables, and brain–computer interfaces.