In Augmented Reality, Virtual Reality and Mixed Reality News
June 5, 2023 – After years of speculation, the day has finally arrived that Apple has announced its XR headset. Today at its Worldwide Developer Conference (WWDC), Apple unveiled the ‘Apple VisionPro’ – a revolutionary spatial computing device that seamlessly blends digital content with the physical world, according to the company.
The entire XR industry has been patiently waiting for the world’s richest company to bring some sort of augmented or virtual reality device to the table, with rumors swirling in recent months as to what form (if any) such a device would take. The wait is finally over though, and the device looks extremely impressive. We’ve broken down the announcement, trimmed some of the superfluous promotional-speak, and summarized the key points of what is probably the most important XR industry announcement of the decade below.
Vision Pro Overview
The Vision Pro introduces a fully three-dimensional user interface that is controlled by a user’s eyes, hands, and voice. Featuring ‘visionOS,’ Apple’s new spatial operating system, Vision Pro will allow users to interact with digital content in a way that feels like it is physically present in their space. The design of Vision Pro features an ultra-high-resolution display system that packs 23 million pixels across two displays, and custom Apple silicon in a dual-chip design.
Immersive experiences on the Apple Vision Pro
For work, the Vision Pro will bring a new dimension to powerful, personal computing by changing the way users interact with their favorite apps, capture and relive memories, enjoy TV shows and movies, and connect with others in FaceTime.
The new visionOS features a three-dimensional interface that frees apps from the boundaries of a display so they can appear side by side at any scale. Apple Vision Pro will give users “infinite screen real estate,” access to their favorite apps, and all-new ways to multitask. It also features support for Magic Keyboard and Magic Trackpad, enabling users to bring the capabilities of their Mac into Vision Pro wirelessly, creating an enormous, private, and portable 4K display.
For entertainment, the two ultra-high-resolution displays on the Vision Pro provide a screen that feels 100 feet wide. The device also features an advanced Spatial Audio system. Users will be able to watch movies, TV shows, or three-dimensional movies.
Apple Immersive Video offers 180-degree high-resolution recordings with Spatial Audio, and users will be able to access a lineup of immersive videos from the device, according to Apple.
For gaming, the company simply noted that “spatial computing makes new types of games possible,” but did not note any specific gaming titles that will be available on the device at launch. Users will also be able to play over 100 Apple Arcade games on a screen as large as they want, and the device includes immersive audio and support for popular game controllers.
Camera and photos on the Vision Pro
Featuring Apple’s first three-dimensional camera, Apple Vision Pro lets users capture, relive, and immerse themselves in memories with Spatial Audio. Every spatial photo and video captured will be able to transport users back to a moment in time. Users will also be able to access their entire photo library on iCloud, and view their photos and videos at a life-size scale, with panoramic shots viewable in an immersive mode that expands and wraps around the user.
FaceTime on the Vision Pro
With Apple Vision Pro, FaceTime calls take advantage of the room around the user, with everyone on the call reflected in life-size tiles, as well as Spatial Audio, so it sounds as if participants are speaking right from where they are positioned. Users wearing Vision Pro during a FaceTime call are reflected as a ‘Persona’ — a digital representation of themselves that reflects face and hand movements in real time. Users will also be able to do things together like watch movies, browse photos, or collaborate on a presentation.
Vision Pro Apps
Apple Vision Pro has an all-new App Store where users will be able to discover apps and content from developers, and access hundreds of thousands of familiar iPhone and iPad apps that will automatically work with the new input system for Vision Pro.
Vision Pro Operating System and User Interface
The new visionOS was designed from the ground up to support the low-latency requirements of spatial computing. The resulting operating system delivers powerful spatial experiences that can take advantage of the space around the user.
visionOS features a brand-new three-dimensional interface that makes digital content look and feel present in a user’s physical world. By responding dynamically to natural light and casting shadows, it helps the user understand scale and distance. To enable user navigation and interaction with spatial content, Apple Vision Pro introduces an entirely new input system controlled by a person’s eyes, hands, and voice. According to Apple, users will be able to browse through apps by simply looking at them, tapping their fingers to select, flicking their wrist to scroll, or using voice to dictate.
Apple Vision Pro also features ‘EyeSight,’ an innovation that will help users to stay connected with those around them. When a person approaches someone wearing Vision Pro, the device feels transparent — letting the user see the person, while also displaying the user’s eyes to them on the outside of the headset. When a user is immersed in an environment or using an app, EyeSight gives visual cues on the front of the device to others about what the user is focused on.
Vision Pro Design
The Vision Pro headset features a lot of technology in a compact design. A singular piece of three-dimensionally formed and laminated glass is polished to create an optical surface that acts as a lens for the wide array of cameras and sensors needed to blend the physical world with digital content. The glass flows into a custom aluminum alloy frame, while a modular system allows for a tailored fit to accommodate a wide range of people, according to the company. The Light Seal is made of a soft textile, and comes in a range of shapes and sizes, flexing to conform to a user’s face for a precise fit. Flexible straps ensure audio remains close to the user’s ears, while a Head Band — available in multiple sizes — is three-dimensionally knitted as a single piece to provide cushioning, breathability, and stretch. The band is secured with a simple mechanism, making it easy to change to another size or style of band.
Apple stated that the Vision Pro is designed to deliver phenomenal compute performance in a compact wearable form factor. Featuring an ultra-high-resolution display system built on top of an Apple silicon chip, Vision Pro uses micro-OLED technology to pack 23 million pixels into two displays, each the size of a postage stamp, with wide color and high dynamic range. Custom catadioptric lenses enable sharpness and clarity during XR experiences, and users with vision correction needs will be able to use ZEISS Optical Inserts to ensure visual fidelity and eye tracking accuracy.
An advanced Spatial Audio system is core to the Apple Vision Pro experience, creating the feeling that sounds are coming from the environment around the user and matching the sound to the space. Two individually amplified drivers inside each audio pod deliver Personalized Spatial Audio based on the user’s own head and ear geometry.
In addition to display and audio features, the Vision Pro also includes a high-performance eye tracking system that uses high-speed cameras and a ring of LEDs that project invisible light patterns onto the user’s eyes for responsive, intuitive input.
These innovations are all powered by Apple silicon in a dual-chip design. The device features Apple’s M2 chipset to deliver standalone performance, while the brand-new R1 chip processes input from the Vision Pro’s 12 cameras, five sensors, and six microphones. According to Apple, R1 streams new images to the displays within 12 milliseconds — 8x faster than the blink of an eye. Apple Vision Pro is designed for all-day use when plugged in, and up to two hours of use with its external, high-performance battery.
Privacy and Security on the Vision Pro
Apple stated that the Vision Pro is built on a foundation of privacy and security, and keeps users in control of their data. Optic ID is a new secure authentication system that analyzes a user’s iris under various invisible LED light exposures, and then compares it to the enrolled Optic ID data that is protected by the Secure Enclave to instantly unlock Apple Vision Pro. A user’s Optic ID data is fully encrypted, is not accessible to apps, and never leaves their device, and ultimately not stored on Apple servers, according to the company.
Where a user looks stays private while navigating Apple Vision Pro, and eye tracking information is not shared with Apple, third-party apps, or websites. Additionally, data from the camera and other sensors is processed at the system level, so individual apps do not need to see a user’s surroundings to enable spatial experiences. EyeSight also includes a visual indicator that makes it clear to others when a user is capturing a spatial photo or video.
Commenting on today’s announcement, Apple CEO, Tim Cook said: “Today marks the beginning of a new era for computing. Just as the Mac introduced us to personal computing, and iPhone introduced us to mobile computing, Apple Vision Pro introduces us to spatial computing. Built upon decades of Apple innovation, Vision Pro is years ahead and unlike anything created before — with a revolutionary new input system and thousands of groundbreaking innovations. It unlocks incredible experiences for our users and exciting new opportunities for our developers.”
“Creating our first spatial computer required invention across nearly every facet of the system,” added Mike Rockwell, Apple’s vice president of the Technology Development Group. “Through a tight integration of hardware and software, we designed a standalone spatial computer in a compact wearable form factor that is the most advanced personal electronics device ever.”
How much will the Vision Pro cost?
Apple Vision Pro will start at USD $3,499, and will be available early 2024 on apple.com and at Apple Store locations in the U.S., with more countries coming later next year. Customers will be able to learn about, experience, and personalize their fit for Vision Pro at Apple Store locations. For more information about Vision Pro, visit apple.com/apple-vision-pro.
To watch the full keynote from WWDC, click here.
Image / video credit: Apple
About the author
Sam Sprigg
Sam is the Founder and Managing Editor of Auganix. With a background in research and report writing, he has been covering XR industry news for the past seven years.
