Doug Magyari, CEO/CTO of IMMY Inc., discusses the use of mirror-based optics, and challenges the existing optical paradigm of using Lens-Based optical systems in Near-To-Eye applications, which produce planar imagery and cause the ‘Vergence/Accommodation conflict’. IMMY offers an alternative approach that utilizes Mirror-Based optics to produce volumetric imagery, which does not cause this Conflict.
Author: Doug Magyari
It is typically stated that humans see the world in 3D. It is far more accurate to say that we see the world volumetrically. As light comes towards us from many angles the eye collects the light data and the visual cortex of the brain renders an image from the data. The actual perceived volume of vision comes as a result of each piece of light reaching the eye at different times from many angles. It is the time differential that creates the volume we see; this fundamental point cannot be overstated. Time is the major contributing factor used by the brain’s visual cortex to determine depth. Stereoscopy or parallax is a subcomponent of depth perception. If stereoscopy were the main factor resulting in our volumetric perception of the world, everything would collapse into a flat plane if we covered one eye. Our brain is not only using parallax to create a 3D view, but it is also using time to generate a volume.
Almost all Near-To-Eye (NTE) systems today use some form of lenses to create their imagery. They have been built to see the world as a camera-lens sees the world by setting an image plane; rather than volumetrically, as our human eyes see the world. Lenses cause light to image to a plane, which is the opposite of a volume. In the real world, there is no such thing as an image plane. This is a manmade contrivance and is the biggest limiting factor with current AR/VR/MR/XR hardware optical strategy. It is the fixed distance, planar imaging that causes the vergence/accommodation conflict (VAC), eye strain, and eye fatigue. Lens – based transmission can be thought of as a compression of a source volumetric image (like a camera looking at the real world) to a projected flat virtual image which is detected by the eye and brain. The resulting image is perceived as unnatural and uncomfortable.
Planar wavefronts are THE problem, Freeform wavefronts are the solution
If a lens-based NTE system tries to create a 3D image, it will use two slightly off-set planar images to create a stereoscopic effect. Lens-based NTE systems use the camera frustum to try and set a perceived distance, but this has no correlation to the real world because the generated image is planar. This forced 3D vision effect is what causes so many people to be unable to watch 3D content for any length of time without eye strain or nausea. The eye and visual cortex are uncomfortable with a planar wavefront entering the eye as this does not happen in the real world. Our vision is based on freeform wavefronts.
To properly design a head-mounted display that meets the basic criteria of no eyestrain, nausea, or eye fatigue; the optical design must follow the rules of how humans see the real world, it must create a freeform wavefront volumetric image.
There is a way to generate a volumetric image and relay it to the eyes – using mirrors
First, it is important to understand that mirrors do not have an image plane. A mirror will simply give you back what you feed into it; you can comfortably see from <1mm to infinity. Mirrors can provide an infinite depth of field and are not limited to image planes.
Since mirrors do not have an image plane, they do not induce the vergence/accommodation conflict – they are accommodation invariant; all objects visible in a mirror are always in focus when you look at them. As an example; when you look in the rearview mirror of your car the kids in the backseat are in focus. If you decide to look 300 feet or a half a mile behind you, that is also in focus, simply because you decided to focus your attention on something else. This happens because freeform wavefronts are being received by the eyes from all depths continuously.
The world is a mirror
Humans do not have the problem of Vergence/Accommodation when we look at the real world. Light reflects off of every object in our field of view, our eyes collect the reflected light energy, and the visual cortex makes an image out of the data (if there are sufficient mono and stereo cues to do so). You could say that the whole world is a mirror, it reflects light into our eyes, and the brain’s visual cortex makes images out of the light rays.
Since everything in a mirror is in focus when you look at it, one can choose what to verge on; this will naturally shift the eye’s accommodation, automatically de-tuning (but not blurring) non-essential visual information in the visual cortex instead of artificially blurring the source image which causes optical and brain distress.
It is unnecessary and undesirable to try and force vergence; that is not how the human visual system works. Forcing the eye to accommodate with blurring and PSF (point spread function) contrast causes eye strain and does not resolve the real-world disparity issue. In other words that is not a real solution. The only approach that does not cause distress is IMMY’s unique mirror approach.
Natural Eye Optics
The IMMY volumetric mirror optic system produces genuine lightfield characteristics so vergence and accommodation occur naturally, hence the name “Natural Eye Optic.” True immersion and true correlation of content to the real world come naturally only as a result of creating a volumetric image.
Utilizing the varifocal attribute of mirrors for the application of near-eye displays requires a multi-stage relay system with a minimum of three bounces to achieve satisfactory wear-ability, wide Field of View, acceptable form factor, large eyebox, low Distortion, and a wide range of IPD accommodation. Meeting such system requirements necessitates the use of highly complex freeform off-axis concave and convex mirror surfaces. Our system produces the effect of placing volumetric virtual images at any depth, thereby exhibiting genuine light-field characteristics.
How it works
First, we convert the original 2D OLED display light rays into a freeform wavefront. The original Lambertian (hemispherical radiation) distribution of light energy is converted into a volumetric display using a bi-concave freeform mirror [see M1 above). The hemispherical wavefront enters the bi-concave M1 mirror with a time differential hitting different locations on the curved surface at different times. This causes a super-sampling (upscaling) of the original 2D flat display to be “seen” as a volume by the M2 mirror. This provides a true volumetric freeform wavefront being relayed into the magnifying mirror, M2 [see M2 above]. Every point of every pixel is visible at every point on the curved M1 mirror; this enables the entire image volume to be simulcast to the M2, even though it now has a complex time signature. This occurs as an off-axis inversion, where the rays from the M1 converge on top of one another on the M2, at approximately a 30:1 reduction in area. This incredibly complex freeform wavefront is relayed to the M3 (combiner) mirror surface via the M2 compound convex freeform [see M3 above] creating a true lightfield volumetric image. The M3 is where the ray bundle is unbundled, collimated, and relayed directly through the pupil and onto the retina.
Our freeform wavefront reflected off of the M1, M2, and M3 freeform surfaces places virtual images at any depth and from multiple viewing angles allowing the eye to move around freely absorbing freeform wavefront light energy from many different angles. This natural process of absorbing and processing both mono and stereo cues produces an in-focus image at the proper depth with a very large eyebox.
Unlike Maxwellian pinhole type optics, which have a very small eye box and large Depth of Field (DoF), the IMMY optic generates an extremely large DoF, with a large eye box. This Mirror Optic method addresses the basic needs of Human comfort, Human safety, Eye comfort, and Eye safety. Below are slides from a SIGGRAPH presentation (cited below) showing the Maxwellian problem and the waveguide problem of small eyebox.
Unlike Maxwellian pinhole type optics, which have a very small eye box and large Depth of Field (DoF), the IMMY optic generates an extremely large DoF, with a large eye box. This Mirror Optic method addresses the basic needs of Human comfort, Human safety, Eye comfort, and Eye safety. Below are slides from a SIGGRAPH presentation (cited below) showing the Maxwellian problem and the waveguide problem of small eyebox.
At the current time, Immy’s approach is unique and lens-less. Every headset other than IMMY’s uses some combination of lenses to create their virtual image. Since lenses produce an image plane at a fixed distance, the VAC is an optically induced problem. Any headset that uses lenses will thus suffer from VAC and will require the implementation of some form of correction. These correction methods are computationally intensive, expensive, and in most cases induce additional eyestrain and human factor issues. The human vision system is far too complex for this issue to be solved artificially. Here are several examples of attempts to solve the VAC published from the same SIGGRAPH presentation. And, perhaps even more important, these attempted solutions are simply changing the focal distance of the generated planar wavefront which is not a solution to the actual problem. The real problem is how to get freeform light into the eye not a planar wavefront.
After decades of attempts to produce an economically practical AR/VR/MR/XR headset, all of which have been a variation on the same theme, it seems obvious that something is wrong with the approach. Billions of dollars have been spent producing more and more complex attempts with no satisfactory results. Although there is far more to this mirror system than I am able to present in a few pages, it demonstrates a material paradigm shift in how HMD’s should be built. The method of operation of the Human vision system is the central theme, and presenting as natural and unadulterated freeform wavefront imagery as possible is the objective. The visual cortex does the rest.
— Ends —
For more information on the IMMY Mirror Optic System and the Immersive Imaging Glasses, you can visit the company’s website at www.immyinc.com, or, you can reach IMMY directly at info@immyinc.com.
For more background on the author, Doug Magyari, IMMY’s founder and CEO, you can visit his website here.
References:
¹ R. Konrad, N. Padmanaban, K. Molner, E.A. Cooper, G. Wetzstein. “Accommodation-invariant Computational Near-Eye Displays”, ACM SIGGRAPH (transactions n Graphics 36,4) 2017.
² PATENT US20120229367A1. Head Mounted Display Device
³ “ Flat Focusing Mirror”, https://www.nature.com/articles/srep06326 authors: Y.C. Cheng, S. Kicas, J. Trull, M. Peckus, C. Cojocaru, R. Vilaseca, R. Drazdys, and K. Staliunas.
About the author
Sam Sprigg
Sam is the Founder and Managing Editor of Auganix. With a background in research and report writing, he has been covering XR industry news for the past seven years.
