The Vision Pro has once again brought augmented reality and virtual reality to mainstream attention, and Apple has begun leveraging its extensive product ecosystem to make the headset look more appealing. Case in point: The company is betting that spatial video will become everyone’s favorite way to relive past memories. So, in this article, we’ll take a look at what spatial video is, how it works, and where you can experience it for yourself.
What is spatial video?
Spatial video is the latest term for 3D video, made popular thanks to the Apple Vision Pro headphones. In contrast to typical 2D videos, spatial videos can only be viewed using VR headsets because they deliver a different image to each eye. You see, we see the world with two slightly separated eyes. Our brains combine these views to judge depth and distance of objects.
Think back to when you were in a movie theater wearing blue and red glasses to watch a 3D movie. These glasses polarize or block certain types of light, allowing a different image to reach each eye. This tricks our brains into perceiving depth even when no depth is actually present. Spatial video is an extension of the same principle, but using headphones instead of two projectors. It’s easier than ever for anyone to record and play back their own 3D content. The next section will detail how to record spatial video.
Similar to how we need both eyes to perceive our surroundings, spatial video needs to be recorded from two cameras simultaneously. This isn’t a new concept, as true 3D movies have been produced using dual-camera systems for decades. However, until recently, this kind of “stereoscopic” capture was relatively elusive for ordinary people like you and me.
How does spatial video work?
The word “space” may confuse some people, but you can’t walk around in 3D space or interact with the video in any way. It’s also different from 360-degree video in that it doesn’t surround your surroundings.
Spatial video does provide a more realistic and immersive experience, but it’s still a flat projection inside a VR headset. Furthermore, the effects can be subtle or impressive. As with 3D videos from just a few years ago, the depth effect is most noticeable on close objects rather than distant ones. This means you’re better off recording intimate moments with your loved ones in this format. Landscapes and city skylines wouldn’t benefit much from this technology.
Despite its optimistic name, the 3D effects of spatial video appear subtle in many cases.
In late 2023, Apple announced that iPhone 15 Pro and 15 Pro Max users would be able to record spatial video without any additional hardware. Of course, the iPhone display cannot display the depth data captured in spatial video. But the good news is that you can record spatial video now and be ready for any upcoming VR/AR headsets you might buy in the future.
iPhone 15 Pro models can only record spatial video in landscape mode, as Apple cleverly uses the main and ultra-wide cameras to capture depth information. The tiny gap between the two sensors replicates the way our eyes see the world.
Does Android support spatial video? What is MV-HEVC?
Robert Triggs/Android Authority
Currently, there are no Android phones capable of recording space videos. Additionally, Apple has developed a new storage-efficient way to record spatial video called Multi-View High-Efficiency Video Coding, or MV-HEVC. It is an extension of the HEVC codec, which is already used by most smartphones to record standard videos. However, it is unclear whether Android smartphones will use MV-HEVC.
Instead of storing two different video sources, MV-HEVC only records the differences between the two cameras. This allows 2D video players (such as iPhone or Android smartphones) to ignore the depth information when reading the file. On the other hand, 3D video applications made for VR headsets can use basic 2D images and accompanying depth data to calculate two different views for each eye.
How to watch space video?
The best way to view space videos is with a VR headset like the Apple Vision Pro or Meta Quest 3. The former offers the most seamless way to watch space videos yet, as it can wirelessly sync videos with your iPhone via iCloud.
On the Meta Quest 3 or other headsets that support spatial video, you’ll need to manually upload the video to the headset. For example, Meta’s official smartphone companion app allows you to upload space videos. You’ll then need to find the video in your headset’s Files app and play it from there.
Spatial video certainly lets you experience memories in a whole new light, but it’s still not enough to convince many first-time headphone buyers. Still, even if you’re not planning on spending $3,500 on a Vision Pro today, I’d still recommend recording spatial video if you own an iPhone 15 Pro. There are no disadvantages other than a slightly larger file size, and you’ll benefit from it one day.