In case you hadn’t noticed, Apple launched its Vision Pro spatial computing headset on February 2, 2024. It’s a bold move, signaling the tech giant’s entry into the virtual and augmented reality market—with an equally bold price tag of $3,500 USD per unit.
One of Vision Pro’s unique advantages is its place in the Apple product ecosystem, so you can expect synergy with Apple’s Macs, iPhones and other products. For instance, you can capture 3D video with an iPhone and view it on a Vision Pro later. Knowing this potential, I’ve been shooting spatial videos over the holidays in 2023 and beyond, intending to relive special moments when the headset finally arrives.
But what you might not know is that, with a little effort, you can view this 3D video without a headset, albeit at not nearly the same fidelity as the Vision Pro. And that’s what we’ll be taking a look at here.
A brief 3D primer
Believe it or not, the quest for 3D image capture goes back to the era of Thomas Edison and beyond. Stereoscopic devices like the kinetoscope used still photos from two cameras mounted next to each other and then viewed as a flip book through a set of goggles. These formats were mainly curiosities until the 1950s when 3D movies were produced in the now-iconic anaglyph format, with filmed imagery from two cameras dyed red and cyan and projected via side-by-side synchronized projectors.
A viewer wearing red and cyan-filtered glasses could perceive depth by having each eye view one of the two perspectives projected on a flat screen. Popular 3D movies from the era included House of Wax and It Came from Outer Space. The craze proved short-lived as it was technically complex for exhibitors and somewhat uncomfortable for audiences.
3D saw additional brief revivals from the 1960s to the 1980s with movies like Jaws 3-D and Spacehunter: Adventures in the Forbidden Zone. Unfortunately, many of the films focused on the effect itself and could have benefitted from better scripts. So, the format remained niche.
Later efforts in the 1990s brought innovations such as IMAX large format 3D capture and polarized glasses. Polarized glasses worked with a synchronized projector to provide a full-color—albeit somewhat dim—image that proved more immersive and less fatiguing to viewers. However, the format also remained niche compared to typical 2D 35mm theatrical releases.
Everything changed in 2009 with the release of James Cameron’s Avatar, a landmark achievement in 3D cinematography and hyperrealistic computer-generated imagery. While the 3D format remained optimized for polarized glasses, the scope of the storytelling and breathtaking imagery proved highly popular, and Avatar remains the highest-grossing movie of all time.
Avatar ushered in a new era of digital 3D capture and major theatrical releases. It also led to new 3D home projectors and televisions with active shutter 3D glasses, which enabled a higher quality and brighter home viewing experience.
Interest in 3D movies has tapered off since then, though it remains highly popular in some markets, such as Asia. But one thing is clear: the ongoing attempts to drive 3D media indicate that manufacturers believe there’s still an appetite for 3D as long as the immersion and emotional engagement is sufficient.
Whether the Vision Pro and its ability to store and share 3D experiences in 8K quality meets those expectations remains to be seen.
The Vision Pro’s Immersive Videos
Apple’s Immersive Videos format is a special container for 3D or “spatial” video. You can capture spatial video to this format either by using the Vision Pro as a head-mounted camera, or with an iPhone 15 Pro or 15 Pro Max. The headset offers better capture because its cameras are more optimized for 3D, resulting in higher resolution and improved depth effects.
However, it will likely be more socially awkward to capture in most situations while wearing the headset until its appearance becomes more commonplace. The iPhone trades in technical quality for a less off-putting experience.
While the iPhone wasn’t designed specifically as a 3D camera, it can use its primary and ultrawide cameras in landscape orientation simultaneously, allowing it to capture spatial video—as long as you hold it horizontally. Computational photography is used to compensate for the lens differences, and the output is two separate 1080p, 30fps videos that capture a 180-degree field of view.
This might not be a match for the stereoscopic 8K that the Vision Pro is capable of—which requires a more expensive, much bulkier setup—but pretty impressive for the camera in your pocket.
MV-HEVC: The Vision Pro’s spatial video format
These spatial videos are stored using the MV-HEVC (Multi-View High-Efficiency Video Coding) format, which uses H.265 compression to crunch this down to approximately 130MB per minute, including spatial audio. Unlike conventional stereoscopic formats—which combine the two views into a flattened video file that’s either side-by-side or top/bottom—these spatial videos are stored as discrete tracks within the file container.
This helps to maintain compatibility with non-stereo devices, which would otherwise display both views simultaneously. So opening a spatial video clip on an iPhone, iPad, or a desktop Mac will play just one of these video channels as a standard flat 1080p, 30fps video. To view both encoded angles in native 3D/stereo at full resolution and full color, you’ll need a Vision Pro. Or you can see them in anaglyph red/cyan 3D with a little conversion magic, which we’ll get into in a bit.
How to Capture Spatial Video on an iPhone
Use the following steps to record spatial/3D clips on iPhone 15 Pro/Pro Max or greater:
- Ensure your iPhone is updated to iOS 17.2 or later in Settings->General->Software Update.
- Next, go to Settings->Camera->Formats and enable Spatial Video for Apple Vision Pro.
- Then, open the Camera app and swipe to Video mode.
- Hold your iPhone horizontally and press the icon that looks like goggles.
- Next, hit the Record button, and you’ll be recording a spatial video.
Here are some tips for getting the best spatial videos, i.e., clips that will be the most comfortable and immersive when viewed in 3D.
- Shoot with plenty of light, i.e., daylight or some lights at night. Avoid complete near or complete darkness.
- Compose the foreground and background carefully.
- Make sure there’s plenty of distance between the foreground and background. If the overall depth in the frame is minimal, it will look more flat when viewed in 3D.
- To emphasize depth, try to have something very close to the camera within a few feet and a background that’s at least five feet away or more.
- Move slowly and smoothly. Remember, this is supposed to represent a person’s direct POV. Consider using a tripod or a stabilized gimbal to smooth out your shot as much as possible, or practice moving and bracing the camera as well as you can.
- Ideally, don’t move at all. With enough object movement and depth in sight, a completely locked-off camera can provide a great sense of presence.
- Shoot at a natural eye level. Avoid a super-high or a low angle perspective unless you’re aiming for a bird’s or bug’s eye view.
- Consider whether the viewer at a human scale will be seated or standing. The goal is to recreate a “you are there” experience.
- When you’re done, stop recording. You’ll find your video in a new Media Type album in the Photos app called Spatial.
Spatial video samples
I’ve put together a small collection of spatial video samples that I’ve shot using the process above. You can download the original MOV files from here if you’d like to follow along.
How to view Apple spatial videos
Spatialify is an iOS app designed to view and convert various 3D formats. It also works well on Mac OS, as long as your Mac has an Apple Silicon CPU. And it supports MV-HEVC, so you’ll be all set. It’s just $4.99, a genuine bargain considering what it does. Find Spatialify here. To view your clips in 3D, you’ll need a pair of red/cyan glasses. I found this perfectly serviceable set here on Amazon.
- Launch Spatialify and click the Browse Spatial Album option at the top of the main window.
- You should now see all your spatial video, assuming you are syncing the Photos app between your iPhone and Mac.
- Select a clip you’d like to view and click on it.
- In the Preview window, toggle to the Anaglyph mode.
- Put on your 3D glasses. It might take a bit for your eyes to adjust to the reduced brightness and limited color, but if you’ve followed the steps, you should perceive depth in the clip.
- To export the clip as a QuickTime movie, click the Share icon in the top right of the preview window.
- Choose the Anaglyph Format. If the depth doesn’t seem correct, toggle the Horizontal Disparity Adjustment on or off.
- Hit the Export button and choose a location to save the exported Anaglyph clip.
Anaglyphic 3D video samples
I’ve converted my collection of spatial video samples using this technique and you can download them from here.
Viewing anaglyphic spatial content
- The clip can now be viewed in any QuickTime-compatible player.
- The clip can also be edited in any NLE because the 3D effect is now “baked in.”
- Remember not to do excessive color grading because you can muddle the red/cyan tinting of each angle necessary to preserve the 3D effect.
Conclusion: Will 3D Catch On (Again)?
Sharing spatial memories has the potential to be a killer app for Vision Pro because they are so immersive and can provide a deep emotional connection. It feels like you are somehow reliving a waking dream instead of just looking at a video. It will be fascinating to see how this new spatial ecosystem develops and what additional experiences Apple has in store for Vision Pro users in the future.
By reportedly selling out its initial supply quickly, Vision Pro is clearly a success story for Apple. But only some can afford it or are ready for one. This workflow is just a taste of what’s possible with the full headset. Seeing a preview of spatial video via this more accessible method might just convince you that this is a product for you.
It should also be mentioned that you don’t actually need a Vision Pro to view spatial video in its native format—Meta’s Quest 2, 3, and Pro VR headsets will also support spatial video playback. And while Meta might not be doing this for altruistic reasons, it’s good news for the format, expanding spatial video’s user base from the estimated first run of 150,000 Vision Pro headsets to the reported 20+ million Quest headsets already sold.
