You don’t have to look hard to see how virtual production is taking the film, TV, and streaming industries by storm.
And like all new developments, the demand for qualified talent is high, while the availability of skilled people is low. This situation is compounded by roles that are constantly being refined and redefined.
So let’s take a look at what’s driving this incredible demand, and where opportunities can be found for newcomers and veterans alike. Starting with how we got here.
Contents
Previously…
Thanks to groundbreaking productions like The Mandalorian, many of us associate virtual production with in-camera VFX and LED volumes.
But the term “virtual production” encompasses many different workflows of which the LED volume is just one. So a better description would be that it’s the intersection of the physical production world with virtual elements.
And when you look at it like this, you can easily see how earlier productions provided the foundations for today’s techniques.
Avatar
James Cameron’s Avatar is one such project, coining the term virtual cinematography to describe the motion capture of both actor and camera movement to drive more realistic computer animation shots. And that stood, in part, on Peter Jackson’s early efforts in The Lord of the Rings (2001-2003) and Robert Zemeckis’ The Polar Express (2004).
Gravity
Another giant leap was taken when Gravity arrived in 2013.
To create the Oscar-winning freeform camera and imagery of astronauts in Earth orbit, the actors wore partial spacesuits on sets illuminated by large LED panels. While the resolution of these panels wasn’t sufficient to have them captured on-camera, they still provided interactive lighting effects on the actor’s faces by playing video footage of the visual effects backgrounds.
In most of the completed space shots in Gravity, the only real filmed objects are the actor’s faces. Everything else was created digitally.
Rogue One and Solo
The same technique was used heavily in Rogue One, where effects shots were played back during filming to provide interactive lighting effects onto the live-action sets and actors’ faces.
Solo took things a step further by incorporating live real-time animation. For example, in shots where Tie Fighters are chasing and attacking the Millennium Falcon, the filmmakers could flash green laser bolts into the shots projected into the cockpit on cue.
And because the Solo filmmakers utilized high-resolution laser projectors and massive projection screens instead of LED panels, they could capture some shots completely in-camera, bringing us a step closer to the LED production volumes of today.
For example, the shot where Han Solo first jumps into hyperspace aboard the Millennium Falcon uses visual effects captured completely in-camera.
The Lion King (2019)
To round off this recap, the 2019 all-CGI remake of The Lion King brought us even closer to today’s techniques by combining motion capture and real-time game engine animation with real-world filmmaking techniques.
Unity was used to create a real-time simulation of the entire film, then real filmmaking equipment—dollies, Steadicam rigs, cranes, and drones—was used to control virtual cameras within the virtual space. The end result was camera movement that looked natural, realistic, and human-driven. Because it was.
What’s most interesting about this approach is how it provided new opportunities for traditionally trained, live-action camera crew on an entirely computer-generated movie.
(On a side note, The Lion King also used virtual scouting, where the filmmakers used VR headsets in a shared virtual reality simulation of a proposed set, allowing them to collaborate on production design and shot coordination.)
And now…
All of which leads us to The Mandalorian, which made baby Yoda a meme and put LED volumes and in-camera VFX squarely on the production map.
Of course, The Mandalorian could have been made entirely using more traditional visual effects techniques. But it probably wouldn’t have had the same level of realism and dynamics. Not to mention being much harder to produce at the high-speed schedules required for streaming and broadcast.
Series creator Jon Favreau took huge risks and asked the impossible from his team, and the results are awe-inspiring.
Based on the project’s success, many other productions have taken note and followed suit—driven in part by creative intent, but also by the limitations of operating during a pandemic.
One of the most recent is Star Trek: Discovery, which incorporated an LED volume into the visual effects pipeline of its fourth season.
Unsurprisingly, the virtual revolution isn’t limited to North America, either. At Studio Babelsberg outside Berlin, Germany, a Netflix series called 1899 recently wrapped production. 1899 is a mystery set aboard a transatlantic ocean liner at the turn of the 20th century and is chalked for release later this year.
Initially, it was envisaged that production would follow the mold of Game of Thrones, making extensive use of location work throughout Europe. But faced with our current health situation, the production was rapidly retooled so that it could be shot entirely within the confines of a giant LED volume.
1899 even took a step further than The Mandalorian by constructing its volume around a massive turntable. This allowed for the modular construction and orientation of huge sets representing various sections of the ship, as well as enormous flexibility during shooting and prep between locations.
Reportedly, when directional sprinkler systems were incorporated to recreate falling rain for stormy seas, the final effects were so authentic that cast and crew sometimes felt seasick.
A very particular set of skills
What should be clear from all of this is that virtual production is not a transient technology.
It also requires a unique combination of traditional methods and new skills, including video I/O, real-time animation, tech support, and theatrical stage skills.
So where do we fit in this brave new virtual world? What opportunities can we find when we look a little closer?
Going remote
We already know that virtual production owes some of its popularity to the realities of the pandemic.
At its most basic, the footprint of an LED volume can be reduced to a camera operator and the actors. Nearly every other role can be performed remotely, either by someone in another room, across the street, or even connected via VPN in a video call from anywhere in the world.
It’s this last aspect—and the relaxation of prior restrictive IP security parameters from content creators—that continues to drive new opportunities that are divorced from geographic limitation.
Potentially, anyone in the world can now participate and contribute in real time to a virtual production via video conferencing and private network technology.
So, hiring managers no longer need to restrict their talent searches to local markets. They can source and work with whomever the best artist may be, wherever they may be. The opportunities could not be stronger than they are at this stage in virtual production.
The opportunities could not be stronger than they are at this stage in virtual production.
But the question remains: what skills are most sought after?
The talent pool
When you take into consideration all of the components required to make an LED volume operate, which include the LED panels and their video processors, the camera tracking systems, and the computers needed to deliver real-time content, it’s already a diverse set of skills.
Factor in the live aspects of the process and then add the typical crew that’s needed to capture digital cinematography, and you have an even more extensive set of requirements.
Game devs
Because so much of the work in virtual production involves game engines—software like Unity and Unreal Engine—game developers are a natural fit for virtual production.
They’re already very familiar with digital assets and environment creation, though they may have to adapt to a film set’s different pace and terminology.
3D artists
Virtual productions are also likely to need 3D artists—and this is a less software-specific field than game development because translation tools exist for all the major 3D DCC (digital content creation) packages like Maya, 3D Studio, Blender, and even CAD software like SolidWorks and AutoCAD.
The challenge here will be to understand the performance parameters of real-time shaders and the optimization of models to perform in real time.
Visual effects specialists
Existing visual effects professionals such as technical directors, supervisors, and coordinators are also well placed to make the move.
In some ways, they’re probably the most suited because they’re accustomed to the evolutionary changes and updated techniques that constantly impact visual effects.
However, they’ll need to handle the inversion of labor concentration. For example, in a traditional visual effects pipeline, most of the work often happens during post-production (like green screen elements captured during production being combined with additional animation in post). In a virtual production for an LED volume, that same visual development and finalizing of assets happen much earlier in pre-production.
At the same time, post-production has a more brief and less complex pipeline.
Live stage professionals
Another intriguing talent source for virtual production is live stage professionals.
Anyone experienced with the pressures and focus of handling media for live performances will be familiar with the pace of real-time animation.
Of course, they may be less familiar with the requirements of cinema-style production and its jargon and workflows. But learning tech and terminology takes less time than building the experience you need to work effectively in a real-time virtual production environment.
Conventional tech specialists
Because so much computer support is needed to operate and feed content to an LED volume, IT professionals, video professionals, and integrators are another vital source of talent.
An LED volume is made up of individual panels assembled as a giant display surface, or series of surfaces. It can include curved walls, ceiling panels, additional frontal screens, etc. Each of these surfaces is connected to a high-end desktop workstation-class computer and video processor with literally miles of ethernet, coaxial, and other cables.
Even just the amount of wireless equipment operating within the same stage may require a specialist trained in bandwidth management. So it’s fair to say there’s plenty of work for IT professionals in virtual production—even if they’re just there to connect the set to the internet.
Unconventional tech specialists
Finally, technologists with experience in VR, AR, and XR are also valuable for consultation on LED volume integration and operation.
There are so many different systems that must work together, in concert, day in and day out, to keep production on schedule. People with experience in the integration and management of so many disparate components will always be in demand for virtual production.
Getting good
There are clear pathways for experienced video and film professionals to follow into virtual production.
But if you’re a newcomer there’s also good news. Fueled by the intense demand of an ever-increasing spectrum of streaming services like Netflix, Hulu, Disney+, Apple TV+, Amazon Studios, Peacock, et al, some companies are taking a proactive stance to talent education and acquisition.
Epic Games
For example, Epic Games offers the Unreal Fellowship, a workshop series aimed at bringing experienced visual effects practitioners up to speed with Unreal Engine.
Each session runs for thirty days and is not just free to participants, but even offers a stipend so that in-demand talent can take time off from their existing work to focus on the fellowship.
Originally arising out of necessity during the pandemic, the Fellowship continues to be an excellent opportunity for anyone looking to make the pivot into real-time and virtual production. And, because space is limited and interest is high, Epic also offers this virtual production primer, a completely free and open course anyone can take.
Gnomon
Gnomon is a longtime technical training school for visual effects, and also offers training courses in virtual production.
These classes are convenient and affordable, and offer practical knowledge from working professionals. I had the opportunity to conduct a guest session on LED volumes hosted by instructor Dane Smith (and also vice president at The Third Floor, a leading visualization studio and virtual production company).
Vū Studios
A company seeing value in collaboration with higher education initiatives is Vū Studios in Florida.
Vū operates a growing network of fully-equipped LED volume virtual production stages that can even operate in tandem to enable talent in different locations to collaborate live. Vū also donates access to its virtual production stages to film students at the University of South Florida.
Virtual production glossary
Another new resource that can help you get your head around virtual production is The Virtual Production Glossary. I had a blast co-writing this with a who’s-who of virtual production filmmakers and professionals. And because it’s a community-sourced effort, we hope it will help get everyone working together with a common language.
Higher education
In addition to these online and specialized resources, higher education and traditional accredited film schools are quickly adapting their offerings to include virtual production and game engine workflows.
One of the largest and most exciting recent announcements was the endowment of the Martin Scorsese Institute of Global Cinematic Arts to include a virtual production LED component.
I’ve also worked directly with NYU faculty and students on virtual production pilot introduction courses, including remote stage sessions. You can learn more about NYU’s virtual production journey on my podcast.
And another team I’ve had the opportunity to work with is Chapman University in California. Chapman moved into LED volume work very quickly and offered current students an in-class option to create their projects with a modest LED wall.
The next part
As we’ve already discussed, virtual production isn’t going away. Estimates vary but reports only project an upward trajectory for this sector of the film and TV industry.
But nor is conventional production, either. And I hope that you now share my view that it’s possible to find a place for yourself in either field, or even cross from one to the other, regardless of where you might be in your career.
Next time, I’ll be taking a deeper look at the specific roles required in virtual production and how they’re analogous to or derived from traditional production and post-production positions. So stay tuned—and in the meantime, why not check out this article on the future of filmmaking?
Featured image © Alex Forge/Netflix
