Back in my last article on virtual production, we looked at this industry sector’s incredible growth over the last couple of years. In addition, we learned how complex virtual production workflows can be, and how this has led to a surge in demand for entirely new, highly-skilled positions.
And we shouldn’t underestimate this demand for virtual production specialists, because it’s driven by a scarcity of qualified, experienced personnel. This means that, for many of us, now is the perfect time to get in on the ground floor of how movies are made.
For others, who may already have a spot in the industry, it might be a great time to pivot from an existing role in the traditional visual effects pipeline to something analogous in virtual production.
So let’s take a look at what these virtual production roles are, the kind of skills you’ll need to fill them, and the conventional production/post jobs that could give you a leg up into the virtual realm.
Contents
1. Volume operator
A volume operator is a crew member from the volume operations (or brain bar) team who is involved with the operations and content related to the LED volume.
The volume operators may include key volume operator, assistant volume operator, etc. Content may run the gamut from simple 2D video played back for rear projection setups to fully 3D real-time game engine environments.
Expertise in both real-time rendering software and in live video environments is important for the volume operator. Generally, a team of volume operators working together will manage anything larger than a single surface LED volume. An eye for detail and how things should look for real-world cinematography are also advantageous to the operator.
Talent pool
Experience with a real-time engine is one of the key skills demanded by the volume operator role. So, game developers and animators accustomed to the software will be solid sources for talent. Because this is a new role and a critical one for successful LED operations, training talent from scratch is often necessary to fill vacant positions.
2. Motion capture supervisor
A motion capture supervisor works with motion capture data, which can come from performers in suits and camera tracking systems.
Motion capture comes in a number of flavors in virtual production. It can be used to enable actors to puppeteer virtual characters, either live for camera or to be later retargeted onto more complex post-rendered animation. Motion capture can also be used to transfer the movements of live-action crews to virtual cameras, enabling a hybrid approach to filmmaking which results in more realistic animation.
Familiarity with motion capture concepts and with technologies is critical for the motion capture supervisor. These include motion capture suits and encoding motion capture rigs. Also, an understanding of real-world physics, actor, and camera movement are all helpful. In many ways, motion capture is a modern version of rotoscoping used in traditionally animated 2D films where animation was traced over live-action footage. The end result is the same—more realistic natural movement.
Motion capture is one of the earliest forms of virtual production and of gaming technology, in use as early as the mid-1980s for video games like Prince of Persia and starting in the mid-1990s with CG animation for films
So there’s plenty of existing talent with experience in motion capture techniques on traditional shoots. And since motion capture is by definition captured live, the talent is generally familiar and comfortable with the live environment of an LED volume.
3. Real-time compositor
A real-time compositor is responsible for compositing various 2D and 3D elements live during production within a volume.
It’s easy to conceive of the contents on an LED wall as a single piece of footage or 3D content, but the image often consists of several separate layers that remain individually adjustable. That’s where the real-time compositor comes in.
Real-time compositing can be done in a variety of software packages, such as Nuke, DaVinci Resolve, or directly within the game engine itself such as Unreal Engine. So, depending on the specific tool being used for production, the software requirements will be specified as the needed skills. Typically a compositor will focus on one compositing package and become an expert in its operation vs. having a general compositing skill across several apps.
Talent pool
Compositors will typically be found in post-production/visual effects environments, where the importance is placed less on speed and more on overall compositing quality and realism. So some accommodations will be necessary to work within the live stage environment and its quicker pace. Compositors can also be found in live broadcast environments such as news and sports productions and will be more familiar with a live pace.
4. Systems administrator
A systems administrator is an IT professional overseeing areas such as network infrastructure, media servers, and spectrum management.
Because so much hardware and software must operate perfectly and live for an LED volume to function, this is a complex role. A movie set is already a vast collection of equipment—cameras, video monitoring, sound gear, communication support, etc—so adding in an LED volume and all of its systems can double or even triple the overall technical complexity.
Most large-scale virtual productions will deploy a variety of systems administrators to oversee all the different systems on a volume. A systems administrator might specialize in computer IT support, video I/O troubleshooting, asset management, spectrum management, storage/network support, etc. There’s plenty of work for each of these specialties on an LED volume.
Talent pool
Systems administrators for virtual production may be sourced from anywhere throughout the tech industry. There should be plenty of available talent with an emphasis in one or more of the needed areas or with enough experience to be retrained on the required hardware. Many of these areas need not directly interact often with the live-action film crew, so less of a live production comfort level can work for this role.
5. Systems technical director
A systems technical director handles the overall responsibility for the operation of all real-time specific hardware on a volume, including but not limited to LED walls, real-time render nodes, tracking systems, DMX lighting control, etc.
This role has some overlap with volume operators and systems administrators. This job includes careful attention to the selection, integration, and operation of all related components.
LED volumes comprise a bespoke integration of a number of different systems, some designed to operate in tandem, others completely hacked into submission. For example, the real-time engine is relatively new to working with tracking, LED, and camera systems. It’s only the current level of development that makes them all work together. Since it’s such a new integration, lots of care, maintenance, and customization are often required to keep everything running smoothly.
Talent pool
The systems technical director role can be a challenging position to source as it requires familiarity with many disparate systems. Dividing this role into several different team members can be the best solution. Vendors themselves can often provide experts to operate their equipment as well as train long-term.
6. LED engineer
An LED engineer is responsible for the operation and maintenance of the LED panels and video processors on a volume.
The panels within an LED volume are individual square modules, usually about 1.5’ x 1.5’. 100s or even 1000s of these panels are required, depending on the size and shape of an LED volume. Corollaries to the LED engineer can be a creative supervisor or screen manager.
Strong expertise with LED technology, video, and projection techniques are required for an LED engineer. Knowledge of cable management, color science, basic IT, and video signal flow technology, and testing is also important to have. Generally speaking, LED panels are more complex for the initial construction and setup—but less so once up and running, as they’re designed to operate in hostile environments for long periods.
LED engineers may work closely with the camera and lighting departments to handle any specific requirements such as desired brightness level, resolution, and frame rates.
Talent pool
Fortunately, LED panel technology has been around for a number of years in live concerts, theatrical/architectural projection, and live broadcasting. So, talent sourced from any of these areas will have few challenges adapting to the technical rigors of an LED volume. And since most of these other areas are typically live either in person or live broadcast, they’ll also already be accustomed to the live pace.
7. Video engineer
A video engineer is in charge of maintaining and routing video signals to and from sources and destinations on a volume.
The live content on an LED volume originates from a render node (typically a powerful desktop or rack-mounted computer), then it goes through video processing equipment which routes the signal to individual panels. This can get very complicated for large volumes with curved geometry. The video engineer helps ensure the signal flow is optimized.
Video engineers need to know all about existing video standards and best practices for calibrating and routing signals. HD-SDI, DisplayPort, HDMI, and other connectors come into the mix. The summing of video processors and multiple render nodes together can drive a combined final volume resolution of 80K or more.
Talent pool
Professionals with experience in live broadcasting for news, sports, live events shows, etc will be right at home at this position. The parts of the job which are LED wall and real-time engine specific can be quickly trained for experienced broadcast professionals.
Existing DITs and camera assistants may be interested in the video engineer role as sometimes there is a need to embed a camera person into the brain bar. This is also a job that can sometimes be handled somewhat remotely as more and more manufacturers are adding remote collaboration support to their hardware.
8. Virtual camera operator
A virtual camera operator is the physical operator of a virtual camera. The virtual camera can be controlled with an input device such as a mouse or a tablet, or a tracked physical item.
For example, in The Lion King, a traditional film crew used physical dollies, pan/tilt heads, Steadicam rigs, cranes, and drones to capture shots looking into the virtual world. So, a virtual camera operator might not interact directly with a real-time engine, but their physical movements are connected to a virtual camera.
Experience and enthusiasm with traditional cinematography are important for a successful virtual camera operator. The best virtual camera movements have believable real-world physicality. If the smoothness of a perfectly synchronized and linear camera movement is all that’s desired, this can be accomplished in the engine with keyframes vs. the need for a virtual camera operator.
Talent pool
Traditional camera operators and directors of photography can make great virtual camera operators, especially if the physical interfaces they use to drive the virtual cameras are set up in a similar configuration to the real-world camera equipment the operators are used to.
Game developers are also often solid virtual camera operators because they know how the engine works and know special tricks and workflows to enhance camera movement and realism. It’s a fun job and a nice tactile way to interact directly with virtual production.
9. Virtual production supervisor
The virtual production supervisor is the overall manager of the real-time visual effects efforts.
This role also acts as the liaison between the real-time crew, art department, VAD, physical production, visual effects, post-production, and any other crew whose work interacts with the virtual production effort.
Deep experience with major productions and virtual productions is key for this role. An understanding of every aspect of the pipeline and of the specific requirements of real-time for asset creation and performance are all important. Equal mastery of traditional filmmaking skills and of visual effects are also key.
Talent pool
Virtual production supervisors typically come from a visual effects supervisor or traditional producer background. This role requires a very special combination of technical mastery and people skills. Being able to translate often less technically-focused desires of the creating time into nuts and bolts tasks for the visual effects teams is a critical aspect of the virtual production supervisor. These are rare people but really know their stuff and can make or break a production.
10. Visual effects supervisor
A visual effects supervisor is responsible for the creative and technical aspects of visual effects.
This is not a completely new role, but it does change in a virtual production because real-time assets often overlap with post-production visual effects and the virtual production supervisor. More and more companies are using real-time tools to replace or supplement parts of their traditional render pipeline because of the faster performance and iteration while maintaining high-quality final renders.
Deep familiarity with the rendering process and the capabilities of the real-time engine are important for a visual effects supervisor. An ability to invert the traditional pipeline where most of the heavy lifting for shots occurs in post-production is important. Visual effects supervisors in a real-time pipeline will discover that the majority of visual development, asset creation, and finalization of assets now occurs in the pre-production phase.
Talent pool
Traditional visual effects supervisors often graduate from visual effects animators, compositors, coordinators, and other personnel involved in the visual effects pipeline. If they have a strong interest in virtual production and in real-time rendering, they’ll be able to make a smooth transition to a real-time visual effects supervisor.
11. Engine operator
An engine operator is responsible for maintaining and operating the real-time engine within the volume, and loading and operating assets.
This is a mission-critical role because the engine is what’s generating the live environment displayed on the screen in an LED volume. This position might be split among several operators, each responsible for different areas such as virtual camera position, lighting settings, playback of virtual interactive elements, etc.
To be an effective engine operator, a person should have deep knowledge of all the relevant aspects of real-time animation software. The software in use could be Unreal Engine, Unity, or one of the bespoke/vendor-specific engines such as ILM’s Helios rendering engine for its StageCraft virtual production pipeline.
The required knowledge could include operation of the virtual camera position, incoming camera tracking telemetry, color science and control, and additional programming for custom functionality.
Talent pool
Engine operators can come from any source that has a lot of experience with real-time such as game developers and game asset creators. Making the jump from the asynchronous pace of gaming development into a higher-pressure live environment can bring some challenges and adjustments. As the entire show rides on the screen content functionality, this role is for someone ready to work in a dynamic environment.
12. Data manager
A data manager manages on-set data, which can include camera telemetry, footage, tracking telemetry, real-time scene metadata, etc.
A data manager may also be referred to as a data wrangler. Depending on the complexity of the workflow and the production, there may be several different data managers working in tandem or independently.
In some cases, the role of the data manager may be an additional responsibility of another primary duty. For example, members of the camera crew may be responsible for data managing anything specific to shot recording. This is a role that demands fanatical levels of organization and a deep understanding of data structure and data verification techniques.
Talent pool
Data managers may come from a variety of backgrounds often starting with software developers and other computer-centric sources. Longtime crew coming from a film-centric background such as DITs and camera assistants can also transpose their meticulousness to data. Assistant editors looking to make the jump to set from the edit bay may also do well at data management.
It’s a critical role because you never want to lose any data, as it can lead to highly costly reshoots or worse if a shot or other piece of captured information is lost.
Fill the role
There’s currently a huge gap between the demand for virtual production specialists and the supply of personnel who can fill these roles. The live nature of LED wall/in-camera VFX and real-time game engines is what’s driving so much of this demand.
Because there are so many new areas of focus, there are new areas of opportunities. The roles we’ve described here are common examples, but there’s potential for many new roles as workflows and technologies demand new skills.
It’s an exciting time to be in this space both for the veteran looking to pivot to a new role and newcomers looking to get in on the ground floor. The sky’s the limit for virtual production and we’re just getting started.
Featured image courtesy of MGX Studio.
