Highlights
- Most of StereoD’s projects involve converting films to 3-D, a skillset that matched the needs of this project surprisingly well.
- StereoD developed proprietary tools to tackle this film, including a special “palette tool” for the challenging colorization process.
- The filmmakers were guided in their decision-making by the desire to portray the events as the soldiers themselves experienced them.
- Peter Jackson’s vast collection of WWI artifacts was an invaluable resource during production.
- Peter Jackson hopes that this documentary will inspire other archives to let filmmakers use their footage in a wider variety of projects.
They say you make every movie three times: once when you plan, once when you shoot, and once when you edit. In the case of Sir Peter Jackson’s recent World War I documentary, They Shall Not Grow Old, most of the construction happened in that final stage — editing. Due respect should be paid to the brave cameramen who lugged cumbersome equipment to the front lines during the “Great War,” of course, but Jackson’s film has taken their efforts to new heights. With some nigh-miraculous work by various visual effects artists, primarily the Deluxe company StereoD, They Shall Not Grow Old transports the viewer back in time to the Western Front, bringing the soldiers to life like never before.
We spoke with Mark Simone, a producer at StereoD who worked on the film, about what it took to resurrect the men who gave their lives in that conflict.
Background
StereoD specializes in 3D conversion, and their credits are impressive: everything from groundbreaking FX films (Avatar) to revisited classics (Titanic, Jurassic Park) to last year’s summer blockbusters (Ant-Man & the Wasp). Though StereoD had done some work with Weta Digital (Jackson’s special effects company) in the past, they were not expecting to be tapped for this project.
After all, “experience it in three dimensions” is not the first thought that comes to mind when someone says “World War I documentary.” Most documentaries about the “Great War,” as it was called, make heavy use of available footage, but varying frame rates, poor image quality, and the limited black-and-white palette constrain the amount of artistry that can go into them. You’d be hard pressed to be transported by them.
Jackson is quoted as saying, “[The soldiers] saw the war in color, they certainly didn’t see it in black and white. I wanted to reach through the fog of time and pull these men into the modern world, so they can regain their humanity once more.”
Put that way, the goal of the project wasn’t so different from the goal of 3D conversion. In both cases, the goal is to completely engross the audience—to make us feel like we’re there. StereoD’s expertise, then, made them the perfect team for the job.
In the very early stages of the process, Jackson’s post-production company, Park Road, sent some sample footage to StereoD, who began working on it. Producer Mark Simone says, “We did this test with [Peter Jackson] over a couple months. We returned the work to him and had very powerful feedback which was great.”
Mark then describes the 2-month period they waited to find out whether the project had been greenlit, and whether they had been awarded the honor of working on it. In October 2017, they finally heard: the project would move ahead, aiming for release around the 100-year anniversary of the Armistice, November 11, 2018. That meant that StereoD had just over a year to clean up all the footage Jackson wanted to use.
Fortunately, in the interim, they spent their time wisely. “We knew that it was going to be quite a lot of work to get this done,” Mark says. “So we spent that downtime between the task and the actual start of production preparing ourselves as best we could to produce volume.”
The process
StereoD developed a few proprietary tools in preparation for the project, such as a “palette tool” that allowed them to select and adjust colors across multiple shots.
But, for the most part, the team was already well equipped to handle the challenge. In fact, Mark explained that they approached the footage in much the same way as their normal stereo-conversion projects. “Stereo-conversion at its core is a visual effects process,” he said. “In that respect, this project felt very similar.” He outlined the basic workflow:
- receive digital files (scans of original footage) from Park Road
- clean up the picture by repairing tears, brightening the image, etc.
- add frames as needed; colorize
- add 3D.
The first step involved rotoscoping the images in every frame so that each shape is distinct. In stereo-conversion, isolating each part of the frame allows the VFX artist to move objects around as though they were in 3D space. Mark’s team uses a program called Silhouette to help them in the process, but there’s still a lot of what he calls “detail work.”
In some shots, parts of the image are “painted in,” so to speak, to allow the camera to fake 3D space. When a film is actually filmed in 3D, with a special 3D camera, the image is recorded simultaneously by two lenses which are set a few inches apart. Each lens captures a slightly different perspective on the object in the frame, so that, when the two images are overlaid, the viewer seems to see “around” the object.
Stereo-conversion mimics this by separating objects in the frame and painting in the parts that are hidden by foreground objects, like a rock or another character’s shoulder. At StereoD, visual effects artists do this day in and day out.
That expertise came in extremely handy in reconstructing footage shot more than one hundred years ago on primitive cameras. The original footage was not shot at 24 frames per second, but at anything from 10 to 18 frames per second.
It introduced mirroring and artifacting, which created edge clean-up work and frame clean-up work we’d have to do.
When the StereoD team inserted additional frames to make the motion look more natural, Mark says, “It introduced mirroring and artifacting, which created edge clean-up work and frame clean-up work we’d have to do.” But again, the team was used to handling that kind of thing. For them, cleaning up the edges of objects is an average Tuesday.
However, They Shall Not Grow Old posed a unique challenge. Most of the time, the team works with footage that needs detail-work, but is more or less complete. The footage they received from the Imperial War Museum via Park Road was torn, discolored, and sometimes incomplete. “There would be big gaps in motion,” Mark says. “For example, you’d be watching a soldier walk in on a field and you could tell that there were probably four or five frames of that soldier’s stride missing. So we would have to basically re-create those frames first before re-timing, so that we had a smooth 24 frames-per-second.”
The team has programs and tools that make the process easier, but it was still a challenge. Stretching one frame to two or three created issues that could only be solved by an artist with a finger on the mouse or a digital pencil in hand. When a large piece of the action was missing, the artist would essentially animate the objects moving across the screen, using the visible part of the action as a reference.
It sounds miraculous, even bizarre, but the VFX team at StereoD is comfortable with using digital tools to enrich the image and motion on the screen. “People start by wondering why a conversion company would be asked to do something like this,” Mark says. “After you talk it through, it does make sense.”
Color
Once the footage was smooth and faultless, the next step was colorization. Since the team had already rotoscoped every part of every frame, coloring was the process of deciding what colors to place where. If that sounds like paint-by-numbers, don’t be fooled: it was much, much more complicated.
Not only were there questions of authenticity (What color would that insignia have been?), there were questions of context (What would that fabric look like in this particular light?). “One of the challenges that was kind of a tricky thing to nail,” Mark says, “was the color of the British uniforms that was sort of this brownish-green color. Looking at it, you don’t really think that’s going to be hard to work with, but it was.”
Mark and the team traveled to New Zealand multiple times to meet with Peter Jackson for feedback and review. The meeting were more than cursory examinations, Mark says. “You would be able to not just review material with him but sit down and really crack into the shots and get an understanding of what he was looking for from a creative standpoint.”
Jackson made minute decisions on everything from the structure of the project to the color of a button on a specific soldier’s hat. That kind of hands-on direction was incredibly valuable for the team, who continued working on the project during the meetings. (StereoD is based in three locations, which allowed them to essentially work non-stop to meet their one-year deadline.)
Jackson is a huge history buff with an unbelievable wealth of knowledge on WWI. His grandfather fought in the war, which gives him a personal interest in the subject. This served as an inspiration for his earlier WWI short film Crossing the Line.
This passion, combined with his passion for collecting memorabilia, has resulted in an enormous private collection of materials and objects from the time period, ranging from ammunition to weaponry to uniforms and badges. “Anything you can think of, he has a physical version of it,” Mark says.
The collection was invaluable as a reference for the project. “The trips to New Zealand were so fruitful because we could sit in a room with him while he held the object and talked to us about it, and he could put it in different lighting scenarios take it outside if it was a sunny day or a cloudy day and say this is kind of how it should look.” In the case of the fickle color of the uniforms, having actual fabric to look at and test gave the StereoD team an enormous advantage in coloring.
“Beyond that,” Mark adds, “before production actually started, Peter himself went to the different locations where these battles were fought and did extensive photography that we used to either refer to or sometimes pull from the place in the actual shots. He would actually line up his camera to the shots we were working on and take photographs. That was unbelievable reference to have for something that happened over 100 years ago.”
Jackson also provided an on-staff historian who would answer the team’s questions. “[The historian] was available to us basically 24/7,” Mary says. “Any question about a particular object in the shot, he would help us get an answer, which was just incredibly valuable to us.”
Like most films, They Shall Not Grow Old evolved throughout production. This was certainly a case of creating an entire film in the editing suite. Jackson and his editor, Jabez Olssen, combed through hundreds of hours of footage and interviews, selected material, and passed it on to StereoD for processing.
As each shot was cleaned up and colorized, Jackson and Olssen had to decide which shots to place before and after the transformed footage. No film is ever really locked until the day it’s released in the theater, Mark admits, and that was especially true in this case. “It was definitely an evolving edit throughout the process.”
With so much control over the footage, the team needed some kind of standard to guide them in making decisions. Whether using the palette tool to adjust the color of a woolen jacket or adding sound effects to footage of a blasting howitzer, the VFX artists needed a way to answer the question, is this what we’re going for?
“The goal was to always be as true to the material as possible,” Mark explains. Jackson’s stated intention was to make the men who fought on the Western Front real for a 21st-century audience. “He wanted realism to be driven into every artist mind. Authenticity was paramount.”
Of course, Mark grants there were times they couldn’t determine the exact color of something – a jersey worn by a soldier underneath his uniform, for example. In that case, it came down to educated guesswork. “We had some freedom there, but it was always done within the context of what likely it would have been. So, it was always informed by reality, if you will.”
The fantastic attention to detail shows in the finished product. Watching the film in the theater (in 3D of course), I was flabbergasted by the footage of tanks rolling across the landscape. The color was so vibrant and the motion so clean, the film could have been shot yesterday.
Application
StereoD has been transforming films in post-production for years. Quite a lot of the work of stereo-conversion involves adding to individual shots, painting in edges of objects that weren’t captured by the camera. This latest project, however, really pushed the edges of what could be created with visual effects in post. The ubiquity of digital editing and effects means that any film can be completely reimaging after shooting has wrapped.
As a film is created in the editing suite, the image itself can be drastically altered, and the boundary between filming something live and animating it later is blurred. For filmmakers, this paves new avenues for storytelling. For documentarians, it reduces the limitations they may have with available footage. And for historians, it provides new ways to look at history.
At the London premiere of They Shall Not Grow Old, Peter Jackson asked other museums to consider running their collections of archival footage through the same process. That call to action has stuck in Mark’s mind. “It’s a tool and process that can be applied to any footage, really,” he says. “I think it’s exciting they think this is potentially something we can do not only for World War I footage, but for any sort of archival film from that era.” The early 20th century cameramen, who risked their lives hauling camera equipment to the muddy trenches on the front lines, would almost certainly be pleased to see how far the footage they worked so hard to capture has come.
