With a little planning before the shoot, you can insert a screen onto a monitor or device convincingly without complications.
One key is how the screen is shot. This is where it’s easiest to get it wrong. This tutorial shows how easily a realistic yet legible screen can look more or less perfect in After Effects.
Of course, most shots are not that straightforward. There are a few pre-production tips that can help you set yourself up for success on the shoot day. In order to implement these, you really need to understand what will happen after the shoot, so let’s start with post-production.
Know your tracker
For this article, we assume something in the shot—either the camera or the device itself—moves, but that nothing obscures the screen. A mobile device with a touch screen brings up a whole separate set of concerns that will have to be part of a separate lesson.
It can be invaluable to gain confidence in your knowledge of what it will take to track a shot. You can then decide on set whether you’re getting shots that will work in post. The main concerns that tend to come up regard reflections and tracking markers. Spoiler alert: I love reflections and hate tracking markers—but more about that in a moment.
Imagineer’s mocha is a powerful tool for screen replacements. In a scenario like the one depicted in the video, where you have a black screen with a light-colored bezel, you basically never need to add tracking markers.
In fact, I even find that the free tool that comes with After Effects, the less fully-featured mochaAE, can very often track a screen perfectly without any need for tracking markers. So my number one tip going in is to be confident enough with your tracker that you can tell the DP not to use them.
Avoid tracking markers unless you must have them
If there’s contrast around the edges—and there usually is—you don’t need tracking markers. A planar tracker—Imagineer’s mocha—does an ideal job of picking up a flat surface and offering you a solid 2D track.
What does “contrast around the edges” mean? In the example, it’s difficult to see where the black screen ends and the black bezel around it begins. However it’s very easy to pick out the light outline outside of that. Even if a perfect contrasting boundary didn’t frame the monitor, everything else behind it is also considerably lighter. Given a well-defined edge, tracking markers waste your time and add effort.
Reflections are good and bad
That’s not my only unconventional suggestion. If you light for them, reflections will sell the shot. A reflectionless screen looks artificial, even to the untrained viewer who may not be able to identify why. Granted, on Avatar, we designed “screens” for the far future that don’t reflect, but in today’s world, we expect them.
Now, it’s not easy to keep the lighting or camera rigs out of the reflection, and in many cases it’s downright impossible. The example in the video is captured with natural window light as the key, and indirect fill lighting. The screen is at an angle where it faces neither the light nor the camera. That’s perfect.
But as you’re well aware we don’t live in a perfect world. If it’s impossible to set up those kind of conditions, you need to take the same precautions as you would with any reflective surface under heavy lighting. Try to avoid angles that are going to reflect a light directly. This is much, much better than some of the hokey solutions I’ve seen like covering the screen with green paper—more on that in a moment.
One final note on reflections and mochaAE; if they’re too strong, reflections will mess up the planar tracker. For the most part you don’t have to worry about this, but if a keylight cuts across the screen, the tracker will probably pick it up. Again, just be sensible, not paranoid.
Dulling spray
Some DP’s use dulling spray to give a blurry, matte finish to the screen being replaced. This isn’t a terrible idea, although once the screen is tracked it’s also possible to soften the reflections in post with a tracked adjustment layer. A better idea is to use less light when shooting a reflective surface.
The whole point is that some impression of ambient light on the screen is generally much more realistic and interesting than none at all. If you get them in camera, they are perfectly realistic for free.
Should the screen be illuminated?
In a straightforward case like this, with no hand interacting with the screen, keeping the screen turned off lets you add everything in post. This is the most natural approach, and it lets you dial in the reflections.
Nevertheless, I recommend the screen be off unless anyone is getting close enough to it that it would reflect light onto a hand. That gets us back into touchscreens, which isn’t really the focus here.
Use preproduction planning to avoid worst-case scenarios
Many productions don’t have a separate lighting day when it’s easiest to test this stuff. Again, this is where it helps to know your tools. Armed with a laptop, you can shoot a quick test on the day replicating what you’re trying to shoot. Even shooting it with a camera phone is probably indicative. If possible, use a stand-in and whatever action occurs in the actual scene.
Transfer the shot to After Effects, fire up mochaAE (or full mocha if you have it) and try tracking and replacing the screen. Comp in your replacement—even just a checkerboard—using Screen transfer mode.
This is a great example of a visual effect that anyone can learn to do. True, Imagineer tools have a UI that is unlike what you’re used to in Adobe. There are excellent tutorials available to quickly get up to speed with a basic screen track. Within an hour or two, or less, you can learn and practice the skills needed to pull off a shot like this.
To see and learn more about compositing and tracking, or just about After Effects, go here.
