Editor’s note: Thanks to Reuben Evans and Mixing Light for sharing their expertise for this article and the Frame.io Workflow Guide.
It’s been 12 years since the RED One camera revolutionized the world of digital cinema, but still today many filmmakers have yet to fully integrate raw video into their workflow.
Despite the RED One’s success in proving that raw capture could be affordable, it also demonstrated the challenges of working with raw video in post-production. Sure, raw video capture is extremely powerful, but as the saying goes “with great power comes great responsibility.”
Luckily for us, the industry has evolved radically in the last decade, and recording and working with raw video has become much easier. As camera technology has progressed, raw capture has filtered down through the highest-end cameras and is now even available in some of the most entry-level models.
But even though shooting raw is now easier than ever, there is still a lot to consider before making the jump.
In today’s article, we’ll explore why raw is so amazing, how it can make your workflow more powerful, and what you need to know before choosing it for your next project.
Contents
Raw: An overview
As with nearly everything technical in this business, raw video is complicated. That means there are some common confusions about the concept among many filmmakers, so here’s a quick recap.
Raw vs RAW
Before we get too far in, let’s get something out of the way. The term “raw” is not an acronym, so it doesn’t need to be capitalized every time you write it.
That said, several major companies use the stylized “RAW” as part of their own proprietary branding. This has led many to believe that the all-caps spelling is the standard, which occasionally causes enough confusion that some think “raw” is different than “RAW.” The good news is that they mean the same thing, so don’t worry too much about it.
While neither term is is incorrect or inaccurate, the all-caps is just not grammatically necessary. So for this article, we’ll be using the term without all-caps.
What is raw?
To fully understand raw video, we need to take a look at how digital cameras work.
Imagine you are shooting a scene. First, light hits the object(s) you want to capture, bounces off the object(s), and finds its way into your camera’s lens. Then the lens focuses the light onto your camera’s digital image sensor.
Because digital cameras are, well, digital, and light is analog, the camera must interpret the light to make an image.
The general process goes something like this; the sensor measures the intensity of light hitting each photosite and translates that information into digital values (1s and 0s). These values are then handed off to an image processor, which uses complex mathematical functions to transform them into visually-displayable information. That’s how a standard image file is made.
But a raw image is the uninterpreted light information that your sensor captures, before it has transformed by the camera’s processor into an image file.
Debayering
A term you will hear a lot when discussing raw is “debayering.”
Put simply, debayering is the step in image production that gives color to your image. But for the curious, here’s a bit more detail on the process.
As you probably know, the sensor on your camera is made up of millions of tiny photosites (light sensors) that measure the intensity (brightness) of light. By themselves, these sensors have no way to detect the wavelength (color) of light, so they can only produce black and white images.
However, by putting a colored filter (red, green, or blue) on top of one of these photosites, the light brightness data can be interpolated into color information as well. How is that? Because if you put a colored filter over the top of the photosite, only that particular color light will be able to pass through. So a photosite, which as we said can only detect brightness, can be used to measure the brightness for just one color of light with this type of filter.
These filters are arranged in a specific pattern in order to generate full color coverage across the entire sensor. The most common of these filters is the Bayer pattern, named after scientist Bryce Bayer. To “debayer” an image is to use a mathematical function to extract color data from the light information of your camera’s image sensor.
One thing that might surprise you about the Bayer pattern is that there are two green filters for every red and blue one. This might seem confusing at first, but the reason for it is based on our biological vision system and how humans perceive color. In short, our eyes are more sensitive to green than they are to red and blue.
So why is this term talked about so much when discussing raw? Because since raw is unprocessed light information, the debayering process can be manipulated after the fact, unlike standard image files where a camera’s color processing is baked in. We’ll discuss the implications of this more deeply later, but for now just know that it makes raw video much more flexible.
Is raw the same as log?
No. Raw and log are not the same thing, though it’s easy to understand why they are sometimes confused.
Many video cameras now give you the option to shoot in a “log” or “flat” color space, which increases the dynamic range of your captured images, and gives you more flexibility in post.
While log video offers similar benefits to raw video, it is still interpreted by the camera, and results in a “normal” image file rather than pure light data from the sensor. White balance, noise reduction, and sharpening are all baked-in with log footage, so you can’t achieve the same level of flexibility as you can with raw.
The power of raw
As we just covered, raw video makes it unnecessary for cameras to do all the hard number-crunching of image processing by themselves. For raw, all the camera needs to do is accurately measure the light hitting the sensor, and record that data. No internal processing necessary (except for data compression, but that does not interpret the sensor light data, it only makes it smaller).
Of course, you still have to process that light information if you want to produce a usable image, but as it turns out, post-production computers and software are very capable in this regard. This is where the magic of raw really begins.
Maximum quality and complete flexibility
Generally speaking, raw video represents the highest quality image you can get from a camera. But this isn’t just dependent on the camera, it has a lot to do with what comes after those files are transferred to a computer.
Since raw image capture is only dependent on the sensor’s light data, it doesn’t experience the same sorts of data loss that occurs when non-raw images pass through a camera’s internal image processors. Instead, raw processing happens on the workstation side of things, where the full extent of the light data can be analyzed and processed to retain maximum quality without degradation.
But raw information doesn’t just let you squeeze out every last bit of quality from an image, it also gives you maximum image flexibility. Color mapping, noise reduction, sharpening, and various other image processing techniques can all be adjusted in any number of ways. That’s why so many filmmakers want to shoot in raw—it provides the most freedom to experiment and fine-tune the look of a film after shooting has wrapped.
Practical advantages to shooting raw
Given the very nature of raw, certain aspects of shooting matter less on set. White balance and ISO, for example, are completely editable in many raw formats, so accidental color casts or exposure issues can be easily corrected in post. That lets you spend less time fiddling with camera and light settings, and more time focusing on the performance/substance of what your shooting.
Perhaps the filmmakers who appreciate raw capture the most are those who learn what raw can do on set and in the color suite. The more familiar you are with raw, the better you will know its capabilities and limitations, which ultimately will allow you to make spur-of-the-moment changes during production, knowing you can appropriately compensate for these changes in post.
Green Screen and VFX Capture
Another scenario where raw capture proves highly advantageous is with shots intended for visual effects work. Because most non-raw formats use chroma subsampling to reduce capture bitrates, the resulting footage can cause problems for VFX work, especially on green screens.
But raw never uses chroma subsampling, because the debayering process is not baked into the file. Thus, raw video can be used to pull a much smoother key around the edges than most standard footage. Additionally, correcting the “spill” (the amount of green reflected on your subject) is simpler because of raw video’s color control.
You don’t even need to carry raw through your entire workflow to gain this advantage with green screen work. For example, if you shot 6K raw footage, but it proves too resource intensive for a particular process, you can transcode to a codec without chroma subsampling (like 4K ProRes 4444), which will give you a much cleaner key without the same computational requirements as raw.
And of course, when it comes to complex VFX compositing, higher original image quality results in more precise and realistic results.
A Hidden Benefit: New Color Science
One lesser-known but incredible benefit of shooting in some raw formats is that old footage can be made to look like new with updated color science.
As manufacturers refine their methods for interpreting colors and image quality, and release those updates in new firmware, those newer mathematical processes can be used on the light data of old raw footage.
That means that an image you shoot today (or years ago) can actually improve in appearance over time. Colors that were previously out-of-gamut can be made renderable with new color science, and exposure can be made to roll off more smoothly across an image.
Naturally, this feature of raw footage has tremendous implications for archival footage. It also makes adapting older projects to new delivery formats much easier (assuming you export from a project’s raw state). For example, if you shot 4K video destined for downscaling to HD broadcast with Rec. 709, but you want to deliver it in UHD HDR with Rec. 2020 one day, successfully regrading the project will be much more achievable with raw.
Raw and your workflow
In spite of the many benefits of raw, there are two major downsides: huge file sizes and extra processing requirements.
With very few exceptions, raw video is almost always larger than non-raw video. That means you’ll need more memory cards, more hard drives, and more time to copy, process and transcode files. Of course, the exact size depends on the type of raw we’re talking about, but some raw formats require as much as 5 times the storage space and bandwidth as non-raw footage.
Raw can also be very demanding for your post-production hardware. While computer workstations are much more powerful than the processors on the inside of cameras, working with and manipulating raw files is still a computationally heavy task. Generally speaking, for a system to be able to handle high-resolution raw footage smoothly, you’ll need a beefy CPU and dedicated GPU (and maybe a fair amount of patience).
Choosing raw
Thankfully, raw video comes in many different flavors that make working with it in post-production easier (pure uncompressed raw footage would be an absolute nightmare, so no one uses it). Unfortunately, it’s not always easy to compare them directly, but here’s a quick rundown of some of the most common varieties:
- REDCODE RAW: Initially developed for the RED One, REDCODE cemented RED as a major player in the digital cinema space. This format allows full control of an image’s metadata (white balance, ISO, etc), has a wide range of compression options, and is compatible across all major NLEs and post-production applications.
- ARRIRAW: Of course RED is not alone in the digital cinema space, and ARRI’s raw format is a marvel of cinematic technology. Like REDCODE, ARRIRAW provides total control over an image’s metadata, while focusing on uncompromising image quality and doubling down on compatibility with high-end systems and software, though at the expense of file sizes.
- ProRes RAW: A relative newcomer to the digital cinema world, ProRes RAW was created by Apple as a solution to the complexity and performance burden of other raw formats. ProRes RAW offers full access to the image quality benefits of raw, while retaining the playback performance of standard non-raw ProRes in NLEs. However, ProRes RAW does not offer the standard color controls for ISO and white balance many have become accustomed to in other raw workflows.
- Blackmagic RAW: Following hot on the heels of ProRes RAW, Blackmagic Design released their own format barely one year ago. Unlike ProRes RAW, Blackmagic RAW allows for more robust control over an image’s metadata in post, and works across more applications. However, it does partially debayer images in camera, which means old footage will not benefit from future refinements to their color science.
Other manufacturers like Sony, Canon, and Panasonic also have their own raw formats, but you may encounter them less often.
Keep in mind, many raw formats deliver smoother playback performance by compressing the data stream, which can have some effect on image quality. It’s quite possible to have very highly-compressed raw footage that looks worse than standard footage captured in a high-quality codec, though for most typical compression ratios the image will be very good.
It’s worth mentioning here that your perception of raw file sizes will be largely determined by the codecs you are most familiar with already. If you come from a ProRes or DNx workflow, some compressed raw formats will seem to be about the same size or only slightly larger. But if you normally shoot with an AVC-based codec, you should expect much larger file sizes than you are used to, especially if you’re also making the jump from 1080p to 4K.
Shooting raw solo
A major determining factor of when/if you should use raw for a project is the workflow’s scale.
For solo shooters juggling many hats on set, like running audio, exposure, focus, power, stabilization, and more, raw’s image quality and flexibility can be very appealing. Being able to compensate for onset imperfections or mistakes is a godsend in many circumstances when you don’t have enough time or hands to get everything right in the moment. You will still need to light your scenes correctly (raw isn’t magic), but it can cover a multitude of sins.
However, remember that while raw gives you more flexibility to patch up issues, adjusting all those settings takes longer than standard color correction, and the extra processing overhead can delay client delivery. That might be fine for some projects, but if you have a tight turnaround, raw might not be the best choice.
Working with raw
Truly, raw is a miracle of modern technology we should be thankful for, but if you don’t want to spend the entire day just being thankful and actually want to get work done with it, you’re going to need a powerful system.
Depending on the raw format you choose and your computer’s configuration, you can leverage the power of a dedicated GPU to speed things along. Some GPUs are designed to provide real-time playback of certain raw formats, like Nvidia’s Turing architecture does for RED’s 8K raw formats. These types of cards offer incredible performance, but at eye-watering prices.
But you will also need to make sure the other components on your computer are also up to par, especially your CPU. Not all applications will allow your GPU to help with playback, so the CPU needs to be able to handle playback sufficiently when necessary. And as with any color or editing system, you’ll need a large amount of ram and storage to prevent bottlenecks.
That said, you’re going to need a lot more than a beefy computer to work with raw. It’s important to consider your entire image pipeline from acquisition to post, when choosing a camera and codec for your next production.
The cost of raw
Raw data is huge. There’s just no getting around that. Even at decent compression ratios, most raw formats will burn through multiple gigabytes per minute.
Obviously, this means you’ll have increased costs for recording media, on-set storage, hot storage for post, backup storage, longer file transfers, slower turn around time, and so on. Your workflow can definitely gain a lot from raw, but it comes at a very real cost.
These are important considerations when evaluating the scope of your project. The good thing is that those are quantifiable costs. You will be able to calculate how much the extra hardware, storage, and time will cost you, and if your clients pay that much for your time and talent.
And it’s not as bad as it once was. Much of raw’s cost pain has been eased by recent technological developments, like ultra-fast SSDs at very reasonable prices, and hard drive costs hitting rock bottom. That means if you combine the efficiency of the best raw formats with new I/O connections, and a smart mix of storage hardware, raw can yield a very real return on investment. But, you’ll need to do your own number-crunching to see what makes the most sense and invest accordingly.
Conclusion
Simply put, raw provides the highest quality and most flexible option for digital image capture. The ability to make vast changes to color, exposure, and many other aspects of an image without compromising quality make it an obvious choice for many projects.
Of course, there’s always a tradeoff to such incredible quality and flexibility. Raw requires a significant investment of time and financial resources to be used properly. That means adopting raw into your workflow requires well thought out decisions about every step of your image pipeline, from the camera all the way to your client, and these decisions should involve everyone who has to shoot with it (DPs), the people who have to handle it (DITs), and the people who have to work with it (editors, colorists, and VFX artists).
If everyone’s on board, and you’ve made the appropriate investments to run things smoothly, then raw will open up a whole new world of possibilities for your work.
