Archiving Your Video Project: What Do You Do When You’re Done?
When you finish a project, it’s often unclear what you’re supposed to do with all the assets you’ve acquired in the process. If you’re a producer or post producer, you’re likely to have questions about what to do with this digital media—beyond the assumption that someone should be keeping a copy somewhere.
Having a solid archival strategy is crucial, and it doesn’t matter whether you’re an indie filmmaker or a Hollywood studio. Even large motion picture studios can lose assets, often catastrophically, as was the case in the Universal Studios fire. So it’s vital you have a plan for archiving your projects to ensure you can access them in the future.
We’re going to focus more on digital archiving here, since film archiving is a separate and increasingly uncommon issue. That said, today’s film stock is incredibly stable—especially black and white—as long as it’s stored in specific humidity and temperature conditions. So your biggest concern with film storage is what to do with the final print, and there are a variety of vendors that will store film stock for you in the correct conditions.
Digital archiving has its own complexities. But as the vast majority of projects now are captured, post produced, and delivered digitally, let’s take some time to make sure you’re taking the best care of your media.
Deal memo or contract
The first step in the archive process is to make sure that archiving terms are spelled out in every contract. If you’re contracting to produce or do post on a feature, or working for a brand on a commercial, you need a paragraph in your contract that spells out precisely what you’re agreeing to in terms of storing the project over the long term.
If you skip this part, you’re risking client assumptions about the storage of media that can potentially ruin your relationship. I have seen clients come back two years later asking for new changes on a project and be furious that the post house didn’t have the project “live” on their system.
Conversely, I’ve also seen clients who were furious to realize that a production company had kept footage six months after delivery that they thought should have already been destroyed for intellectual property reasons. There’s no industry standard for who’s responsible for what, so spell it out in your deal memo or contract.
“I’ve also seen clients who were furious to realize that a production company had kept footage six months after delivery.”
It can be helpful to have a default set of terms you include as boilerplate. Even if the client decides that they want something different, it’ll open a conversation at the beginning of the project that hopefully leads to clarity about what everyone’s expectations are.
Running hot and cold
For many years my default language was something roughly like “We will keep your project ‘hot’ on our system for 60 days after delivery for rapid changes that might need to be made. We’ll then take your project ‘cold,’ taking it off our systems but keeping it on a hard drive in our archive for an additional year. After that we don’t guarantee the backup will boot or that we will keep it.”
This language was rarely questioned, and when it was, it was helpful; it forced the client to think about how long they assumed things would be available, and helped us avoid notes for changes coming in long after delivery.
It also had the added bonus of helping the post engineering team have clarity about when to move media off the server and local drives into a backup. If they needed space, anything that was delivered more than 60 days ago was free to move.
Timeline from delivery
The first element in your archiving workflow is the delivery date. This is the date that the project is handed over to the network, distributor, streamer, or client and is off your books. Delivery is often the point where most of the energy of a company is focused, and it’s relatively common for things to dissipate quickly after delivery when the main team moves on to other projects.
You never want to archive too quickly after delivery since we all know that the moment you archive a project will be the moment your client comes back with “just one more thing.” You’ll get to know your clients and how many notes trickle in, but a good benchmark is putting 30 or 60 days in your contract, then archiving in about 90 days.
It’s up to you, but you might want to build in an additional 30 ‘hot’ days beyond the time specified in your contract just for those “seriously, this is the last one” notes.
The other major question that bedevils most of us is how many copies you need to keep to feel reasonably confident that you can recover the project when you need it. The acronym most archivists use is LOCKSS, for “lots of copies keep stuff safe.” The more copies you can reasonably afford, the better. In motion pictures, that often manifests as at least three copies, in at least two different storage formats, to be confident you’ll be able to recover the project if you need to.
For short-term backup, if you are moving footage off of a ‘hot’ storage medium and onto something that’s only expected to last a year or two, then that storage can be backed up to the original project hard drives. But when it comes time to the real archive, things get trickier.
What to archive
Before you embark on that full archive, you need to determine precisely what elements you want to archive for any given project. The more data you store, the more expensive it’s going to be, whether it’s on physical media or in the cloud. Hard drives cost money, real estate costs money, cloud storage costs money. All of those elements together make it vital that you think through precisely what you do and don’t want to store long term.
Picture elements, final deliveries, and dailies
At a bare minimum, you need to store your final master export file. This is likely either going to be an Apple ProRes 4444 LogC file or a DNxHR 4444 LogC file, or your DCP if your project had a theatrical release. These files have the benefit of being widely supported formats with broad industry adoption across many platforms. This makes them far more likely to be recoverable in the future even if technology changes dramatically.
If you made a DCP and a “video master” in ProRes or DNx, archiving the DCP as well is often a very wise decision, but an argument could be made that you only really need either your DCP or your ProRes master, but not both. A DCP is generally under 500GB, so if you took the time to master directly to XYZ for your DCP and do a separate RGB master for video, it’s likely worth the effort to archive both. But if you made your DCP just by using a LUT to convert from ProRes, there isn’t much need to keep your DCP, since you could always recreate it the same way in the future.
“Some will want to save every single thing that was created—and that’s fine as long as they are willing to pay for it.”
So you need to discuss with your client, director, or team precisely how many of the dailies of the project will be saved. Some will want to save every single thing that was created—and that’s fine as long as they are willing to pay for it—but the expenses go up fast once you start looking at storage for the massive volume of data that entails. A 4k video master and DCP together will generally still be under 1TB. A feature film might shoot 20-30TB of dailies media, which can increase archiving costs 20-30 times.
Because of this, the vast majority of projects back up either just the final ProRes or DNx master, or the final master and a consolidated version of the edit with the assets needed to reconstruct the edit, with handles of 12 or 24 frames. In addition it’s often a good idea to archive the stems available in the music mix, which typically are a dialogue stem and M&E stem (music and effects).
Archiving these two elements along with a consolidated edit and your stems, can be tremendously helpful for future proofing your production. The best examples of this come from foreign releases and music licensing.
Let’s say for some reason in ten years one of the lead actors in the independent film you are archiving gets cast in a major studio franchise, or gets elected to Italian parliament. With that new exposure, there’s now an opportunity to license your movie abroad. This often includes the requirement to do a local dub of the movie (this is actually a legal requirement in Italy, hence the importance of preserving your stems if that’s how your lead actor gains fame).
If you’ve kept the dialogue and M&E stems separate, that is much, much easier to do to recoup that additional revenue. If you only have your final mix with dialogue and music mixed together, dubs get much more complicated.
The reasons for keeping the edit as a consolidated, media-managed project with handles are similar. Music licensing is continually shifting. More than once, I’ve been brought onto a project that was long since delivered to change out the music based on a new license or a change in status. And this is much easier if you can tweak the edit a bit here or there to match, shifting things around while preserving the overall runtime.
“If in the future you lose the license on a shot and need a new one, it’ll be much easier if you have room in the shots around it.”
The same is true for documentary work and archival footage licensing; if in the future you lose the license on a shot and need a new one, it’ll be much easier if you have room in the shots around it to shuffle things.
If you care about the future of the movie both as originally released, but also as an asset open to future licensing, you should ensure there are audio stems and a lightly editable version of the film available. The rest of the dailies, however, generally are not preserved.
Another item that often gets bundled into the archive is the marketing collateral for a project. This might not seem especially relevant on short form projects, but if there are instagram cutdowns, social media cards, or other marketing material, it’s a good idea to keep that all together with the final asset. Doubly so for feature films or television projects.
Don’t assume that the marketing or advertising agency is going to be doing this archiving themselves. It’s unfortunate but true that their archiving systems are generally not as robust as those of a filmmaker. So if the project is brought back to life in the future, it’s good to have access to all those marketing materials, too. If you leave it to the agency there’s no guarantee they’ll have these materials.
The other factor to think about is how many of the creation elements you want to archive along with the film. Creation elements are primarily the project files that you used for creating the project, but can occasionally also involve finding a way to archive the software used to make those projects.
The bare minimum you should archive is whatever final projects files were used in finishing, such as your audio projects, your editing projects and bins, and your color grading project files. Some applications have this baked in. For example, Premiere Pro and Resolve have project management tools that backup your project files and collect the media you need to keep working on the project, which can be exceptionally useful.
While it’s becoming less practical as we lean more heavily on the benefits of Cloud-based software, you might also consider archiving the installer file for the software that you did your finishing in—like DaVinci Resolve—if it’s available.
You might also want to consider exporting an EDL (edit decision list) of your edit alongside the native project files. This export function is available in most NLEs, and allows you to keep a record of the basic editing parameters of your timeline. It’s a format that’s often used to package edits to hand off to colorists.
As well as allowing you to open old edits with current software (regardless of the age gap), EDL is a cross-platform standard that just about every NLE can import. So even if you’ve lost access to the original software, you’ll be able to rebuild the basic edit in your application of choice.
Physical backup solutions
Now that we have our elements (our final deliverables, audio stems, consolidated media managed project files, and marketing assets) all together, it’s time to put them somewhere for long term ‘cold’ storage. This generally falls into one of two categories; physical media or online storage—which is still physical media, just far away.
The physical media tool that we rely on for archiving is LTO, or Linear Tape-Open. It’s the format of choice because the cost/TB ratio is extremely low, as low as $5 per terabyte if the data is compressed for storage.
To recover the data, you’ll need to load the tape into an LTO drive so that it can recover the data from the tape into a file that a computer can then read. These drives are expensive, and it’s a very slow storage medium compared to conventional hard drives, but LTO is a long-lasting format with a strong track record.
It also benefits from not being targeted at the film industry. LTO is a public facing format that was designed for massive industrial backup, and it’s widely used in businesses with strict legal requirements for document backup, like law firms and medical organizations. And that’s a good thing, since it means it’s much, much cheaper than it would be if it was film industry-specific.
You can get an LTO read/write solution for under $5000, though if you’re only archiving one or two projects a year, it’s better to find a vendor in your area that can do the backup for you for a small fee. Tapes are under $100, so making multiple copies is cost effective.
Of course, when you need to recover the project, you’ll need to get your hands on an LTO system again. It’s worth noting here that new LTO drives can read older formats, but newer tapes cannot be used in older drives. A search in your area will likely turn up some vendors, even if you aren’t in a production hub like New York or LA.
Because of its wide adoption in industry, it’s generally considered safe to trust in the long term viability of LTO. Given that it’s still being updated and supported, with the LTO-9 standard ratified in January 2022, it’s fair to say it’s not going anywhere for the foreseeable future.
For many, the archive plan for “three copies across two formats” means making two LTO backups and one online backup. You take those two LTO backups and store them separately, in two fireproof safes as far apart as convenient (one in your house, one in your office), and then you can move on to your online backup.
Online backup solutions
The other solution that is part of many filmmakers’ archive strategy is an online backup solution. These come from cloud vendors that offer an online archive for your projects.
Some might rely on tools like Microsoft OneDrive, Dropbox, Box, and Google Drive, which are all incredibly popular as ‘hot’ working tools. But my view is that they’re not designed for and shouldn’t be used for archiving or long term storage. All of them have had outages and lost media at some point, and they prioritize your most frequently accessed media while moving your less frequently used media to colder versions of storage. You don’t get to choose.
My recommendation is to consider online platforms that are better suited to this task, like Amazon Glacier or Frame.io’s Archive, which have been designed for stable long-term storage of your data.
The key thing to bear in mind is that media you store in ‘cold’ services won’t be available quickly—you’ll need to put in a request and wait for assets to be retrieved, which can take a few hours. But it’s worth noting that Frame.io Archive continues to store file proxies, comments, and any presentation and review links in ‘hot’ storage, so you’ll still be able to instantly access and work with this data even after archiving the bulk of your assets.
Both of these services have flexible storage options that can be adapted to your particular use cases. Glacier storage is around $3.50 per terabyte per month (not including retrieval costs), and your Frame.io paid plan already has an archive allowance built in, with more storage added in 1TB blocks when you need it.
It’s fair to say that Glacier can be a complicated solution that’s better suited to developers and enterprises with in-house AWS expertise. There are third party apps like Cloudberry, which are designed to make Amazon storage management a little easier, but given that it takes just three clicks to archive a project on Frame.io, I prefer the path that doesn’t involve a steep learning curve.
It can also be incredibly slow to upload your projects to Glacier, since even with fast internet 1TB can take a long time. But to be fair, Amazon has an interesting solution to this, called the Snowball. It’s a hard drive that is designed for shipping with an eInk shipping label. They ship it to you, you load it up, ship it back, and they archive it. They also have an equivalent semi-trailer for major corporate backups, but hopefully most filmmakers haven’t acquired a semi-truck worth of data during shooting.
With Frame.io Archive, it’s a lot simpler. Depending on how you use it, much of your data will already be in the cloud, so it’s just a case of choosing your project and selecting the Archive Project option.
It’s true that digital archiving is more complicated than just taking a film print and sticking it in a salt mine in Kansas, but that doesn’t mean you should ignore it. Be decisive about what you want to archive. Get a few LTO tapes and a cloud archive. Spread those LTO tapes out in the world in fireproof document safes and keep your cloud storage subscription up to date, and you’ll protect your work far into the future.