The Hidden Villain Behind Your NAS Storage Woes

If you have physical media, and you like making backups of those media to your own personal server using a NAS, then you may have noticed something a little odd when it comes to file sizes.

Specifically, when you encode older, classic movies from DVD, Blu-ray (or even VHS), the final compressed file can be significantly larger than a more modern movie, so what gives?

Film Grain: The Beauty That Breaks Codecs

The answer is pretty simple—film grain.

Dusty scratched and scanned old film texture.

According to the Society of Motion Picture and Television Engineers (SMPTE), film grain is inherently random, and “no frame has the same grain” or size of imperfection—meaning codecs can’t easily find repeating patterns to shrink.

That’s as opposed to footage that’s been captured digitally, because there’s no film grain at all. It’s relatively predictable, and many of the pixels in one frame don’t change for several frames, allowing for efficient compression. Film is chaotic at the pixel level. Every frame contains millions of unique variations that resist the shortcuts codecs rely on to work.

Film grain isn’t “noise” in the sense that we use it in digital video. It’s a physical record of how the photon struck the tiny silver crystals in the film stock. That unique frame-to-frame pattern of grain is a major part of the organic, textured feel movies and shows created with film have.

The codecs we use look for spatial (within frame) and temporal (across time) redundancies as part of the compression strategy. As this fascinating Medium article on video compression explains, “video compression depends a lot on spatial and temporal redundancy…film grain is quite the opposite due to randomness.”

The Codec and Container Problem

Obviously, this is a known problem, and you can do something about it at the encoding phase of making backups. There are three main options for encoding video these days, and they have different levels of success when encoding video with natural film grain.

H.264 (AVC): This codec is getting on in age these days, but it has almost universal support on even the cheapest devices, with dedicated hardware decoding practically guaranteed. So it remains immensely popular and common. However, it’s pretty inefficient when it comes to preserving film grain in a compressed file. So expect bloated backups.
H.265 (HEVC): This is the successor to the above codec, and assuming your device supports it, it’s likely the codec you’re watching on major streaming services. HEVC offers significantly smaller file sizes for the same quality compared to AVC, but the grain itself might not be adequately sharp and preserved in the final product.
AV1: The latest darling in mainstream video compression, with growing hardware compression and decompression support, the AV1 has specific technology built in to handle grain. According to a paper by Andrey Norkin and Neil Birkbeck, the AV1 grain-synthesis tool can cut bitrates by up to 50 percent on grain-heavy footage.

According to Netflix, it’s already using AV1 and its grain-synthesis feature to restore classic films to their proper state on the service.

Balancing Quality and Space on Your NAS

Knowing what you do about how grain impacts compression, you’ve probably already thought of some ways to combat it, but let’s talk about some explicit strategies here:

If you use something like Handbrake or FFmpeg, you’ll want to look for grain-tuned presets or grain-preserving options.
You can also try using two-pass encoding, which is something I always do anyway. Compressing video in two passes usually results in a smaller, better-quality file. So if you have the time for it, go with this option.
You can also choose to filter out the grain, just like James Cameron controversially did for the 4K release of Aliens. However, this can lead to losing the look and feel of the movie. So consider using a milder grain filter, or, of course, the grain-synthesis option in AV1.

AV1’s grain synthesis works by removing all the grain before compression, but it saves metadata on what the grain looked like to the file. Then, when you play it back, the grain is synthesized in real time based on that metadata. This preserved the look of the movie without the bloated size.

Preserving Film Without Filling Your Drives

Film grain isn’t a flaw, and it shouldn’t be removed from a movie. At least that’s my (and many other cinephile’s) opinion. If you’re happy to remove all the grain, that’s your personal preference, but you don’t have to spend a fortune on storage to keep it.

By using smart encoding, or the latest codec features to preserve grain, you can have your grainy cake and eat it. You can even check how successful you were by comparing the compressed video with the original DVD or Blu-ray. If the grain still looks the same, you’ve done it!