The Foundry Un-Rolls Your Shutter

The Foundry have released a $500 plug-in for Nuke and After Effects that attempts to remove rolling shutter artifacting, AKA “Jell-o cam,” from CMOS footage (RED One, Canon HV40, Canon 5D Mark II, etc.). It’s based on a technology demo that they showed at NAB earlier this year.

I’ve tried it, and it works—sometimes. The Foundry are well known for being leaders in the motion estimation field, and they have harnessed their unique experience in this area to attempt the impossible: a per-pixel reconstruction of every part of the frame, where it “would have been” if the shutter had been global open-close instead of read out a line at a time.

When it works, it’s brilliant. But the more motion you have in the frame, the more likely this plug-in, mighty though it may be, will get confused. Unfortunately, when this happens, parts of the image turn to scrambled eggs.

It’s a very similar problem to re-timing 30p footage to 24p actually—in fact, I wish The Foundry had added an option for frame rate conversion to this plug-in. Although, in fairness, I would only rarely use it.

Folks are always asking me about converting 30p to 24. I responded in a thread on the Rebel Café:

…The more motion you have, the more likely it is that any optical flow re-timing system is going to encounter problems.

Not even getting into the combat shots, here’s the Apple Compressor method referenced in Philip’s tutorial failing on one of the more sedate shots in After The Subway:

Now I’m not saying that you won’t occasionally see results from 30-to-24p conversions that look good. The technology is amazing. But while it can work often, it will fail often. And that’s not a workflow. It’s finger-crossing.

On a more subtle note, I don’t think it’s acceptable that every frame of a film should be a computer’s best guess as to what happened between captured frames. The magic of filmmaking comes in part from capturing and revealing a narrow, selective slice of something resonant that happened in front of the lens. When you do these motion-interpolated frame rate conversions, you invite a clever computer algorithm to replace your artfully crafted sliver of reality with a best-guess. I feel (and feel free to disagree, I won’t bother arguing) that this artificiality accumulates to create a feeling of unphotographic plasticness. Screw that. Didn’t you select the 5D because you wanted emotionally resonant imagery? You’d be better off with a true 24p video camera that works with you rather than against you, even if it doesn’t give you the convenient crutch of shallow DOF.

To be crystal clear, that’s Apple Compresser failing to properly estimate the motion in one particular frame when trying to convert from 30p to 24p. Nothing to do with The Foundry or their new plug-in, which actually tackled that same shot quite well.

The Foundry knows that they’ve made a tool to help us limp along while camera manufacturers sort out this CMOS issue. Like any crutch, I wouldn’t plan on leaning too hard on it—but kudos to The Foundry for attacking this problem head on, and making a product out of a technology demo in record time.


on 2009-08-06 21:32 by Stu

Of course, there will always be some rolling-shutter artifacts that can never be fixed:

NAB 2008

Lots of grading solutions but no buyer’s remorse about our new Film Master system.

But man, is that Lustre shape tracking sweet.

For those grading on the cheap, you now have a control surface.

The Orphanage is cancelling RED One reso in favor of a Scarlet. Funny given that Scarlet is currently a lump of metal—just like RED One was two years ago.

3D is the RED One of NAB2008. Except, whatever.

With the exception of The Foundry. Nuke’s new stereo tools are useful, innovative. If you must.

Even more whatever: FED display at 240fps! Aparently no one told them humans max out at ~60fps.

Redrock Micro has a packed booth in the middle of nowhere.

As usual, after one day I’m ready to come home.

Nuke 5.0 Released

Now featuring: Good taste.

What was once the fugliest compositing app on the planet is now a joy to behold.

There are other fancy new features too, and the same speed and power that set Nuke apart from the moment it was released. But the new UI should provide a nice, smooth on-ramp for Shake users interested in giving it a go.

EDIT: Heh, no wonder I like it so much, you can't tell where this screenshot stops and my (old) blog layout starts!

DV Rebel's fxguide

I love the fxguide podcast, and it seems that the feeling may be mutual. I had the pleasure of joining John Montgomery for my second interview on the podcast over the weekend, and the episode is now up. We rapped about The Guide and about the funky state of compositing software right now. The episode is a companion to the review just posted on

The DV Rebel's Guide reviewed on fxguide