In many of my writings about color correction, both here on ProLost and in The Guide, I’ve talked about the balance between an aggressive “look” that helps tell your story through the use of a pervasive palette, tone, style, and feel; and the preservation of appealing skin tones. When grading a scene, you can push your look much further if you don’t lose track of appealing skin tones. Or, if you so desire, you can make a strong visual statement by choosing to allow your skin tones to get subsumed by your look.

The truth is, skin tones are just one of a small handful of what I call “memory colors.” Memory colors are colors that are, in the minds of your audience, inseparable from certain common objects or events. For example, the sky is so associated with blue that you might feel that you see those two words together as often as you see them individually. The same goes for green and grass.

The most basic idea of color correcting is that you are making colors correct, which is to say that you are making objects on the screen appear to be the colors that we know them to be.

The funny thing about this seemingly simple task is that it can be quite difficult. And it’s difficult for exactly the reason that it’s important.

The human brain is so tied in to our eyesight that we internally auto-correct for certain colors. This is the very definition of a memory color. For example, if you grew up in the United States, you know that a stop sign is red—so you tend to see an image of one as being red even if the color is way out of whack. In the shot below, we recognize the bald head as that of a Caucasian male, even though the white balance is incorrect.

We see his head as skin-colored, even though empirically it is actually almost perfectly gray! You might not believe me, so here’s a crop of the back of his neck to prove it:

This a variation of a common optical illusion called the Same Color Illusion. We “know” that square A and B are different shades of gray “in real life,” and that knowledge prevents us from seeing that they are in fact the exact same shade in the image (click the image to see proof).

UPDATE: Aaron points out below that the Color Constancy Illusion may be a better model of the problem.

Back to the head. Even though they “know” what color it is, your audience will respond more favorably to a memory color object if their knowledge of it matches their experience, rather than fights it. And so it falls to the colorist to correct the color of the head, to make it head-colored rather than gray.

In 2008 I pointed out an example of this from the trailer for The Incredible Hulk (Steve Bowen, colorist). Edward Norton’s face appears the same color whether in a cool scene or a warm scene.

Preserving skin tones is important, but so is preserving other memory colors. Here’s a shot from Jumper (Steven J. Scott, colorist). Sam Jackson is about to walk through a crowd of people. His and their skin tones are accurate, even though their world is a faded, monochrome olive drab.

But back up a few seconds in the same shot and notice that in this faded world, brake lights are perfect, vivid red, and New York taxis read as the correct yellow-orange. This is an establishing shot, and if the grade abused the hue of the taxis too harshly, we might not read “New York” as readily.

Here’s a very short list of memory colors I try to keep in mind when coloring:

• People are pink/orange (a color I like to call porange)
• Grass and summer trees are green
• Water and skies are blue
• Fire engines, stop signs, and blood are red

You could also add just about any food to that list. Unless you’re deliberately trying to make something look unappetizing, it’s probably good to render food as accurately as possible—as I’ll show you in a moment.

I welcome your suggestions of other memory colors. And bear in mind that memory colors might vary from film to film and even scene to scene. In Stomp the Yard, there’s a scene at the beginning where almost nothing is red. Later, there’s a red jacket color so important to the story that it leaps out of every scene in which it appears.

So what’s the big deal? Objects have colors, and the colorist makes sure those things stay those colors. Easy, right?

Not necessarily. To achieve the look of that Jumper shot—where key colors pop but unimportant ones blend into a complimentary shade of blue-green—requires practice, skill, and taste. It’s hard enough under the best of circumstances, but lighting, atmosphere, bounce light, flare, camera settings, and a hundred other factors can conspire to force objects to render on-screen in colors quite unlike their real-world hues.

The good colorist first picks the memory colors important to the scene, and then ensures that they stay consistent, often combating these factors to do so.

Here’s a very simple example. I bought some espresso beans today from my favorite local roaster, Blue Bottle coffee. As I was transferring them to an air-tight container, my 7D was right there, so I popped off a quick 720p60 shot of the process—because who doesn’t like seeing coffee beans tumble in slow motion?

When looking at the footage on my computer, I noticed a funny thing. The beans, which in life have a vivid, sumptuous brown tone, appeared gray-black on my screen. I almost didn’t notice, because I know they are brown, but on close inspection it was clear that I had been fooled by my brain into seeing what I knew rather than what was actually there. The cool color temperature of the indirect sun lighting the shot was reflecting off the beans and cooling their color down to near neutral.

There’s nothing unnatural or wrong about this, except that the audience for my espresso epic doesn’t know about the cool light source outside of the frame. They don’t even necessarily know what the falling objects are. I have to communicate that visually, so I need to preserve—or, in this case, recreate—the memory color of perfectly roasted coffee beans.

Here’s the shot with a Colorista Power Mask for just the beans:

And here’s that same shot with an overall look applied after the bean color fix.

To really see the importance of the local correction, look at the shot with the look, but without the bean fix:

Not only do the beans look more appetizing with the fix, they also survive the subsequent look adjustment better. In fact, since the look cools down the shot a bit, the warm color of the beans stands out all the more. Without the bean fix, the look utterly clobbers the brown beans. As a bonus, in the corrected version, the metal canister and the corner of the grinder on the right take on a steely blue color, better matching the viewer’s idea of what color metal should be.

If you pick your memory colors for a scene, and preserve and enhance them through your look, you’ll wind up with shots that pop without looking clobbered by a heavy-handed “preset” look.

Want to learn more? Check out some great books on color correction at the ProLost Store, or find more ProLost posts tagged with Color.

Update on Monday, February 22, 2010 at 3:43PM by Stu

Folks following me on Twitter know that I’ve been posting the occasional color correction before/after example there. I’ve now collected them here, including a higher-res version of the coffee bean example.

A frequent concern about shooting to a heavily-compressed digital format—something the DV Rebel often finds herself doing—is the degree to which the footage will be “color correctable.” Will the shots fall apart when subjected to software color grading? Or will you be able to work with the footage as fluidly as you tweak your raw stills in Lightroom?

It’s a valid concern. The movies that the current crop of HDSLRs shoot are highly compressed. This compression is perceptual, meaning that it takes advantage of visually similar colors and shapes, and represents those regions with less accuracy than the detailed and varied parts of the image. This makes perfect sense, but often in color grading one seeks to enhance color contrasts—to make a face pop off a similarly-colored background for example—and so you may well create high contrasts between colors that were once nearly identical, and as such were given short shrift by the camera’s compression.

You might have noticed a similar phenomenon in audio. An low-bit-rate MP3 that sounds decent enough can suddently sound awful after even a tiny amount of EQ. Another case of perceptual compression limiting your options.

While you will never find as much data and detail in your HDSLR video as you do in that same camera’s raw stills, the H.264 movies created by the Canon 7D, 5D and 1D Mark IV will withstand some massaging in post. Here are some tips (similar to those found in greater detail in The DV Rebel’s Guide) to help you get the best results.

• Shoot flat. If you read Flatten your 5D, you know that I am a proponent of setting up a “flat” Picture Style using the camera’s built-in controls. The same settings I specced out for the 5D Mark II apply to the 7D and 1D Mark IV as well, although with the 7D I’m less likely to use Highlight Tone Priority, as this setting can increase shadow noise, and the 7D is not as noise-free as the other Canon HDSLRs.
• Chose WB wisely. Use a white balance preset that gives you as nuetral an image as possible. Shooting with an incorrect white balance reduces your dynamic range, because you wind up with an image that’s prematurely blown-out in one color channel, dark and noisy in others.
• Expose to the right. Make the brightest image you can without clipping something important. A rule-of-thumb considered gospel by many photographers, but our reasoning is a bit different. Yes, we, like the stills guys, wish to avoid excess noise in the shadows, but that’s not our main concern. Remember that term perceptual compression. Dark areas of an image get less bits. If you underexpose, you’ll have to brighten the image in color correction, and you’ll reveal all kinds of nastiness the camera thought you’d never see.
• Do denoise. It doesn’t really matter what denoising software you use, but use it. When you carefully and subtly denoise your footage, you rebuild your pixels anew, which is especially nice when you follow the next tip:
• Work at high bit-depths. If you start with an 8-bit image and do a gentle de-noise, you’re blending pixels values together to create new colors. Although there’s no such thing as something for nothing, doing this at a higher bit-depth means those new colors have massivly more gradations than the original image. Your subsequent color work will hold up much better.
• Sharpen last. Your flat Picture Style removed the camera’s built-in sharpening. Add your own at the very last step. The amount you use will vary depending on the output medium, so test test test.

By folllowing these guidelines you can make good-looking shots even better with color correction. But what about a shot that isn’t so great to start with? Turns out there’s hope. Below is a 7D shot that I grabbed in an uncontrolled situation. In my haste, I underexposed, and used the “cloudy” white balance when I probably should have used tungsten. But with a little denoising, careful analysis of the colors in the image, and a Colorista Power Mask, I was able to rescue this shot.

Yes, you can color correct your HDSLR footage, and you should. Color correction can make a good shot great, and in a pinch, put an unusable shot back in the game.

I wonder if someday, maybe when I climb both Mount Fuji and Yellow Mountain in the same week, I will understand how Canon names their SLR bodies.

Today Canon announced the EOS 7D. The name makes it sound like the 5D, but it is decidedly unlike it. It is vastly superior in every way, except in pixel count, and by a very significant difference in sensor size. The 7D is an APS-C body, with a crop factor of 1.6.

The main reason you’re reading about the 7D here is that it records HD video. Canon has done with the 7D what they either could not or would not do with the 5D Mark II: they have included a plethora of useful frame rates (revised, see update below).

• At 1920x1080: 23.976, 24, 29.97, 30
• At 1280x720: 50 and 59.94 (often called 60p)

Check, check, and check. That’s pretty much awesome. I mean, we can dream of 50 and 60 fps at 1080p, but in all fairness those frame rates are usually only found at 720p even on very high-end HD cameras.

Combining that with the same manual control that came at long last to the 5D Mark II, and you have a camera that addresses some of the biggest shortcomings of DSLR cinematography.

Some, but not all. Does the 7D skip lines to create its HD images? Seems so. Is the compression still aggressive and unfriendly to post? Probably. Rolling shutter Jell-O? Likely. And there’s still no video-friendly autofocus such as we find in the Panasonic GH1.

(I know that we classy filmmaking pros are supposed to hate on autofocus, but as I wrote in the DV Rebel’s Guide, the truth is that, for the crew-of-few, it’s a nice option to have. Especially when it’s smart enough to track faces, even specific faces. Focus pulling is hard. Focus pulling while operating a clumsy camera with no focus aides of any kind and a VistaVision-sized chip has had me secretly longing for video autofocus on my 5D Mark II. Focus pulling while swinging your SLR around on a Steadicam Merlin is actually impossible. So please forgive my pining for fingertip on-off smart-as-heck autofocus on my big-chip digital cinema camera.)

What’s missing from the 7D?

• Autofocus (see above).
• Focus-assist while recording, in the form of an edge-detection display overlay.
• Gentle compression (although it seems the 7D’s video bitrate might be a little higher than the 5D’s).
• Funky frame rates. Even Jackie Chan shoots his fight scenes at 22 fps. After a big lunch, he might call for 21 or even 20fps.
• Decoupling shooting frame rate from playback. I’ll probably shoot a ton of stuff at 60p for slow-mo, adding more fun to the workflow puzzle posed by these cameras that crash the video party without a SMPTE invite.
• Video ergonomics. The 7D lacks a flip-out LCD. They call it that because if the 7D had included one, I would have seriously flipped out. I’d like to lose the double chin.
• Quality. The 7D still skips lines to make its HD images, resulting in aliasing and color artifacts. And it still has rolling shutter issues.
• Recordable HD HDMI output.
• Full-frame.

Yeah, let’s talk about that last one. Yesterday, when the 7D’s specs were “translated” from Chinese into English by some sort of cruel, sadistic computer, many people, presumably 5D fans, called its APS-C sensor a “deal breaker.”

The 5D Mark II is the camera I want for shooting stills. How luxurious that it also shoots video, albeit with many limitations. I very publicly hoped Canon would shore up those shortcomings, and they did, some. It did occur to me though, when the Canon/Nikon DSLR arms race grew to encompass video, that my ideal stills rig may not always happen to be my perfect motion rig.

Ever since I treated myself to the original 5D, I’ve been hooked on full-frame. Or should I say re-hooked, as it was the 5D that made digital photography once again as rewarding as it had been with my 35mm Nikormat. I have no interest in a crop-sensor DSLR for shooting stills. But is APS-C an acceptable sensor-size for filmmaking? Ask any RED One owner. Or for that matter, any 35mm motion picture camera owner. APS-C is quite similar to the Super-35 imager size regarded as a holy grail in digital cinema.

APS-C is a perfectly awesome sensor size for filmmaking. The proof is… movies.

But it’s not that simple.

RED, Panavison, Arri, and others have optimized their cinema prime lens offerings for that Super 35 size. Canon’s range of big, expensive primes make sense on their big, expensive, full-frame cameras. Canon’s 24, 35, 50 primes become 38, 56, and 80 equivalencies, respectively. There’s a pricey 14mm that becomes a 22, but nothing between that and the 24(38)mm. To achieve a nice range of focal lengths for 7D filmmaking, you’ll have to use zooms. Since you have to work harder to achieve shallow depth of field, you’ll need expensive, fast-and-wide zooms. The optional kit lens will kill your DOF fetish faster than a $3500 1/4” JVC GY-HM100U. The 16–35mm f/2.8L will be a popular lens for filmmakers—it is to the 7D (roughly) what the beloved 24–70mm f/2.8L is to the 5D.

UPDATE: Several commenters have pointed out that the less-expensive, longer-ranged EF-S 17–55mm f/2.8 IS (27–88mm equivalent) might be an even better choice if you don’t care about compatability with a full-frame body. The 17–55 seems like a terrific lens, but I will stick to lenses that I can use for both video on the 7D and stills on my 5D Mark II.

The twice-as-vast area of the 5D Mark II’s sensor means more flexibility with more lenses, and more control over DOF. Yes, this includes the option to push it to a fetishistic extreme, but also the ability to achieve cinematic DOF with the slower stop of an affordable, flexible zoom lens. So Canon, if complaints about the 7D’s sensor size get annoying, you have no one to blame but yourself. It was you who hooked us on the glorious excess of affordable digital VistaVision filmmaking.

So now there’s a camera that, technically speaking, does everything better than my 5D Mark II. Better autofocus, more stills-per-second, better weather sealing, and better flash control.

But I don’t care. For stills, I’ll stick to my 5D Mark II. Eight-perf VistaVision, what stills folk think of as “full frame,” is the right sensor size for my kind of photography.

But APS-C, roughly the same as four-perf 35mm and what cinema folk consider full-frame, has been well-established as a wonderful size for filmmaking.

So you got me Canon. I’ll probably buy a 7D, and use it for nothing but video. I may have to switch up my lens shopping list, bumping up the 16–35 ahead of the 70–200. At least I can buy lenses that I can use with both my stills rig and my interchangeable lens, variable-frame-rate, Super 35(ish) sensor HD video camera.

That costs $1700.

And that’s the punch line: An interchangeable lens, variable-frame-rate, Super 35(ish) sensor HD video camera for $1700.

Canon, you’re gonna sell a bazillion of these. And you deserve to.

It’s tempting to project right past the 7D and imagine that maybe the rumored successor to the 1Ds line will capture all the new stills improvements of the 7D and combine them with a full-frame sensor. For a mere several thousand dollars you could have it all. Maybe. At some indeterminate time in the future.

Screw that. The 7D costs as much as a nice lens and it’s real.

If you can get your hands on one.

Do I still love my 5D Mark II? Absolutely.

Do I wish Canon would sprinkle it with magic 24p dust? Of course.

Is the 7D the perfect DV Rebel Camera? No. I still recommend a good, solid HD camcorder for filmmaking that doesn’t require fetishistic shallow depth of field. Which, FYI, is almost all filmmaking.

Has DSLR cinematography arrived?

Yes.

As of today (and barring any unforeseen surprises), the answer is yes.

Pre-order your 7D from Amazon now.

I sure did.

Update on Tuesday, September 1, 2009 at 12:13PM by Stu

Chuck Westfall clarified the 7D’s frame rate options on the CML today:

The accurate video frame rates for the EOS 7D are stated correctly in the Canon USA press release: The 30p setting is actually 29.97 fps; 24p setting is actually 23.976 fps; and 25p setting is 25.00 fps. In the 720p and SD video modes, the accurate frame rates are as follows: The 50p setting is 50.00 fps and the 60p setting is 59.94 fps.

He’s right, the press release reads like this:

The EOS 7D Digital SLR camera will record Full HD at 1920 x 1080 pixels in selectable frame rates of 24p (23.976), 25p, or 30p (29.97); 720p HD recording at 50p or 60p (59.94) and SD video at frame rates of 50p or 60p (59.94).

I’ve revised my frame rate list above to match.

Readers of The DV Rebel’s Guide know that I like to set up my cameras to record as much dynamic range as possible, resulting in a low-contrast, low-saturation “digital negative” that allows more flexibility for grading in post. If you’ve seen any video I’ve shot with the Canon 5D Mark II lately, whether in the Red Giant TV tutorial or on fxphd, you may have wondered how I set up the camera to achieve this. The answer comes in the form of some in-camera Picture Style settings.

I posted a while back about using Canon’s PictureStyleEditor software to create these custom Picture Styles. Since then I have found that the controls in the camera are more than sufficient for creating a more post-friendly look for the 5D’s video files.

Starting with the “Neutral” setting, I make the following modifications:

• Sharpness all the way down
• Contrast all the way down
• Saturation two notches down

Save that as on of your User Defined Picture Styles. Then hop over to the Custom Function menu, select C.Fn II: Image, and enable Highlight Tone Priority.

UPDATE: Or don’t. See Update 1 below.

Your settings should look like this:

This will remove the contrasty, video-like tone curve from your future recordings, and eek out a little more highlight detail. Here’s a shot made with the default settings:

Video frame made with Standard Picture Mode, Highlight Tone Priority offHere’s that same shot with the Stu settings:

“Stu settings”

You can see the reduced contrast, the increase in shadow and highlight detail. It’s closer to a raw image with a linear tone curve. Zoom in and you can see the difference in highlight handling on the reflections:

Default settings — harsh, clipped highlightsStu settings — smooth rolloff in the highlights, increased detailYou can also see the difference in sharpening artifacts. My settings reduce, but do not eliminate, the moiré effect endemic to the 5D Mark II’s line-skipping:

Default settingsStu settings

You’re going to put all that contrast and most of that sharpening back in post of course, but in your own way, and with more control, and after any shot-to-shot evening out or clever power windows. The ability to design a “zeroed out” or CINE_LIKE-esque Picture Style is one of the things that makes the 5D Mark II’s video dangerously close to usable.

I recommend assigning this setting to one of your Custom notches on the mode dial, so that you don’t inflict these settings on your stills. They don’t affect raw files of course (UPDATE: That’s not actually true, see Update below), but they do get baked into the JPEG previews that accompany those raw files. I have the above settings registered as C3 (along with manual control and a 1/60th shutter), so I can quickly pop into my ideal movie capture settings.

Update on Friday, August 21, 2009 at 9:11AM by Stu

If you do set up your 5D as described above, make sure to do the last step with the custom function so that these settings only affect your video shooting. You probably don’t want to have Highlight Tone Priority on for your stills, especially if you shoot raw. What it does is “push” (reduce) your ISO setting by one stop, causing you to capture a full stop of additional highlight information through no more sophisticated means than simply underexposing. For video and JPEGs, that additional highlight range is non-linearly mapped into the image, creating a very film-like “shoulder.” For raw, it simply passes metadata to your raw decoder that may or may not be honored.

Lightroom, for example, interprets HTP as a simple instruction to boost exposure by one stop. Fair enough, except that this won’t match what you saw on the camera LCD at all. Blacks will be too crushed and that extra highlight detail will be missing (until you futz with the Develop controls to bring it back). Trying to bring more detail back into the blacks will bring in nasty noise that was not visible in the JPEG preview.

Underexposing to capture extra highlight info can be a fine idea, but it’s best implemented case-by-case, and in a predictable way.

Highlight Tone Priority hides the underexposure from you and might cause you to think you’re capturing a more robust negative than you really are.

But for video mode, it creates a more film-like low-contrast, high dynamic range image, with just a touch more noise from the one-stop push.

I now roll with two custom function settings, one with HTP on, one with it off. For daylight exteriors with plenty of light, HTP: On. For lower-light situations, or more controlled lighting, HTP: Off.

Update on Monday, August 24, 2009 at 2:09PM by Stu

There’s an interesting discussion of the nuances of HTP and ISO performance over at cinema5D.com. Interpret tests results such as these with caution though — as the OP says, he’s trying to show off the noise, so it’s not so much a real-world test as it is an attempt to learn more about what’s happening in the camera. Turns out there are some surprizing variations in noise preformance at different ISO settings, possibly having to do with some ISO settings being “virtual,” i.e. metadata offsets of native settings.

As seen this morning on my NAB panel. My contribution to the emerging Subway Short genre of camera tests, with a twist—what happens after all the furtive subway hopping and phone dialing?

To see it in HD, you’ll have to go to YouTube (and please do). Why no HD on Vimeo? Vimeo only serves HD at 24p, and as lovely as that sounds, this short was shot on the Canon 5D Mark II, and therefore is at 30fps. Vimeo does a sloppy convesion to 24p, which is not cool.

Update on Tuesday, April 21, 2009 at 12:18AM by Stu

Thanks for all the comments below guys! Here’s a bit more about the short.

First of all, I knew going into this that the look would be that heartbreaking amalgamation of cinematic depth of field and video-like motion. But I didn’t want to send this camera back to Vincent without shooting something. I still consider this a “camera test” more than anything—but really, the best thing to come of the shoot was meeting The Stunt People, who are both the stars and co-creators of this peice. I can’t wait to work with Eric and the guys again.

I won’t shoot anything else at 30p, ever, unless I plan on slowing it down to 24 for a slight overcrank effect. I’ve seen entire TV commercials shot 30 for 24 though, so it could happen. I harassed the Canon reps at NAB today about 24p and manual control until they were afraid of me.

As devastating as the frame rate is, the lack of shutter control is worse. Most of the nasty video-like look is due to the camera’s frequent use of a shutter speed slower than 1/60. There’s plenty of 1/30 in there, which is the equivalent of a 360 degree shutter on a film camera—an impossibility, and a dead giveaway of video origination (even at 24p).

There are people out there enjoying some luck converting 5D Mark II footage to 24p using tools such as Twixtor and the built-in retiming in After Effects, but that only works on relatively sedate subjects (of which this short has very few), and even then, there are often nasty artifacts to contend with. Didn’t we choose this camera for image quality?

I had two big dropouts, which I fixed using that After Effects retiming feature. I immediately bought some Extreme IV CF cards to avoid that problem in the future.

I shot with only one lens, the slow-but-lovely 24–105mm f/4 L lens that comes with the camera. We were mostly at f/4.0, although I can’t be sure we always were. I stopped doing the half-twist trick after I realized how much I and the camera would be moving around. So as shallow as the DOF is, it’s nothing compared to what we could have had if I’d used primes. That’s a big leg up the 5D could have over the Panasonic GH1—even with slow glass, the Canon can make DOF that can be used as a storytelling aide, such as the moment where our guy sees the two adversaries in the distance. They appear as blurry blobs, just the way I like ‘em.

I had access to the full Redrock Micro DSLR bundle, but wound up stripping it way, way down to something I could be agile with and still pull focus as a solo operator. This stripped-down rig became the inspiration for the rig I blogged about recently.

I used my HV20 for one shot. Can you guess which one?

Editor Gregory Nussbaum (The Spirit) cut with Final Cut Pro, using half-res proxies that I rendered out of After Effects. When we locked the cut, I exported XML from Final Cut and imported that into Premiere Pro CS4. I then loaded that Premiere project into After Effects. From there, I could create precomps for each clip and swap in the full-res originals.

I color corrected entirely using Magic Bullet Colorista, sometimes several layers per shot. Many Power Masks to sculpt lighting and lift faces, as well as to unite the various colors of light present in the location. I used no lighting or bounce of any kind during the shoot.

I used the DV Rebel Tools to add new color correction layers and to preview my work using a configurable thumbnail view. The result is a deep, but very organized After Effects timeline.

And I will post more about most of these things!