Cameras

Apple’s “Let Loose” iPad Event was Shot on iPhone — With Panavision Lenses

May 09, 2024

Apple unveiled their new line of iPads yesterday in a pre-recorded video titled “Let Loose.” As with the previous “Scary Fast” MacBook Pro launch video, “Let Loose” ends with a tag proclaiming “Shot on iPhone” — this time adding “Edited on Mac and iPad,” and the fine print: “All presenters, locations, and aerial footage shot on iPhone.”

During the live stream I actively wondered if the iPhone acquisition of “Scary Fast” had been a one-time thing. “Let Loose” looks great, as all Apple videos do, but some shots featured a shallower depth-of-field than is possible with an iPhone-sized lens and sensor combo. At the end of the event, I wondered publicly on Threads about this. Replies speculated about additional lens rigs, some improved version of Cinematic Mode, or maybe blurring the background in post.

Panavision Lens Relay System

After Apple released a behind-the-scenes video about the production of “Scary Fast,” the Internet did its internet thing and questioned the “Shot on iPhone” claim, as if “Shot on iPhone” inherently means “shot with zero other gear besides an iPhone.” These takes were dumb and bad and some even included assertions that Apple added additional lensing to the phones, which they did not.

But for “Let Loose,” they did.

“Let Loose” was shot on iPhone 15 Pro Max, and Apple says that for several shots where a shallow depth-of-field was desired, Panavision lenses were attached to the iPhones using a Panavision-developed mount called the “Lens Relay System.” This rig is publicly available for rent from Panavision today, although not currently listed on their website.

There’s just enough shallow focus in these shots to make me wonder how they could be shot on iPhone

What’s a “lens relay system?” Think of a telescope. Instead of focusing an image on a plane of film or a sensor, it creates what’s known as an “aerial image” that you capture with another lens system — your eye. If you’ve ever smashed your phone up to a pair of binoculars successfully, you’ve made a lens relay system.

I used a Frasier lens relay system from Panavision to get small lenses into tight spaces for a Ruby Tuesday commercial I directed. The Frasier is a pericope lens, one physical unit that contains both the taking lens and the capture lens.

I don't know for sure that Panavision's system uses an aerial image rather than a ground glass like this rig from Beastgrip, but the image quality and name make me think it does.

Either way, with Panavision’s system, the iPhone’s own lens rephotographs the image created by any Panavision lens you like. The iPhone provides the image capture, in ProRes Apple Log, of course.

Panavision showed off this “director’s viewfinder” iPhone rig in April. Not exactly what Apple used, but similar idea. Image from Y. M. Cinema, click through for more.

There are, of course, many systems for mounting lenses to iPhone (Beastgrip is working on a new one), but it’s certainly notable that Panavision made one, as they exclusively serve a market of high-end professional filmmakers.

Why Lenses?

If “Scary Fast” could be shot without add-on lenses, what does Panavision’s rig bring to the table? Apple Log allows the iPhone to capture highly-malleable, 10-bit ProRes footage that fits into a professional pipeline alongside footage from high-end cinema cameras, but the one thing it can’t do is capture the shallow depth-of-field that we associate with high-end productions. The look of “Let Loose” is a collaboration between the iPhone’s clean capture, the focus control of the Panavision lenses, and top-tier color grading — masterfully overseen by an uncredited cinematographer.

Don’t You Start Internet

So is it fair to say “Let Loose” was “Shot on iPhone” if it was done with the help of gear the average iPhone owner could never afford? Of course it is. Feel free to re-read this: What Does and Doesn't Matter about Apple Shooting their October Event on iPhone 15 Pro Max — but in short, the fact that Apple can drop an iPhone into an otherwise-unaltered professional film production and match the look of their previous videos without anyone even noticing is meaningful.

In fact, “Let Loose” is the first Apple Event finished and streamed in HDR, pushing the iPhone’s capture abilities even further than “Scary Fast.”

You don’t need to add cinema lenses to the iPhone to make great-looking images, but the fact that you can is cool.

You also don’t need to twist yourself into knots wondering why you might choose an iPhone over a professional cinema camera when you have a Panavision lens budget. Personally, I’m more excited about the run-and-gun possibilities, but a vote of confidence from the most elite cinema rental house only bolsters the story of iPhone as professional video camera.

Or just make note that Apple confidently intercut footage shot with the most respected cinema lenses available with shots lensed with unadorned iPhones.

The Real Story Here is Apple Log

None of this would be possible without Apple having adding Log ProRes recording to the iPhone 15 Pro. Log is to video what raw is to still photography, and the story of how Apple Log transforms the definition of “Shot on iPhone” from a dalliance to a responsible, even desirable strategy for filmmaking is still ongoing.

Truly “pro” features like color-accurate OLED screens on iPads and ProRes Log on phones don’t just sell a few devices to a few filmmakers. They preserve and elevate Apple’s reputation as the choice of creative professionals in all fields. Apple hardware is vastly overpowered for most of its customer’s uses, so as Apple looks around for folks in need of their very best, they find the Zbrush artist, the Redshift renderer, and now, improbably-but-deservedly, the professional cinematographer.

iPhone ProRes Log in Peru and Taiwan

January 30, 2024

This is a blog post about a video, which is about new color-conversion LUTs for Apple Log footage from the iPhone 15 Pro and Pro Max (updated from my first set). The video is also a mini-travelogue of my recent trips to Taiwan and Peru. After my explainer video on Apple Log, this one dives a bit deeper into practical LUT workflows, and my state of mind about shooting digital-cinema-grade footage with a device I always have with me.

There’s a lot going on here.

Or Just Skip to the LUTs

Conflicted in Peru

Stu Maschwitz shooting video of a parade in Lima, Peru, using an iPhone 15 Pro Max with an external drive. Awkwardly, a real camera is also strapped around his neck. — Me relaxing on vacation. Photo by Forest Key.

I always have a moment when packing for a trip: Which camera to bring? Which lenses? I know I’m always happier when I pack less, like just a single prime lens. But sometimes FOMO gets me and I pack three zooms.

For my trip to Lima, I brought my Sony a7RIV with the uninspiring-but-compact Sony 35mm F2.8 prime. I lugged it around for a few days, but wasn’t really feeling it.

Meanwhile, my iPhone 15 Pro Max was calling to me with its ProRes Log video mode. “I’m 10-bit!” It would say. “Think of the fun you’ll have color grading me!”

I told my phone to shut up, and proceeded to shoot very little with it — or my Sony. Like a squirrel in the middle of the street, drawn in two different directions at once, I creatively froze.

Photography, for me, is made up of a lot of habits, and shooting iPhone video with aesthetic intent is just not yet baked into my travel muscle memory.

Made in Taiwan

A month later, I took a family trip to Taiwan, one of my favorite places in the world. I’d had some time to process my Peru deadlock, and decided to stop judging my own creative impulses, and let inspiration guide me in which camera I pulled out.

I wound up shooting a lot of video.

Me relaxing on vacation. Photo by Josh Locker.

I loved shooting ProRes Log in Taiwan with the iPhone 15 Pro Max. I’d occasionally reach for Blackmagic Camera, but I often just used the default camera app. I stuck my phone (with its crumbling case) out of taxi sunroofs and skyscraper windows, held it above teeming crowds and shoved it between chain-link fences. Seeing the broad dynamic range I was capturing in scenarios from noontime sun to neon-lit nights got me excited about grading the footage later.

It’s exactly the way I feel about shooting raw stills with my Sony, knowing that I’ll be able to go crazy on them in Lightroom. The photographing act is just half of the process.

Step through the frames below to see how color transforms a single shot from the video above:

Uncorrected Apple Log

Straight out of the camera. I mean, phone.

Look & LUT

An overall correction applied under PL-HERO LUT.

Local Corrections

Various windowed corrections under the Look help guide the viewer’s focus and give the natural-light capture a cinematic feel.

LUTs, Looks, and Magic Bullets

There’s been a bit of a gold rush of people hawking creative LUTs that apply a particular “look” to iPhone Log footage. My day job is, in part, helping make color tools like Magic Bullet Looks, which can do so much more than any LUT. Creative LUTs are great, and by all means support the folks making them — but that’s not what my iPhone LUTs were or are.

The Prolost iPhone LUTs convert Apple Log to various other color spaces, and support three kinds of workflow:

Grade Under a Display Transform LUT

Apple Log is a totally decent color space to work in, so color correcting Apple Log can be as simple as applying Magic Bullet Colorista and choosing one of my Monitor & Grade LUTs. That’s what you see me doing in the video above. Colorista (set to Log mode) does its work on the native Apple Log pixels, and the LUT converts the result to look nice on video.

Many other systems work like this, including LumaFusion, which ships with Prolost Apple Log LUTs.

The key is color correcting under the LUT.

Bring Apple Log into an Existing Workflow

Color work is often done in an intermediate color space. This is usually some kind of wide-gamut, log format, such as Davinci Wide Gamut/Intermediate, or one of the ACES log spaces.

The Prolost ACES LUTs convert Apple Log to either ACEScc or ACEScct log, allowing you to grade your iPhone footage alongside any other professional camera, and output them all through the same pipeline.

Shooting Through a LUT

The Blackmagic Camera app allows you to load any LUTs you want and preview through them without baking them into your footage. With my LUTs, you can shoot with the same LUTs you grade under later, for a truly professional (no joke!) workflow.

The real stars of this update though are the FC LUTs. They add an informative False Color overlay to the Shoot/Grade LUTs, making sure you always nail your exposure. Watch the video to see them in action. I already can’t imagine shooting without them.

These LUTs work well in Blackmagic Camera or even on an external HDMI monitor.

Adjusting exposure with a variable ND filter until the 18% gray card lights up yellow, for perfect exposure. PL-HERO-FC LUT in Blackmagic Camera.

Gathering Resolve

I’ve never edited a whole actual thing in Resolve before.

As if this video wasn’t enough work already (I shot the A-roll in mid-December), I decided to use it as a personal test-case for creative editorial in DaVinci Resolve. It’s the ACES LUTs that allowed me to incorporate Magic Bullet Looks into my Resolve color workflow.

Maxon just shipped a really nice update to Magic Bullet Looks, with simplified color management made possible by more and more apps we support doing darn fine color management at the timeline level.

So in Resolve, I can use my LUT to convert Apple Log to ACEScc, and then apply Magic Bullet Looks, which can now be set to work in ACEScc with a single click.

The new streamlined color options in Magic Bullet Looks. Choose Custom to get the full manual control.

I can sneak additional Resolve corrector nodes between those two for local corrections. Resolve is great at this, and Looks is great at creative look development, so this is a match made in heaven.

Then, at the end, I use an ACES Transform node to convert to Rec. 709 video.

An expert Resolve user could replace my LUTs with Resolve’s built-in Color Space Transform nodes, but the LUTs make this process easier and more reliable.

Gear Inspires

Every photographer knows the feeling of lusting after new gear. We know it so well that we remind ourselves constantly that “next camera syndrome” is debilitating, and that “most cameras are better than most photographers.” Gear is not the answer. Go shoot.

There is, however, a counterpoint to these truths: As shooters, we take inspiration where we can get it. And sometimes a new technique, a new locale, or even, yes, a new bit of kit is what provides it.

The key is to listen for that inspiration, and don’t judge it.

Even if it’s coming from your phone.

Jiufen village, Taiwan. Two a7RIV shots stitched in Lightroom.

Get the Free LUTs

Three cameras that can make photos with blurry backgrounds.

iPhone 7 Plus Depth Effect is Legit

September 29, 2016

The star feature of the iPhone 7 Plus is its dual-camera system. The typical 28mm-equivalent iPhone camera is joined by a 56mm-equivalent. This allows for a 2x optical zoom at the touch of a virtual button, but, more interestingly, it also opens the door to some interesting computational photography.

56mm equivalent is not "telephoto" Apple.

Two cameras don't really make for a light field camera, where a computer model is built of the captured light rays, allowing them to be projected onto a virtual image capture plane, through a synthetic aperture. That's what Lytro is doing with their plenoptic Cinema Camera (see previous post), and more analogously, what Light is promising with the 16-lensed L16 camera (two posts on that one so far).

Computational Photography is Here (and Has Been for a While actually)

I'm pretty sure that with only two cameras, you can't build a useful light field. But can you do computational photography? That's a trick question, as the iPhone, and many other mobile phone cameras, are already doing computational photography. Already the iPhone will automatically perform an HDR merge of two exposures, for example. But even when the iPhone snaps a single, non-HDR exposure, the amount of post-processing it does is considerable.

We've gotten to test this first hand recently with Apple opening up raw capture to developers. Adobe jumped on this right away with Lightroom Mobile, having already implemented raw in their Android version. The first thing you notice when shooting raw DNG files with your iPhone is how noisy the image are. Turns out Apple's been doing a ton of noise reduction on their photos for a few generations now. It's entirely possible that they are using multiple exposures to aid in this process, but I don't know if anyone's ever confirmed that.

Portrait Mode, Depth Effect

Apple calls their initial two-lens computational photo offering Portrait Mode, and the most recent developer beta of iOS 10.1 includes a beta version of it. Under the right circumstances, this mode enables a so-called "Depth Effect," where both cameras fire simultaneously, and a depth map is built based on the subtle stereo disparity between the captured images. This nine-level depth map is used to drive a variable-radius blur. The result is a photo with simulated shallow depth of field.

This process can never be perfect, but can it be good enough?

Oh hell yes it can.

Why Do We Care?

When I first started testing Portrait Mode, I was alone in my backyard, with only inanimate props. I took some shots where the Depth Effect shined, and some where it flopped. I posted some samples on Instagram, using an unforgiving split-screen effect that dramatically highlights the imperfections of the processing.

Most notably, the processing gives the foreground a bit of a haircut, which you can see clearly in this example.

This stands to reason. The depth map is very likely computed at a reduced resolution, and I bet it’s noisy. Any smoothing is going to also eliminate certain edge details, and Apple's engineers have, I'm surmising, estimated that eating into the edges a bit overall is better than seeing a halo of crisp background between the foreground subject and the blurred background.

The next night, my family came over for a cookout. As we ate and drank into the evening, reveling in global warming, I remembered that I had a new toy to play with. I pulled out my phone, toggled over to Portrait Mode, and snapped a few shots of my brother-in-law and his adorable son.

This is the photo that convinced me that Portrait Mode is a real thing. Here it captured a fast-moving, uncooperative subject, at ISO 500 lighting, and produced results that are not just good, but actually a photo I cherish.

Is it perfect? No. The effect ate some of the little guy's flyaway hairs. But "perfection" would be a strange goal for a process designed to simulate an artifact. Without a side-by-side comparison, no one will miss those hairs.

So don't ask if Depth Effect is perfect. A better question is if its failures are distracting. And I have certainly taken some test photos where they are. But the funny thing about test photos is that there's often nothing worth photographing in them, so you just stare at the problems. In my own testing, whenever I've pointed Portrait Mode at something I actually care about, the results have been solid.

So back to the question of whether we should care about a fake blur applied in post to a telephone photo. When I tweeted the above shot, someone replied with a reasonable question: wouldn't I love the photo just as much without the effect? I replied no, and added:

Composition matters, and focus is composition in depth.

Portrait Mode photos aren't just photos with a blur applied. They have the potential to be photos that are more about what they are photos of. It gets back to one of the oldest, most durable posts on this site: Less is More. We frame our shots carefully, and shallow depth of field allows us to frame our shots in depth as well.

Sometimes that makes the photo prettier. Often, it can make the photo.

As an example, here's another photo of something I care about: my son, and his uncharacteristic (for a 7-year-old) love of raw oysters.

In the non-depth-effect version, the background is so distracting that I probably wouldn't have shared this photo, but the shallow-depth-of-field version not only looks better, it succeeds in communicating my feelings at the moment of capture.

See, our eyes actually have very deep focus, but our brains and our hearts fire at ƒ/0.95.

The instrument of our eyeballs "sees" everything, but we don't see with our eyes. We see with our brains. And our brains ignore stuff that doesn't matter (to a hilarious fault). This is maybe the number one failing of amateur photography, and certainly mobile photography—we take a ton of photos that wind up being more about weird ugly background details than the reason we wanted to take a picture.

So I posted this oyster shot on Instagram, and someone pointed out how the white blob on the left is eating into my son's elbow a little. This is true, and it's definitely a technical failing of the depth effect. It also reminded me of something that VFX master Dennis Muren once said in dailies when I was a young artist at ILM. Someone pointed out a very real flaw with a shot. But instead of demanding that it be fixed, Dennis brushed the concern aside and declared the shot final. The flaw was not near the subject of the shot, so it didn't concern him. "If they're looking there," he said, "we've lost them."

I sent this shot to a photographer friend who knows how I took it and what problems to look for. His review of the shot is the one that resonates with me: "Your son is growing up too fast, and I want some oysters."

I have a bunch of shots that show Portrait Mode failing. Small foreground objects getting blurred along with the background, edges being eroded noticeably. But I'm not going to post them here because it's not really very fun, and there's a whole internet happy to bash Apple's beta 1 drop of anything. The edge problems will be worth fixating on if they don't improve.

What Are You Giving Up?

It so happens that I also have a full-frame 35mm ƒ/2.0 shot of my kid eating oysters at that same restaurant, earlier this year:

Some obvious differences:

The wider 35mm angle of view, vs the iPhone's 56mm equivalent. Shallow DOF at wide AOVs is an expensive look.
Bloom! The boke of the background lights blooms and pops as one would expect.
Foreground blur: Apple's Depth Effect mostly blurs things behind the point of focus, only very subtly softening anything in front of it. That's either a magic cheat or a huge failing, depending on your point of view. I think it's brilliant that Apple decided to call this "Portrait Mode" and describe it as blurring the background, rather than a more ambitious, and ultimately inaccurate, description of it as a shallow depth of field mode.
All the little frizzy hairs rendered in tack-sharp 42 megapixel glory.
And then there's raw. No, not the food, the photo. This photo was shot with my Sony RX1R II, in raw. I edited both it and the iPhone shot in Lightroom, using the same presets from my Prolost Graduated Presets set, but this one took the edits way better, because it's raw. Slurp!

Of course, the RX1R II is expensive, at almost $4,000. It's tiny for a full-frame camera, and I do bring it with me almost everywhere I go, but despite being smaller than the iPhone 7 Plus in two out of three dimensions, it's not something you slip into your jeans pocket.

Big cameras that shoot raw from a big sensor and fast glass are still king, no question. It's not useful to think of Portrait Mode as competition for them. I look at it this way: Despite all my badass cameras, I will continue to take photos with my iPhone, as I always have. And now they can sometimes look a lot better.

Okay Wanna Nerd Out for a Bit?

Now that we've celebrated that emotional impact matters more than technical accuracy for like the billionth Prolost time, I thought it might be fun to geek out on the images a little and consider just how mighty the task is of producing emotionally resonant fake depth of field on a telephone.

There's been a lot of conjecture about the specific type of blur Apple is doing. I've heard Apple pundits say authoritatively that it's a "gaussian blur." Spoiler: It's not, at least not in any way I've ever seen.

The "right" way to fake a focus blur is with a blur kernel that looks like the boke shape you're going for. Here's a simple example:

Original Image

Gaussian kernel, gaussian blur

Focus blur kernel, focus blur

As you can see, the gaussian blur (approximated here with five box blurs, which is how Photoshop does it!) looks like mush, and in no way resembles an out of focus image. The focus blur kernel gives better results, at the expense of much longer processing time. And the results are still not great. We all know what an out-of-focus Christmas tree looks like, and that is not it.

There are two more things we need to do to truly simulate focus blur. If we do them, our simulation becomes very, very accurate. I've heard other Apple pundits claim that a synthetic blur can never convincingly match the real thing. That is simply not true. But you gotta do it right.

To do it right, you need to do three things:

Use a properly-shaped boke kernel.
Process the blur in linear-light color space, not gamma-encoded space.
Have an HDR source.

What's linear light? It's something I've talked about a lot here over the years. This video explains it well (though it simplifies gamma down to a power of 2).

Here's what it looks like when you take these three steps:

Focus blur in gamma space

Focus blur, in linear space

Focus blur, in linear space with HDR source

Okay! That was a lot of work, and the fans on my iMac spun up a bit, but we got there.

Why is it a lot of work? Let's go back to that list:

Use a properly-shaped boke kernel. This is considerably more computationally expensive than a regular soft blur, because you can't iterate a cheap function or easily parallelize the processing for GPU acceleration. There are some optimizations you can do if you keep the center of the blur flat, but for the most part, you just gotta brute force it.
Linear light: This requires high bit depths to avoid banding, and high bit-depths are expensive in both memory footprint and processing.
HDR requires not just high bit depths, but possibly even floating-point color. It also requires that you have enough overhead in your single-exposure raw to count as HDR, because you're probably not going to do an exposure merge with only two cameras.

I did my blurs in 32bpc floating point, using the cleverly optimized software lens blur in After Effects. It took several seconds to process each image, and my 4 GHz Core i7 iMac's fans spun up audibly.

This problem of realistic depth of field has been the subject of much research. With GPU processing, which one must imagine is how Apple is going about this, it's entirely possible that a kernel-based blur isn't the fastest approach. They could be tracing rays, or, maybe more likely, doing a recursive approximation of a disk-shaped blur.

It's heavy math, and Apple is doing it in real-time while you're framing the shot. Impressive.

iPhone 7 Plus Portrait Mode Live Preview (iOS 10.1 developer beta 1) pic.twitter.com/fc5xSAaMkr
— Stu Maschwitz (@5tu) October 1, 2016

So now we know what it takes to compute a realistic lens blur. So how do Apple's results compare? Click to enlarge:

Sharp BG from a Portrait mode shot

Blurred BG from a Portrait mode shot, Depth Effect version

Sharp BG with lens blur in gamma space

Sharp BG with lens blur in linear space

Gaussian blur in gamma space

Gaussian blur in linear space

Real boke from the iPhone's built-in camera

Focus blur in linear space, with artificial HDR highlights

Apple's blur is not a perfect match for any of mine, but that makes sense. However Apple is doing the blur, it's probably not the last thing in the image processing pipeline.

To my eye, Apple's blur is obviously not gaussian, or even gaussian-esque. It's some kind of sharp-edged circular blur kernel, maybe computed at a lower spatial resolution than the final image, which would account for some of the softness—and the miraculous speed with which the iPhone 7 Plus can do the job. Their blur is neither as flat as my gamma-space example, nor as distinct as my linear-space simulation, so I can't quite tell where Apple's doing the blur in their order of operations. But it looks closer to my gamma-space version.

This is an area of possible improvement: If Apple could operate on pure linear-light pixels, the blur could become more realistic and pleasing.

They might even get a hint of HDR-esque highlights, as raw files are effectively (modestly) HDR images.

They'll never be as poppy as a real photo of bright, out-of-focus lights though, because there's just not enough headroom to hold all that exposure in an iPhone's raw capture. So why not artificially boost the highlights like I did in my final example up there? Again, I think Apple is being conservative here. Highlight boosting can go horribly wrong, resulting in glowing teeth or eyeballs. It requires manual dialing-in to look acceptable, and even then, it never looks perfect.

Canon 5D Mark II, Canon 50mm F1.4 at ƒ/1.4. Because why should anything be in focus, really.

Another reason that Apple's blurred highlights won't look as punchy as photographed boke is that Apple is usually applying their blur to a image that is already slightly out-of-focus. This means the boke shape, even from a perfectly crisp kernel, will always be a little mushy, because the source was not a pinpoint. Again, Apple could try to account for this, but it would be nearly impossible to do that in an unattended algorithm.

So in conclusion, here's my assessment of the Portrait mode blur processing:

It's good. Apple is doing it right. It's not a gaussian blur.
There's opportunity for improvement.
It's a miracle that a telephone can do all this processing as fast as the iPhone 7 Plus does.

Canon 5D Mark III, Canon 50mm F1.2L at ƒ/1.8. And manual focus, if I don't say so myself.

I didn't geek out on the depth map generation and segmenting, but I could. There's a lot going on there too. And I imagine, somewhat optimistically, that this is an area where we will see improvements in future builds.

The Power of Low Expectations

I didn't think the results would be this good. Apple, uncharacteristically, undersold them. And this created room for a delightful surprise when Portrait Mode turned out to be something I will most certainly use.

One might fairly take me to task for being soft on Apple when I have been so hard on others. I gave Light.co a pretty hard time about their (still unproven) claims of being able to match the multi-dimensional, layered boke from a Canon 7D shot.

One of the things I love about photography is that I simply never know what's going to get me excited about it all over again. It could be something strange and stupid, like an app that makes you wait an hour before viewing the photos it takes, or one that takes pictures that look like an Macintosh Plus screen. It could be a strange old lens with fungus growing in it, or a broken chunk of glass held up in front of a matte box. I reserve the right to be delighted by things, even things that were specifically designed to delight me.

Well done, Apple.

A photo of a man punching himself in the face at a bar

A photo of the mother of a kid who loves oysters

Canon 5D Mark IV

August 25, 2016

The First 5D I’m Not Buying

Canon has announced the latest DSLR in the seminal 5D line, the 5D Mark IV. It features a 30.4 megapixel sensor for stills, a touchscreen, and 4k video. It’s available for preorder on Amazon and B&H for shipment on September 8. And I won’t be buying one.

1.74x Crop Factor for 4k Video

As has been well covered by Dan Chung at Newshooter, the only way to shoot 4k video with the 5D Mark IV is using a 1:1 center crop of the 30.4 megapixel sensor, which results in a 1.74x crop factor. That means that, when shooting 4k video, your 50mm lens gets an equivalent focal length of 87mm.

UPDATE: Cinema 5D lists the crop factor as 1.64, and my math (using horizontal measure) agrees. So your 50mm lens matches the horizontal AOV of an 82mm lens at full frame. See more in the update below.

Compare this to the Sony a7S II, which can shoot 4k using a full sensor readout of its full-frame, 12.2 megapixel sensor.

The Sony also has 4:2:2 UHD HDMI out for high quality 4k recording to an external device, where the Canon is limited to HD output only.

No Log Color, Ancient Codecs

The use of a crop for 1:1 4k video is ostensibly about quality, but that assertion contradicted by the lack of a log color mode for video, and by Canon’s single codec option of Motion JPEG. Don’t let the 500 Mbps data rate fool you, as this is a codec with only intra-frame compression, so it’s far less efficient than the a7S II’s 100 Mbps interframe XAVC S format.

The only leg up over Sony here is that Motion JPEG, while ancient and inefficient, is 4:2:2, where XAVC S is 4:2:0. Both are 8-bit.

You Will Never Use the Dual Pixel Raw Mode

Much noise was made of rumors that the Mark IV would feature a dual-pixel raw mode that would allow refocusing in post. I think a lot of folks imagined Lytro-style post-focusing, but fxguide has excellent coverage of the sobering truth: you’ll be able to adjust focus within a very small range only.

For this “image micro adjustment,” you pay the price of double the file size, and using Canon’s proprietary DPP raw processing software.

This is not a thing.

I have asked Adobe if Lightroom might someday support this focus shifting, and I’ll let you know if they give me an answer.

The Touchscreen Focusing Looks Awesome

Touch the screen to rack focus, at user-selectable speeds. Focus then tracks the selected region. This is super useful, if not particularly new. It’s something I’d love to see from Sony.

This is a Giant Camera with a Big Mirror that Flaps Around and Stuff

The 5D is, first and foremost, a stills camera, and it looks like a great one. I still use my 5D Mark III from time to time, and it has many advantages over my Sonys. But it also has some profound disadvantages, shared by the Mark IV.

Contrast-detect autofocus is limited to an APS-C crop within the full-frame sensor
The rear LCD doesn’t tilt
Size and weight
Mirrors. Still a thing?

I’m not 100% sold on mirrorless just yet, but there’s nothing about the 5D Mark IV that’s slowing my conversion.

Canon is Canon

The DSLR video revolution that Canon sparked with the 5D Mark II was an exciting time, and Canon was at the center of it, completely by accident it seems. They had the great good fortune of relenting when Vincent Laforet persistently asked to borrow pre-production units to shoot Reverie, and the rest is history. It seems now Canon is happy to reserve high-end video features for their high-end digital cinema cameras, and to let DSLRs be the stills cameras they were always meant to be. That Sony is the company now exciting us with cutting-edge video features in cameras that are also great for stills is no accident at all, and it’s also OK.

It’s also quite refreshing to see a pre-order button for a fancy new camera that I have no interest in clicking.

Now I’m off to shoot some stills with a full-frame 42 megapixel camera with a tilting LCD, that fits in my pocket.

The Canon 5D Mark IV is available for preorder on Amazon and B&H for shipment on September 8. The Sony a7S II is available now.

Update 2016-08-26

Great comments below. Here’s a brief summary:

If you’re heavily invested in Canon glass and maybe skipped the 5D Mark III, this camera probably seems like a sweet upgrade. I agree, as long as your priority stills, and you’re not yet mirrorless-curious enough for some expensive experimentation.

Cinema 5D has a great hands-on review (this might be the last time their website name makes any sense beyond the historical). Here are some tidbits that struck me:

The Mark IV is 60g lighter than the Mark III. Nice!
The horizontal crop factor for 4k video is 1.64, not the 1.74 diaginal crop factor reported by Newshooter. My own back-of-the-napkin math agrees: 6,720/4,096 = 1.640625. As Samuel H. points out below in the comments, horizontal measure is the only calculation that really makes sense for this kind of thing when comparing images of different aspect ratios.
No peaking for manual focus. This is a big bummer for video, but I’ve also really gotten hooked on using focus peaking with my Sonys for stills. Because it’s so easy to find inexpensive E-mount adapters for almost any type of lens, focus peaking makes it more than just fun t shoot with funky manual focus lenses, it’s actually quite fun, and I have a great success rate with it.
The Neutral picture style comes with sharpening turned all the way down, corroborating my Prolost Flat assertion that in-camera sharpening is always bad.

Capture Everything: Lytro Cinema and the Filmmaking Process

April 12, 2016

When I wrote about the Light L16 camera, we had some fun thinking about a world where one small camera could become just about any camera you could imagine, from a large-sensor DSLR with a fast 50 to a razor-sharp telephoto beast. Where make-or-break factors such as critical focus could be adjusted in post, and where creative decisions like depth of focus could be tweaked in post.

Of course, I mentioned Lytro then. Pioneers in light-field imaging with two curious but ultimately unpopular consumer offerings, Lytro has now “gone pro,” first with a promised VR capture device, and now with Lytro Cinema.

Lytro Cinema is a high-end light-field camera for capturing moving images. It’s (apparently) a massive 6-foot-long behemoth with an imaging plane half a meter wide. It is said to capture 755 raw megapixels (for 4K output) at up to 300 fps, and promises 16 stops of dynamic range.

Lytro is good at getting lots of breathless press coverage of their impressive announcements, but the only real coverage I‘ve seen so far is from fxguide. John Montgomery interviews Lytro’s Jon Karafin on the fxpodcast, and you really should give it an attentive listen.

With the L16, I allowed myself to get excited about what a compact light-field camera could mean for stills. For some reason, I’m more reserved in my reaction to Lytro Cinema. Maybe it’s partially in response to Lytro’s announcement of the Immerge VR capture device, where they presented renderings of a camera but no photos, and faked demonstrations without any disclaimers. I publicly grumbled at the press’s willingness to go along with Lytro’s claims of being “first” at something they had not yet actually shown themselves doing.

But even if we take Lytro at their word that the Lytro Cinema actually exists — and I think we can, as they’ll be showing a film shot with it next week at NAB — I still see reasons to pause for some thoughts about the filmmaking process.

Lytro Cinema promises that most of the decisions typically locked-in at the time of capture, from f-stop and depth of field to shutter angle and frame rate to even the actual position of the camera, would become matters of post-production flexibility. The potentially problematic implications range from obvious to terrifying. A studio could decide well into post that a film should be released in 48 fps 3D, when the filmmakers thought they were making a 24 fps 2D film. The editor could not only zoom and reframe a shot, but even choose to dolly a bit left or right, and add in a rack-focus from one actor to another.

To their credit, Lytro is aware of this possible minefield, and addresses these issues in their interview with fxguide.

Lytro Cinema is not just about postponing or refining creative decisions. Any director would love to save a great take lost due to blown focus, but the post-production superpowers only begin there. As with the L16, scenes shot with the Lytro Cinema camera could have post production flexibility we currently only dream of, from relighting, to object replacement, to perfect mattes without greenscreens, to blending of multiple takes in 3D space. It makes live action filmmaking nearly as fluid and re-editable as a Pixar animation.

As Lytro’s Jon Karafin says in the fxguide interview, “With great power...”

I can’t help but think about the filmmaking process, and what it means to me. My old colleague Alex Lindsay put it very well in a tweet:

So, you could film a scene... or capture all of it... pretty crazy tech... https://t.co/j03c1xrJvM
— alexlindsay (@alexlindsay) April 11, 2016

“Film a scene, or capture all of it.”

To some, the latter seems like nothing but win. But here’s the thing.

I like filming the scene.

I like making decisions and sticking with them.

I like a set where decisions matter.

Over the years, I’ve done a lot of VFX shoots, and one thing I’ve tried to do is avoid greenscreen whenever possible. The reason? When a crew sees a a greenscreen, they could think, at least a little, that their creative work might all be superseded later. It’s a gigantic area of the set over which their creative contribution holds no sway. Any manager will tell you that if you want someone’s best work, you have to empower them, but greenscreens disempower film crews.

We celebrate cinematographers as singular artists, even when their work is a massive collaboration. Can we simultaneously celebrate a device that disempowers their on-set decisions?

The great Vittorio Storaro was recently interviewed (PDF) about his transition to digital cinematography. This passage stood out to me:

Don’t be stuck in just gathering information through this instrument. There is a moment when, as the cinematographer, you have to follow the emotion of the story and go beyond the system.

The analogy that comes to my mind is in recording music. Most modern music is recorded in pieces, with each voice and instrument in isolation. The producer builds the track out of these components, and is, in many ways, the final author of the song. Compare that to a live performance where everyone plays together, at the same time, and riffs off each other.

When we start to think of the camera as an information gatherer rather than a creative tool — an instrument to be remixed with infinite flexibility in an antiseptic room far from the set rather than played in concert with those around you, I wonder what we risk losing in exchange for all that flexibility.

Maybe it will be great, but it’s worth thinking about.

I believe that great films come from great process, and it’s not just OK to choose to make films using the processes you enjoy — it may actually be the only way to make something good.