iPhone

iPhone ProRes Log in Peru and Taiwan

January 30, 2024

This is a blog post about a video, which is about new color-conversion LUTs for Apple Log footage from the iPhone 15 Pro and Pro Max (updated from my first set). The video is also a mini-travelogue of my recent trips to Taiwan and Peru. After my explainer video on Apple Log, this one dives a bit deeper into practical LUT workflows, and my state of mind about shooting digital-cinema-grade footage with a device I always have with me.

There’s a lot going on here.

Or Just Skip to the LUTs

Conflicted in Peru

Stu Maschwitz shooting video of a parade in Lima, Peru, using an iPhone 15 Pro Max with an external drive. Awkwardly, a real camera is also strapped around his neck. — Me relaxing on vacation. Photo by Forest Key.

I always have a moment when packing for a trip: Which camera to bring? Which lenses? I know I’m always happier when I pack less, like just a single prime lens. But sometimes FOMO gets me and I pack three zooms.

For my trip to Lima, I brought my Sony a7RIV with the uninspiring-but-compact Sony 35mm F2.8 prime. I lugged it around for a few days, but wasn’t really feeling it.

Meanwhile, my iPhone 15 Pro Max was calling to me with its ProRes Log video mode. “I’m 10-bit!” It would say. “Think of the fun you’ll have color grading me!”

I told my phone to shut up, and proceeded to shoot very little with it — or my Sony. Like a squirrel in the middle of the street, drawn in two different directions at once, I creatively froze.

Photography, for me, is made up of a lot of habits, and shooting iPhone video with aesthetic intent is just not yet baked into my travel muscle memory.

Made in Taiwan

A month later, I took a family trip to Taiwan, one of my favorite places in the world. I’d had some time to process my Peru deadlock, and decided to stop judging my own creative impulses, and let inspiration guide me in which camera I pulled out.

I wound up shooting a lot of video.

Me relaxing on vacation. Photo by Josh Locker.

I loved shooting ProRes Log in Taiwan with the iPhone 15 Pro Max. I’d occasionally reach for Blackmagic Camera, but I often just used the default camera app. I stuck my phone (with its crumbling case) out of taxi sunroofs and skyscraper windows, held it above teeming crowds and shoved it between chain-link fences. Seeing the broad dynamic range I was capturing in scenarios from noontime sun to neon-lit nights got me excited about grading the footage later.

It’s exactly the way I feel about shooting raw stills with my Sony, knowing that I’ll be able to go crazy on them in Lightroom. The photographing act is just half of the process.

Step through the frames below to see how color transforms a single shot from the video above:

Uncorrected Apple Log

Straight out of the camera. I mean, phone.

Look & LUT

An overall correction applied under PL-HERO LUT.

Local Corrections

Various windowed corrections under the Look help guide the viewer’s focus and give the natural-light capture a cinematic feel.

LUTs, Looks, and Magic Bullets

There’s been a bit of a gold rush of people hawking creative LUTs that apply a particular “look” to iPhone Log footage. My day job is, in part, helping make color tools like Magic Bullet Looks, which can do so much more than any LUT. Creative LUTs are great, and by all means support the folks making them — but that’s not what my iPhone LUTs were or are.

The Prolost iPhone LUTs convert Apple Log to various other color spaces, and support three kinds of workflow:

Grade Under a Display Transform LUT

Apple Log is a totally decent color space to work in, so color correcting Apple Log can be as simple as applying Magic Bullet Colorista and choosing one of my Monitor & Grade LUTs. That’s what you see me doing in the video above. Colorista (set to Log mode) does its work on the native Apple Log pixels, and the LUT converts the result to look nice on video.

Many other systems work like this, including LumaFusion, which ships with Prolost Apple Log LUTs.

The key is color correcting under the LUT.

Bring Apple Log into an Existing Workflow

Color work is often done in an intermediate color space. This is usually some kind of wide-gamut, log format, such as Davinci Wide Gamut/Intermediate, or one of the ACES log spaces.

The Prolost ACES LUTs convert Apple Log to either ACEScc or ACEScct log, allowing you to grade your iPhone footage alongside any other professional camera, and output them all through the same pipeline.

Shooting Through a LUT

The Blackmagic Camera app allows you to load any LUTs you want and preview through them without baking them into your footage. With my LUTs, you can shoot with the same LUTs you grade under later, for a truly professional (no joke!) workflow.

The real stars of this update though are the FC LUTs. They add an informative False Color overlay to the Shoot/Grade LUTs, making sure you always nail your exposure. Watch the video to see them in action. I already can’t imagine shooting without them.

These LUTs work well in Blackmagic Camera or even on an external HDMI monitor.

Adjusting exposure with a variable ND filter until the 18% gray card lights up yellow, for perfect exposure. PL-HERO-FC LUT in Blackmagic Camera.

Gathering Resolve

I’ve never edited a whole actual thing in Resolve before.

As if this video wasn’t enough work already (I shot the A-roll in mid-December), I decided to use it as a personal test-case for creative editorial in DaVinci Resolve. It’s the ACES LUTs that allowed me to incorporate Magic Bullet Looks into my Resolve color workflow.

Maxon just shipped a really nice update to Magic Bullet Looks, with simplified color management made possible by more and more apps we support doing darn fine color management at the timeline level.

So in Resolve, I can use my LUT to convert Apple Log to ACEScc, and then apply Magic Bullet Looks, which can now be set to work in ACEScc with a single click.

The new streamlined color options in Magic Bullet Looks. Choose Custom to get the full manual control.

I can sneak additional Resolve corrector nodes between those two for local corrections. Resolve is great at this, and Looks is great at creative look development, so this is a match made in heaven.

Then, at the end, I use an ACES Transform node to convert to Rec. 709 video.

An expert Resolve user could replace my LUTs with Resolve’s built-in Color Space Transform nodes, but the LUTs make this process easier and more reliable.

Gear Inspires

Every photographer knows the feeling of lusting after new gear. We know it so well that we remind ourselves constantly that “next camera syndrome” is debilitating, and that “most cameras are better than most photographers.” Gear is not the answer. Go shoot.

There is, however, a counterpoint to these truths: As shooters, we take inspiration where we can get it. And sometimes a new technique, a new locale, or even, yes, a new bit of kit is what provides it.

The key is to listen for that inspiration, and don’t judge it.

Even if it’s coming from your phone.

Jiufen village, Taiwan. Two a7RIV shots stitched in Lightroom.

Get the Free LUTs

A still from Apple’s “Behind the scenes: An Apple Event shot on iPhone” video

What Does and Doesn’t Matter about Apple Shooting their October Event on iPhone 15 Pro Max

November 07, 2023

Apple Shot Their “Scary Fast” October Event Video on iPhones And We Had Feelings

You’re somewhere on the spectrum of occasionally shooting video on your iPhone to a professional-ish video maker with some gear, and you see at the end of Apple’s October “Scary Fast” event announcing new Macs with M3 silicon that the entire event was “Shot on iPhone.”

This makes you feel a certain way.

Actual footage of me watching the event.

Then Apple posts a behind-the-scenes video showing how this was done, revealing a rare and imposing glimpse into the scale and scope of their industry-leading launch videos. At the center of it all, instead of their customary Arri Alexa (a digital cinema camera costing $35–150K before you even add a lens, used to shoot everything from Avengers: Endgame to Barbie), was an off-the-shelf iPhone 15 Pro Max, gripped into truckloads of professional support gear.

At this point, some folks felt differently about what was implied by “Shot on iPhone.” There have been bad takes on this, and good takes on those bad takes.

Anyone who knows the tiniest bit about video production knows that the camera is a small, but important, but small, part of the overall production picture. “Shot on iPhone” doesn’t promise “and you can do it too” any more than Stanley Kubrick lighting Barry Lyndon with candlelight means anyone with candles can make Barry Lyndon.

But when the camera is the least expensive piece of gear on the set after clothespins and coffee, it does feel strange. I’ve been on a lot of productions like this, having played an active role in the DV filmmaking revolution of the late ’90s-to-early-2000s. It was an odd feeling to scrounge for the adapter plates required to mount a $3,000 DV camcorder purchased at Circuit City to a Fisher dolly that literally has no purchase price.

Apple, of course, has no burden of best-practices logic for their decision to shoot their “Scary Fast” event on iPhone — it’s a marketing ploy, a cool stunt, and a massive flex. A thing to do for its own sake. In the filmmaking community, it was the mic drop of the year. We greedily soaked up all the details in the behind-the-scenes video, and made a hundred tiny calculations about which aspects of this lavish production actually mattered to the question of the iPhone 15’s validity as a professional camera, and which did not.

With all that gear and production support, which aspects of the event really matter to you, the iPhone-curious filmmaker? What can you learn, and which aspects can you safely ignore?

Let’s take it one at a time:

That They Did It: Does Matter

As camera features have played a larger and larger role in Apple’s marketing for new iPhones over the years, you might have begun to feel a bit of cognitive dissonance. Apple tells you about how great, and even “pro,” these new iPhone cameras are — but would never have dreamt of using them to shoot their own videos or product stills. Apple was effectively saying “pro enough for you, but not for us.” Valid, but a bit dissatisfying.

A still frame from Apple’s October 30 “Scary Fast” event video, shot on iPhone 15 Pro Max.

Apple has set the aesthetic bar impossibly high with these pre-recorded events. They’re not just executives teleprompting in front of Keynote slides — they feature “slice of life” moments shot on huge sets and real locations. Elaborate visual effects transition between locations and settings that might be partially virtual. These videos have looked great ever since Covid pushed Apple to find an alternative to executives-on-stage-in-front-of-slides, and even as Apple is now once again able to welcome guests to in-person product launches, these lavishly-produced videos are the new gold standard in pitching the world on a new iThing.

With “Scary Fast,” Apple repeated their now well-established high-production-value playbook, but yoinked out the professional cameras and lenses, and dropped in a commodity consumer telephone in their place.

And crucially, none of us noticed.

It’s a big deal.

They Shot ProRes Log: Matters So Much It Would Be Impossible Without It

There is one single feature of the iPhone 15 Pro that made this stunt possible: Log. As I detailed here in words and video, the “iPhone video look” is designed to win consumer popularity contests, not mimic Apple’s own marketing videos, nor plug into professional workflows.

It may be hard to imagine that a slightly different bit of signal processing when recording a video file from a tiny sensor can make the difference between consumer birthday-cam and professional viability, but that is exactly the power of log. Apple Log has catapulted the iPhone into filmmaking legitimacy.

They Used Big Lights: Does Matter, With An Asterisk

Apple’s event was set at night, with a dark, Halloween-inspired look. It takes a lot of professional lighting gear to illuminate a wide shot of Apple’s campus, and professional skill to balance this lighting with the practical sources on the building itself.

You can do this for cheaper than it looks.

Lighting matters more than any camera, more than any lens. As I wrote in 2009:

Photos are nothing but light — it’s literally all they are made of. Timmy’s birthday and Sally’s wedding are reduced to nothing but photons before they become photographs. So getting the light right is more meaningful to a photo than anything else.

Should you look at the giant lights in Apple’s video and feel dejected that your own productions will never afford this level of illumination? I say no, because a) you’re probably not lighting up the whole side of an architectural marvel, and b) you’re probably not designing your production around one of the world’s highest-paid CEOs.

For Tim Cook’s appearance, Apple’s production had their giant LED light panels on camera dollies, which is not typical. The two reasons I can image they did this are to be low-impact on the campus itself (rubber wheels instead of metal stands), and to be able to adjust the lighting quickly out of respect for Cook’s valuable time. It makes the lighting rigs seem more complex than they really are.

What they really are is big, bright, and soft. And rather minimalistic — mostly key, a bit of fill.

Big, soft LED lighting is actually quite affordable these days. I have two medium-power bi-color lights from Aputure, and together they cost less than my iPhone 15 Pro Max. I couldn’t cover Cook’s opening wide shot with them, but I could get close.

That big softbox overhead? Now *that’s* expensive.

I might also be willing to compromise on my ISO settings to work with a smaller lighting package, where Apple seemingly was not. More on this below.

So the lighting is important, but the quantity of it and the support gear it’s on is specific to this rarified type of time-is-money, night-exterior production. Don’t be distracted by the extra equipment, focus on the fact that the lighting itself is actually rather spare.

They Attached the iPhone to Cranes and Gimbals and Drones and Dollies: Does Not Matter, Except for One Little Thing

The behind-the-scenes video is almost comical in its portrayal of the iPhone gripped into all manner of high-end support gear.

You do not need any of this stuff.

I mean, every filmmaker needs a crane shot — but this is why small cameras are so empowering: everything is a crane when your camera weighs less than a Panavision lens cap!

Check out this video from filmmaker Brandon Li. He uses a gimbal on a hand-held pole to create a perfect crane shot for the opening of his action short. Toward the end, he achieves a nifty top-down shot by... standing on a railing. All with a camera substantially more cumbersome than a phone.

Apple used cranes and remote heads designed for big cameras because that’s how they know how to shoot these videos. Apple’s marketing department is large, and knows exactly what they need on these productions. One thing they need is for a dozen people to watch the camera feed, making sure everything is committee-approved perfect.

This is just the DIT cart, not even the client monitor.

This kind of client-driven support gear compounds on its own requirements. As Tyler Stalman points out in his excellent breakdown video, some of what’s bolted to the iPhones is simply a metal counterweight so that a gimbal head, designed for a much larger camera, can be properly balanced.

You can plug an external drive into the USB-C slot on the iPhone 15 Pro Max, or you can plug in an HDMI adapter for a clean client feed. If you want to do both, you need a USB-C hub, which at that point requires power. So now you’ve got an Anton-Bauer battery pack mounted to this tangle of gear.

When you don’t have clients, you can skip all that and just shoot. This means you can replace most of the gear you see here with a cheap consumer gimbal — or a tripod.

And here’s the key takeaway for this point: Apple achieved optimal image quality from the iPhone in a number of ways, and one, I’m betting, was by turning off image stabilization — which is only advisable when this tiny camera is physically stabilized.

So you don’t need all the stuff Apple used, but if you want comparable results, you need a way to mount your iPhone to something solid. Maybe not a whole powered cage, but certainly a simple tripod mount. Then you can eek out that last bit of extra image quality by turning off image stabilization — which brings us to our next point:

They Used the Blackmagic Camera App: Matters as Much as Log

I don’t think this exact rig was used to capture what we saw in the video, but I believe the settings are representative.

The Blackmagic Camera app has the option to turn off image stabilization, yes, and also like a million other features. Manual, locking exposure control is the top of the list, but there’s a ton more. The app includes a false-color mode to help match exposure from shot to shot. It can load a preview LUT, so you can shoot log but view something closer to what the audience will see.

It’s silly to be grumpy with Apple for not offering this power in their own camera app when they clearly worked with Blackmagic Design to have this app available day-and-date with the iPhone 15.

Oh, and it’s free.

They Used a 180º Shutter: Matters More Than You Think

One slick feature of the Blackmagic Camera App is that you can choose to express the shutter speed in degrees, like a cinema camera, rather than fractions of a second, which is more typical in stills. A 180º shutter — where the shutter is open for half the duration of a single frame, e.g. 1/60th of a second at 30 fps — is important for a pro look. Anything slower and you get smeary blur and a camcorder look. Anything faster and your footage looks like you shot it on a phone, because 99% of the time our iPhones are using insanely fast shutter speeds to handle typical daylight. Look at any of your own daytime iPhone video — I’d be surprised if you see any motion blur at all.

Compare Apple’s motion blur to mine. 180º shutter vs. probably something more like 1º.

Relatedly:

They Shot at ISO 55: Matters to Apple’s Goal of Maximum Image Quality

Here’s where the level of professional control over the lighting starts to really matter: If Apple decided that they must shoot at ISO 55 (the lowest, although possibly not the native ISO of the 1x camera) for the highest image quality, and with a 180º shutter for the most pro-camera look, that means they have no other control over exposure. The iPhone 15 Pro 1x lens does not have a variable aperture, so shutter speed and ISO are your only exposure controls.

When shooting in uncontrolled environments, the typical method of limiting the amount of light entering the lens is via ND filters, sometimes variable ND filters. I don’t see any evidence that Apple used filters on this shoot, which would fit with their overall prioritization of image quality over all else. So this goes back to lighting — Apple’s team controlled that lighting perfectly, because they opted out of any exposure control they might have had in-camera.

I'm curious to learn more about this setting though. YouTubers Gerald Undone and Patrick Tomasso did some tests and found that the best dynamic range from the iPhone 15 Pro came from ISO 1100–1450, with 1250 being their recommended sweet spot. Did Apple prioritize low noise over dynamic range?

They shot 30p: Doesn’t Matter

Apple has used 30 frames-per-second for these pre-recorded keynotes since they started in September of 2020. They’re not trying to be “cinematic,” they’re trying to make a nice, clean video that can take the place of a live event. 30p is a choice, and a fine one for an on-stage presentation. You might choose 24 or 25 fps for a more narrative look, and that’s great too.

Note that Apple’s native Camera app offers 30.0 and 24.0 fps, but the Blackmagic Camera app adds support for 29.97 and 23.976 fps, which are the actual broadcast frame rates Apple uses for their productions.

They Focused Manually: Doesn’t Matter

The Blackmagic Camera app truly has a dizzying set of features, some seemingly part of an attempt to win some kind of bizarre bet. Like support for wireless external follow-focus controls? I mean, wow, but also, really?

Sure makes for a cool behind-the-scenes shot, but I bet you can live without this.

They Used a Matte Box: Does Matter

While Apple did not attach any additional lenses to their production iPhones, they did put stuff in front of the built-in lenses — notably teleprompters, of course, and comically-large matte boxes.

Matte boxes might feel like affectations in this context, but shielding that tiny lens from glare is actually a significant way to improve overall image quality. Luckily, you don’t really need a full-on matte box to do this. A French flag will do it, as will your hand.

If a bright light is dinging your little lens, you’re leaving a ton of image quality on the floor.

They Exclusively Used the 1x Camera: Does Matter — to Apple

The 1x camera, with image stabilization turned off, gives the highest-quality image available from the iPhone 15 Pro Max. Are you detecting a theme here? Apple imposed a number of limitations on how they used the iPhone camera, seemingly always in the name of maximizing image quality.

As we’ll discuss below, you may or may not share this priority.

They Edited in Premiere Pro? Doesn’t Matter

Eagle-eyed viewers noticed that an Adobe Premiere Pro timeline appears behind the editor of “Scary Fast,” not Apple’s own Final Cut Pro. But we’re not in a real edit suite here — we’re actually on one of the sets from the production. Did Apple really edit in Premiere?

I have a feeling that this and Stefan Sonnenfeld’s interview were staged on Apple’s standing sets rather than in real studio environements to keep the production close to home — for cost, control, and secrecy reasons. So let’s assume Apple really did cut in Premiere. This means next to nothing. All editing software does the same job, and it’s unlikely Apple would impose a workflow on the production company they hired to both shoot and post-produce the video.

Other than of course to ensure that it be cut on a Mac. It’s interesting to note that Apple’s Pro Workflows Group, representatives from which are interviewed in the behind-the-scenes video, are a part of the hardware division at Apple. Their charter is to promote and support professional use of Apple devices, regardless of which software they’re running.

Should Final Cut Pro users be nervous that Apple might send it to live on a farm with Shake and Aperture? It’s hard to regain our trust here, but Apple did just release a very nice version for iPad a few months ago, and substantial updates to that and the Mac version just yesterday.

So there’s really nothing to see here. Move along.

They Colored in Resolve: Doesn’t Matter

Apple hired Company 3 to produce this video. Company 3 is best-known as a color house. In my visual effects and commercial directing career. I’ve worked with several amazing colorists there, from Dave Hussey to Siggy Ferstl to their CEO, Stefan Sonnenfeld, who is prominently featured in the behind-the-scenes. Stefan is one of the most prolific, talented, and well-known colorists working today. I modeled half the presets in Magic Bullet Looks after his famous grades on films like Transformers, 300, John Wick, Man on Fire, and hundreds more.

If you can get Stefan to color your video, that’s what matters — not the tool he uses. Resolve is “free” (with many asterisks), but a session at Company 3 is four-figures per hour.

Whether you use Resolve, Magic Bullet, or whatever else, what matters here is that shooting log means color grading is not just possible, but essential, and great care was taken with this part of the process.

This All Makes Sense. Why Do I Still Feel Weird About it?

As much as I might disagree with an accusation that Apple was disingenuous to say “Shot on iPhone” about a massive production with seemingly unlimited resources, I understand where this feeling comes from. “Shot on iPhone” carries with it the implication of accessibility. We are meant to be inspired by this phrase to use a tool that we already carry with us every day to capture videos that might transcend mere records of our life’s moments, and become part of our artistic pursuits.

And we should absolutely feel that way about the iPhone 15 Pro and Pro Max.

The reason we feel slightly disconnected from Apple’s impressive exercise is not that they were dishonest — it's that their priorities were different from ours. Apple wants to sell iPhones, and to accomplish this, they spared no expense to put the very highest image quality on the screen.

As a filmmaker, you care about image quality, but you care about other things too — probably more.

Sticking to the 1x lens gives the best image quality, sure, but choosing the right lens for the story you’re telling might matter more to a filmmaker. You’ll probably gleefully use all the focal lengths Apple supplied.

As much as you might value the clean image that comes from shooting at ISO 55, you might get more production value for your budget by using smaller lights (or available light!) and accepting some noise at higher ISOs.

You might truly appreciate the value of a 180º shutter, and simply not always have a camera case/cage that allows you to mount a variable ND filter to your telephone. I ordered one from PolarPro the day the iPhone 15 was released and it still hasn’t shipped, so I’ve shot next to no 180º shutter footage so far.

You might well understand that turning off image stabilization will improve your image — unless your shot is now wobbly, because you’re shooting handheld out the window of a moving car. So maybe you’ll leave stabilization on, and be a human crane, or gimbal, or dolly, or all of the above.

Never use the 5x lens. Never use image stabilization. Never shoot through a dirty windshield. And never get this dope-ass shot.

Let’s be honest: If image quality is your top priority, there are much better options for your next production than a consumer telephone. You’d probably choose the iPhone more for its accessibility, nimbleness, ubiquity, and low cost. Those are great reasons, and when you pair them with image quality that can be mistaken for high-end cinema camera footage by a veteran colorist, you’ve got something magic.

Give It To Me In Bullets You Long-winded Monster

So we may well ignore much of Apple’s implied advice, but we would do well to follow some of it if we can:

Use camera support. Not crazy camera support, but some.
Use lights. Not crazy lights, but some.
Use a camera app that allows manual control, like Blackmagic Camera
Use 180° shutter, if you can (ND filters will help)
Keep light off of the lens using a $8,000 matte box or a bit of black tape
Hire literally the world’s most famous colorist. Or just do some color correction.
And most importantly, shoot in log, with a good preview LUT

The incredible production apparatus of my helicopter tunnel shot from my last post.

Shot on iPhone Means What it Means to You

All of this is academic if you don’t go put it into practice. If you got this far and feel empowered to wring the most out of your iPhone 15 Pro with just the right amount of gear, that’s great. If you actively forget all of this and occasionally flip on the Log switch so you can play with the color of your iPhone videos in post, that’s great too.

Because here’s the thing: movies have already been shot on phones. No production’s decisions validate any camera for all other production needs. You decide what “Shot on iPhone” means to you, if anything. And the way you decide is by getting out there and shooting something.

I went to Peru with nothing but my iPhone 15 Pro Max. I shot some stuff with zero gear, and I’m having a blast color grading it various ways.

Lightroom Adds Video Color Editing, with Prolost Presets

June 15, 2022

From the Lightroom Blog:

The same edit controls that you already use to make your photography shine can now be used with your videos as well! Not only can you use Lightroom’s editing capabilities to make your video clips look their best, you can also copy and paste edit settings between photos and videos, allowing you to achieve a consistent aesthetic across both your photos and videos. Presets, including Premium Presets and Lightroom’s AI-powered Recommended Presets, can also be used with videos. Lightroom also allows you to trim off the beginning or end of a video clip to highlight the part of the video that is most important.

Premium Presets in Lightroom for iPhone.

Not all of Lightroom’s color controls are working for video, but it’s enough to be useful. Not every video workflow requires a timeline and dedicated color tools, so I welcome these features.

In fact, I was graciously invited to contribute to them, in the form of a set of Premium Presets available to Creative Cloud subscribers.

Update Lightroom to the latest version to get the new video features, and find my presets under Video: Creative.

New Photography Shortcuts Using ToolBox Pro

November 14, 2019

My love affair with Apple’s Shortcuts took a bit of a hit during the transition to iOS 13, but my fascination with this on-device development environment has been rekindled thanks to a new app called ToolBox Pro.

ToolBox Pro is a free iOS/iPadOS app that adds powerful new actions to Apple Shortcuts. An in-app purchase unlocks even more functionality. For the definitive write-up on this amazing app, see Federico Vitici’s exhaustive review on MacStories.

What I love about Shortcuts is that it lets me make genuinely useful tools that probably couldn’t sustain the overhead and/or dubious business model of being actual apps. I’ve shared a few of these free Shortcuts at the Prolost Store, and today I’m adding two new ones that take advantage of free features in ToolBox Pro.

Burst Slice

Burst Slice converts a photo burst from the iPhone/iPad camera into a tiled panorama. You control the size of the “slices,” which frames to use, and even the start and end crop of the slices.

I created Burst Slice before ToolBox Pro was available, but the experience wasn’t great, because Shortcuts can’t show you an image and a menu at the same time. ToolBoxPro allowed me to create a simple, visual interface to adjust the many parameters of this fun and useful tool.

Pano Man

Pano Man creates tiled panoramas for Instagram that can include multiple images. It uses that same visual UI functionality to give you pushbutton control over the padding, spacing, and cropping of the tiles.

I hope you find these tools useful. I welcome any feedback on them, and am always happy to see examples of them in use!

Apple Shortcuts: The Bicycle for the Mind is Back, but it’s Electric

October 22, 2018

Apple Embraces Tinkering for the First Time in 20 Years

Steve Jobs famously described the computer as “a bicycle for our minds.” Whether or not you’ve heard that before, it’s worth refreshing your memory on the context by watching this short video. The gist is that humans aren’t very efficient at getting around when compared to the rest of the animal kingdom, but because we are “tool builders,” we can more than make up for it. The tools we create magnify our capabilities. They make us better versions of ourselves. And the computer, Jobs believed, is “the most remarkable tool that we’ve ever come up with.” So what a bicycle can do for our bodies, a computer can do for our minds.

I love this analogy for many reasons, but primarily because it’s not simple. It’s far-reaching and it works from many different angles. One thing I draw from it, that others may or may not, is that the bicycle-like properties of the computer are strongest for those who use this remarkable tool to make tools of their own. A computer is a bicycle that makes bicycles, and a human is most magnified when they are a tool builder, not just a tool user.

To me, whether Jobs intended it this way or not, the “bicycle for the mind” is the tool that empowers you to repurpose it for your specific needs, not just to consume things with it, or use it in the same way as everyone else.

Or, in the words of former Apple Product Manager of Automation Technologies Sal Soghoian, “the power of the computer should reside in the hands of the one using it.”

Creators Build Tools

Many people use their computers for creative work, but, as I covered in my last post, everyone who does any kind of work also does some kind of meta-work, creating tools and systems that support their process. You might build a small business entirely around pre-packaged services like eBay or Shopify or Squarespace, but at some point you’re going to go beyond just using computers and services and start creating tools to make your work easier. This might be as simple as a customized spreadsheet to model your business plan, or a set of TextExpander snippets that automate your customer support replies. Or it could be as complicated as a multi-step Photoshop Action, or even a proprietary asset management system.

Creators build tools, which means they need tools for building tools.

Computers, Hammers, and Saws

The first home computers were tools, but also expressly and proudly tool-creation devices. The blinking cursor on an Apple II or Commodore 64 was as much an invitation to create your own BASIC program as it was to load and run someone else’s.

Early home computers were powerful for the time, but also simple. As combination tools-and-tool-creation-systems, they were a lot like a hammer or a saw that you might find in a woodshop. You can use a hammer and saw to build a bookshelf, or you can use them to build a jig that will speed up your production of a dozen bookshelves.

You may discover that a jig can speed up your work by tinkering away with the tools you use every day. Tinkering is a word I use a lot. It was years of tinkering with expressions in After Effects that led me to create my short film TANK. Tinkering with color correction led directly to Magic Bullet. Tinkering means experimenting, trying things out, and expanding your capabilities. Going farther.

In a woodshop, the same tools you use to do the work are also the tools you use to to the meta-work of automation. This means the woodworker is fully empowered to build their own tools. And they inevitably will, thanks to the power of tinkering.

Two Bicycle-free Decades

Early home computers shipped with BASIC. Early Macintosh computers shipped with Hypercard, which you were invited to use to create recipe books and home inventories, but was also famously used to create the hugely-popular game Myst.

In 1998, marking the triumphant return of Steve Jobs, Apple released the iMac G3. The “i” was for Internet, and this computer that would come to define the new Apple was in every way a consumer device, designed for every kind of person to use, not just computer nerds. Instead of a blinking cursor, it prominently featured the Internet Explorer web browser. The emphasis was on using the tool, on consuming content from the web.

This same year, Apple discontinued Hypercard. The iMac — the foundational device of the Apple we know today — was the fist Mac that didn’t come with an obvious way to customize its capabilities. It was a tool, but not fundamentally a tool-making tool.

And that would set the trend for Apple devices for the next twenty years.

“Anyone Can Code”

No, they can’t. Not just anyone can make an app. Exhibit A: People don’t.

That first iMac did ship with AppleScript, a well-intentioned, if misguided attempt to dress up code as plain English. OS X shipped with a terminal app that could run bash and Python.

Scripting is powerful, and significantly less intimidating than coding an app. But scripting is still coding, and most people find writing code intimidating, becuase it has nothing to do with their creative work.

No normal people ever used these tools to make their computer-centric work easier. Building apps, large or small, became exclusively the domain of programers. A rift was starting to form between computer users and the super-users who could repurpose the computer to do their will.

Xcode is how you make iOS apps, and it’s about as anyone-friendly as an Apollo capsule.

Programing is great, but it’s not using a hammer to build a jig. It’s using a hammer-factory to build a hammer. When the tool-making tools don’t look anything like the work-making-tools, the worker is disempowered. Unless your work is making code, coding isn’t tinkering.

Computers from 1998 to the present day have disempowered their users by not shipping with simple, easy-to-use tools for tinkering.

But What About Automator?

I’m being a little unfair with that last assertion. Apple did take a stab at code-free scripting in OS X, with an app called Automator. It first shipped with Mac OS X Tiger in 2005, and on the surface, it would appear to be exactly the kind of tool-making tool that I claim has been missing since Hypercard.

Automator’s express mission was to allow people to create custom tools using a graphical model of stringing actions together. It would seem, on the surface, to be exactly what I’m asking for.

I realize that to some, the above screenshot seems as intimidating as the Xcode screenshot. But there's a key difference: Just about anyone can tell what is happening in the Automator workflow above. When you download an Automator workflow, you can open and dissect it like this, and see how it works. This gets back to my “blinking cursor” theory of computers that natively ran BASIC: If running it is a lot like writing it, you can learn by example and tinker your way to the next level.

Automator was a great idea that seemed to be held back by a lack of support, or interest, within Apple. It can do a lot, but never quite enough to become an essential part of my workflow. The few times I tried making anything useful with Automator, I would inevitably hit a wall, and resort to scripting. Any useful Automator workflow I ever got my hands on was essentially a wrapper around a script.

As just one concrete example, Automator has nothing like an if/then action, so there’s no way to do the simplest kind of flow control. Here’s an Apple support thread about, you guessed it, working around that limitation with scripting.

Also, it sounds glib to say it, but I’m sincere when I point out that no one ever made Myst with Automator. Automating repetitive tasks is fine, but we’re all better off when our tool-making tools allow us to have fun, to express our creativity, and to make art, games, and stories — not just resize and rename a bunch of JPEGs.

Third Parties Step Up as Best They Can...

There are several great Mac and iOS automation tools from third-party developers.

TextExpander is an easy first step into automation for many. It allows you to type something short, which gets replaced with something long. If you email Slugline support and I sign off my reply with “Feedback like this helps us prioritize future development, so thanks again for getting in touch!” I hope you don’t mind knowing that I don’t type that out every time by hand.

Keyboard Maestro is a deep and powerful tool for automation on Mac. I’ve been using it a ton since the discontinuation of Quadro for iPad, and I’m sure you’ll hear more from me about it.

Hazel is another wickedly useful Mac tool for Automation. I use it for simple things, like tidying up my downloads folder, but I also built a remote render-monitoring system with it for TANK.

In 2015, a company called DeskConnect released an ambitious iOS app called Workflow. The easiest way to describe Workflow is “Automator for iOS.” To my great surprise, where I struggled to do useful things with Automator, I immediately found myself creating useful tools with Workflow.

I also had a similar reaction to many others, which was that Workflow was so awesome and so powerful that I genuinely wondered if Apple would continue to allow it in the App Store.

Workflow. I stole this image from macrumors.com.

On iOS there’s Drafts, which offers automation for text. It’s a syncing notes platform that can spit out your text in all kinds of interesting and powerful ways.

Also on iOS there’s an amazing text editing app called Editorial that has an Automator-like UI for processing your text. It’s wonderful, but it hasn’t been updated in a long time.

Editorial has some nice features in its worklfow editor, such as collapsing and color-coding actions.

It wouldn’t surprise me if the creator of Editorial had trouble building a business model around it. I know first-hand how hard it is to make money with a productivity app. I’ve never invested too much effort into making anything complex with Editorial, because I didn’t know many other writers who used it. As much as I love TextExpander and Keyboard Maestro, the topic of discussion here really is about automation tools that ship with the computer, like BASIC and Hypercard did. Attempts to keep Hypercard alive as a paid product failed. The bicycle-making capability, I believe, must be ubiquitous. It must be fundamental to the computer, like a blinking cursor.

But unfortunately, in 2016, Apple took a very public step away from this idea.

...While Apple Steps Down

In 2016, Apple eliminated Sal Soghoian’s position, prompting him to leave the company. Here’s what John Gruber of Daring Fireball said about that:

...a huge part of my argument for why I feel so much more productive on a Mac than an iPad revolves around the automation technologies that Soghoian’s group developed. [...] I find this to be a profoundly worrisome turn of events for the future of the Mac. I hope I’m wrong.

Gruber wasn’t wrong. Automator hasn’t been noticeably updated since Sal left Apple, and the Mac still suffers from a lack of tinkering tools, and a massive rift between users and coders.

But while Gruber was right to be concerned that the Mac has not taken any steps forward in empowering its users to create tools, his iPad (along with his iPhone) has, and profoundly so.

Taking Shortcuts

That iOS app I mentioned above called Workflow? Apple did not kick it out of the App Store for being too powerful. Instead, in 2017, they acquired it. The reaction from the community was hopeful, but I and many others found it hard to be optimistic. Apple has a history of acqui-hiring, and of killing off beloved professional apps like Aperture and Shake.

But at their 2018 WWDC conference, Apple revealed the best possible news for Workflow fans. Workflow, under the new name Shortcuts, would be updated and integrated deeply with Siri in iOS 12.

Apple was not just going to ship an automation tool. They were publicly tying it to Siri, one of their most important initiatives. Apple shouted from their biggest stage that automation was a key part of the future of iOS.

When I watched this Keynote, I think my legs started pedaling involuntarily.

Shortcuts, Shortcuts, and Shortcuts

Apple did make the namespace a little confusing by renaming Workflow to Shortcuts. iOS 12 also debuted the idea of “Siri Shortcuts,” which are hooks that apps can add to allow Siri-based voice access to specific features. This is separate from, but related to, the Shortcuts app, which is literally just an updated version of Workflow. With it, you can download and/or create Shortcuts, which are graphical mini-scripts that can do all kinds of useful things on your iOS device.

Confusing? Sure. But wait a minute: Download and/or create. Shortcuts encourages you to do both equally. To dissect and learn, or to jump in and tinker. In other words, it’s the 2018 equivalent of the blinking BASIC cursor.

Bicycling

The bicycle is back, and I couldn’t be more excited.

Do you, like many people, have a love/hate relationship with your smartphone? Maybe you agree with Mat Honan, who wrote in his hilarious review of the Google Pixel 3, “When I am not looking at my phone, I become slightly anxious. And then, when I do actually look at it, I become even more so.”

Shortcuts, the app, is my favorite thing to happen to the iPhone since the camera got decent. I truly feel that my relationship to my iPhone has fundamentally changed, because I can now build things on it that make it do things that only I would ever want. Useful things, but also fun things. Complex things and simple things.

My wife was annoyed that her iPhone would make noisy alert sounds when she was trying to take a nap. I created a super-simple shortcut that asks how long she plans on napping, and turns on Do Not Disturb for that amount of time. It took five minutes to create and share with her via AirDrop.

The entire source of Naptime, and what it looks like running in Shortcuts.

I had an idea for an app that would help me prep portrait-orientations images for Instagram. The problem is, there’s no way such an app could ever be profitable. I didn’t really want to make money with it, but I couldn’t afford to hire a developer to make it, so the idea sat. Now I’ve tinkered it into existence as a Shortcut.

Instamax add white borders to photos that are too tall for Instagram.

Prior to Shortcuts, the only Apple-supplied way to solve this problem would have been with a Mac running Xcode, and all the encumbrance of building a custom iPhone application. With Shortcuts I was able to tackle this problem while sipping a beer at my favorite lunch spot.

For this genuine use case, the iPhone is now its own development platform.

There’s so much you can do with Shortcuts that it’s difficult to know where to start. My recommendation is that you start very simple. Like maybe so simple that you wonder if it’s even worth doing.

There’s a pizza place near my office that serves one kind of pizza each day. They post it on their website in a consistent enough way that I was able to make a super-simple shortcut to extract just that information. I assigned it a Siri phrase, so now I can say to my phone, “Hey Siri, what’s the pizza of the day?”

My pizza shortcut running in Shortcuts, and via Siri.

Shortcuts can also do useful things with images beyond just cropping. These pretty screenshots adorning this post? I made them with another shortcut I put together that masks iPhone screenshots and arranges them in a row.

Big Tennis Screenshots takes your iPhone X, XS, and XS Max screenshots and makes them round and pretty. I love my curvy screenshots.

And hey, it’s no Myst, but I even made a shortcut that turns plain text int a simple interactive “Choose Your Own Adventure” story.

Mini Adventure is a very short game, and a very simple interactive story engine.

Pushing the Limits

Making a simple shortcut is easy. I showed my 9-year-old son how to do it, and before long he’d created a short quiz to test my knowledge of Pokemon and marine biology, and mock me for getting answers wrong. I asked him if programing a shortcut felt like building in Minecraft. He said “No, it’s much easier.”

It’s also possible to make deep and complex shortcuts. I made one called Next Small Thing that attempts to be a genuine project-based to-do list manager, with the unique feature of only showing you the next available task, so as not to overwhelm you — a feature that tends to only be available in overwhelming to-do apps.

Send any iPhone text to Next Small Thing, and it becomes a project that you can deal with one action at a time.

The most complex shortcut I’ve made so far is called Drinking Buddy. It logs your alcohol consumption and gives you a rolling calculation of your Blood Alcohol Level, which you can optionally record to Apple's Health app. It started simple, but I kept tinkering, and before long I had something almost app-like.

Drinking Buddy running as a Widget in the iPhone Today View.

A Touch Painful on Reentry

But shortcuts are not apps, as you will be constantly reminded when using them. They feel most app-like when run from their Today View widgets, like Drinking Buddy above. Alas, just as often, you’ll see their alerts and menus presented over the Shortcuts library, or even over the actual guts of the shortcut.

If you send text from Notes to my Next Small Thing shortcut, you’ll see the actions of the shortcut displayed behind the text field. If you press Cancel, you’re left staring at the equivalent of source code.

Shortcuts can be a bit brittle, and tricky to troubleshoot. Living with them can feel a bit like breakfast with Wallace & Gromit, where the trappings and mechanisms of your automation are dangling in plain view, and you may spend more time tinkering with them than you ostensibly save by using them.

The power of Shortcuts to tie apps together can be both a strength and a limitation. One of my favorite shortcuts is also one that you’re least likely to try out, because it requires up to four apps and services to work. I use Gear Checklist to build custom checklists for all my photo and video shoots. Based off a master equipment list stored in either iCloud Drive or Bear, you select a project name and prep date, and then pick which items you need for the shoot. The shortcut then creates a project in Things, my to-do list app of choice. But to do so, it relies on the aforementioned Drafts, and an extension to Drafts called Things Parser. If you’re willing to install all this stuff and configure it, you, too, can conveniently create photo gear checklists at the touch of a button.

UPDATE: I’ve updated Gear Checklist to work without Drafts and Things Parser. Now all you need is Things.

Gear Checklist is the most genuinely useful Shortcut I’ve made.

Shortcuts can be run as Action Extensions (the things that show up when you press a Share button in iOS), which is very cool and powerful. For example, you can find a photo in the Photos app and share it to my Panogram shortcut that splits wide images up into tiles. But if you do, Panogram will probably crash, seemingly for lack of memory. This seems a bit silly on a phone with 4GB of RAM. Running Panogram from the shortcuts app works fine.

Panogram splits a photo into tiles that can be reassembled into a multi-image post in Instagram.

There are some simple things that Shortcuts just can’t do, like automatically run at a particular time, or set an alarm, or create a contact. There’s no way to copy/paste actions in a Shortcut, nor are there any tools for navigating your list of actions, so creating complex shortcuts requires patience and a lot of scrolling.

The good news is, a strong and vocal community has formed around Shortcuts. Apple must be noticing just how deep a chord they’ve struck with this addition. What we have now is truly just the beginning, and it’s wonderful to have reason to believe that Apple is committed to placing the power of the (tiny, portable, and ever-present) computer into the hands of the ones using it.

Back to Bicycles

So, is the “bicycle for the mind” back? I think so, but with a little caveat.

Job’s bicycle analogy was about efficiency of locomotion. Without a bicycle, we’re highly inefficient animals at just getting around, but we can build tools, like bicycles, which put us at the top of the list.

But I also take from it that riding a bicycle is good for you. It makes you stronger. Buying a tool and using it is like driving a car — you’ll get to your destination, and efficiently, but you’ve done nothing to better yourself. Every day that car will take you exactly the same distance.

But each day you ride a bicycle, your legs get stronger. You get where you’re going, sure, but maybe more importantly, the more you ride, the farther you can go.

This is the way I look at the Jobs bicycle analogy: When you build your own tools, you make your mind stronger, and able to go farther the next day.

As I use Shortcuts more and more, I feel myself somewhere between driving a car made of apps built by other people and riding a bicycle of my own creation. I’m creating genuinely useful tools, and I’m pushing myself farther each day I ride. But I’m also just using someone else’s app. I’m not sure if I’m truly living Sal Soghoian’s dream of the power of the computer residing in the hands of the user.

But I know something fundamental has changed in my relationship with my iPhone. It’s now something I can tinker with, in a way Apple hasn’t put their full weight behind for 20 years. The result, for me, is that my phone is less anxiety-inducing. It feels more personal, more mine. At times, it’s downright inspiring.

So, as I said on Twitter, Shortcuts is like an electric bicycle — maybe you’re not getting quite as much exercise as you should, but it’s better than driving.

Apple should be proud that they’re back in the bicycle business. Maybe, just maybe, the key to keeping our phones from ruining our lives is to put the power to control them back in our hands.

Download the Shortcuts Mentioned Here

Update 2021-10-11

We’re now on iOS/iPadOS 15 and this post is overdue for an update. Many of the limitations I detailed above have been amply addressed, as Shortcuts has been aggressively updated each year. That’s the good news—the bad news is that with the updates have come bugs. But the important long-term takeaway comes from this year’s WWDC announcement that Shortcuts is coming to macOS, and there’s even a rich migration path from Automator to Shortcuts. Craig Federighi, Apple's Senior Vice President of Software Engineering, put it in no uncertain terms when he said “Shortcuts is the future of automation on Mac.”