Smoking Pipes to Smoking Guns: The Deepfake

Two years ago, I wrote a post about some new technologies that were allowing video editors to create realistic videos of things that never happened. Today, we call them “Deepfakes.” I think it’s worth a re-read. In it, I argue that education–particularly arts education–is the most effective way to combat the Deepfake. After all, Deepfakes aren’t really new. We’ve been dealing with “fake news” and propaganda for hundreds of years. How would you have responded to this story from February of 1898? The U.S. went to war.

USS_Maine
Image Source

But it did have me wondering. What do we do with a Deepfake? How should we treat some kind of shocking video or audio revelation? I don’t exactly have a checklist, but I think we need to look at scandalous revelations holistically. What else supports this shocking news? Do the facts add up? Where did the evidence come from?

The problem is (and I acknowledge that Hollywood is largely at fault for perpetuating this myth), we are obsessed with the “smoking gun.” There will be some single, irrefutable piece of evidence that proves our point and wins the day. But as I say repeatedly on this blog, movies aren’t real life. “Smoking gun” evidence rarely exists, and when it does, it’s usually not enough to prove much of anything.

Strangely, Trump’s infamous “Access Hollywood” tape offers a perfect case study for Deepfakes. Here we had a presidential candidate caught red handed saying absolutely repugnant things about women, about mothers, daughters, sisters, friends, and loved ones. The Democrats thought this was it for his campaign. They finally had hard evidence of what a narcissistic, sexist pig he was. But it didn’t matter.

Everyone already knew all of that. They knew he didn’t respect women. They knew he was vulgar. They knew he was lewd. It didn’t matter. Trump didn’t even try to deny it. He could have decried it as a Deepfake, but he didn’t need to. The smoking gun was just that, all smoke and no fire.

Is it possible for some video of dubious origin to crop up at an inopportune time in efforts to sway the public’s mind? Yes. Should the NSA stay abreast of Deepfake technology? Of course. But I think the situation is overblown. McKinley didn’t need video evidence of the Maine explosion to invade Cuba in 1898. Bush didn’t need video evidence of WMDs to invade Iraq in 2003. The question is less about what kind of evidence we have and more about what kind of sources we gather that evidence from. Keep that in mind as the conversation continues.

Screenwriters, do yourselves a favor.

HBO recently released Craig Mazin’s miniseries ChernobylWhile nothing in this life is perfect, Chernobyl comes pretty darn close. From acting to directing to art direction to sound design, Chernobyl is a masterclass in filmmaking.

But the biggest story is probably the story itself. In the television world, screenwriters hold the creative power and, as writer and executive producer, Mazin made a variety of bold and effective decisions. For example, the explosion takes place in the first few minutes of the mini-series. He doesn’t make the audience sit through a lengthy first act or ordinary world, and it’s spectacularly powerful. But his reasoning behind the decision is what will really make things click for filmmakers.

In addition to the show, Mazin recorded a companion podcast with NPR host Peter Sagal to accompany each episode. In it, he explains his creative decisions. He shares insight about story structure, adapting true stories, portraying gore on screen, sound design, and even accents. It’s entertaining, engaging, and informative. It’s unfiltered information coming from a filmmaker at the top of his game.

Taken together, Chernobyl and the companion podcast are worth far more to aspiring filmmakers than anything you can find in a university catalogue. The podcast is free and HBO Now has a 7 day free trial. You have no excuses. If you want to learn about the craft of filmmaking, Chernobyl is a must.

Why Are Night Scenes Blue?

You’ve either seen it on one of those posts that point out some of the ridiculous “things we’ve learned from movies” or maybe you’ve even noticed it yourself. But right there along with always finding a parking spot in front of the courthouse and every tank you shoot with a bullet is highly explosive, you’ll discover that everything at night is perfectly visible, but slightly blue. What’s up with that?

Well, the first thing is, it has to be visible. As I mentioned in a previous post, film is a method of preserving light. No light? No movie magic. Blue helps indicate that something different is going on, mainly the passage of time. It’s one of those subtle things that Hollywood hopes you pick up on, but don’t really notice. But why blue?

Color Temperature

If you’ve been shopping for lightbulbs recently, you’ve probably seen this little diagram.

LED-Kelvin-Color-Temperature-Landing-Page_09

That’s right. Lights are different colors. Chances are, you’ve never even noticed this before. Your brain has a habit of calibrating to subtle differences. That’s why you don’t go insane with all of the noise in a city and you stop noticing the tint on your rose colored glasses after a while.

A Look At The World Through Rose-colored Glasses.

But if you pay attention, you’ll start to see nauseating scenes like this.

LED Hell
Image Source

Filmmakers put a lot of effort into making sure this kind of catastrophe never appears on the big screen.

The thing is, you really only notice the different colors on their own. If the entire wall were orange or blue or green, the lights would have that distinct hue, but your brain would tune it out. It’s the comparison that makes it noticeable. What color is the sun? Well, it looks pretty blue when you compare it to a campfire.

campfire

Moonlight, after all, is just reflected sunlight. I don’t know about you, but I’d say that night scene is pretty blue looking.

The other major source of light we’ve had as humans has been the incandescent lightbulb, which also glows at a warm orange 3200K*.

lighted string lights hanging
Photo by Vitaly Vlasov on Pexels.com

So why color temperature?

There’s a bit of physics involved here, but the basic idea is that if you heat something to a particular temperature without it bursting into flames, it will start glowing. So an incandescent bulb (which glows because it’s being heating) is, in fact, pretty close to a temperature of 3200K.

Daylight is measured between 5500K and 6500K. The sun’s light is diffused by the atmosphere, so it’s a little bit of a moving target. But the actual surface temperature of the sun? 5778K. Not too shabby.

Fluorescent lights and LEDs  do not use heat to produce light, but are still measured on the same scale.

More Science

Human eyes detect light with two different kinds of cells: rods and cones. Rods simply detect light, not color. Cones are sensitive to particular wavelengths of light and separate out small (blue), medium (green), and long (red). The other colors are made by blending wavelengths. At low light levels, only your rods will activate and the world will be a series of shadows and silhouettes. But as more light is introduced, the small cones may be activated, and you’ll start to see a little bit of blue first.

So don’t feel blue the next time you’re riding off into the moonlight, even if everything may look it.

Day_for_night

 

*Color temperature is measure in Kelvin (K), named after Lord Kelvin who created the absolute temperature scale. Since 0K is so cold molecules would stop moving, it’s also called absolute zero. (Yes, Andre 3000, that’s cooler than being cool.) Also, Kelvin is just Kelvin, not degrees Kelvin.

The Last Watch. Thankfully.

On Sunday night, HBO suckered Game of Thrones fans in for what was touted as a two-hour behind-the-scenes documentary, a glimpse into the magic of the making of Game of Thrones. I’ve seen most of the bonus features on the Game of Thrones Blu-rays. They’re well-produced and informative. This sounded like it would be a great retrospective on the series and an emotional farewell tour. Instead, HBO gave us a cloying, aimless, slice-of-life piece that ranks somewhere between a vacation slideshow and college project. 

There are plenty of behind-the-scenes shots in The Last Watch, but they completely lack context or explanation. My wife called it pretentious. “It’s like they’re saying, ‘You don’t understand what we do, and we’re not going to explain it to you.'” I agree. Hundreds of people worked on the show, but the documentary only covered a handful of crew members and none of them very well. No one from the camera, grip, electric, props, sound, video, or AD departments were interviewed. There was no story. There was no narrative (ironic for a show whose finale centers on a speech about great stories). There were some heartfelt moments, such as when makeup artist Sarah Gower explained that because both she and her husband worked on the show, neither of them were at home with her daughter. Sad? Yes. A two hour story? No.

In many ways, it felt as though these crew members had drawn short straws and were being saddled with the BTS crew because no one else wanted to talk to them. That may have been the case, but the documentary did have a way out. Andrew McClay, a background actor who played a Stark soldier for multiple seasons, seemed to love the BTS crew. He was the perfect, humanizing connection between an epic fantasy series and the audience at home. Just an average Joe trying to make a living. But the documentary failed in some very basic ways to craft that story. Can we see where Andrew lives? What did he do before GOT? How did GOT change his life? What do his friends and loved ones think of all this? With so many unanswered questions, maybe he’ll get a spinoff series…

The documentary avoided discussing creative decisions in the final season, in depth interviews with major cast members or the show’s creators, or even a broad representation of the cast. Fingers crossed, those things will appear in the Blu-ray. Last night however, we were given a voyeuristic opportunity to fawn over Emilia and Kit (or Keeeet as the Spaniards call him) and a very brief glimpse at the humanity of a very small slice of a very large crew. Not exactly the kind of documentary quality I’ve come to expect from HBO. Did it fill two hours of programming and keep some of Game of Thrones fans tuning in for another week? Yes. But it could have been so much better.

Lights, Camera, Action (Part 2)

Psychology of Lighting

Like all aspects of filmmaking, lighting has subtle, psychological effects on the audience. Knowing what these effects are and knowing how and when to use them will greatly enhance any film you make.

Low Key*: Darkly lit or low light scenes evoke a sense of mystery and danger, like this shot from The Shape of Water. 

shape-of-water-online-678x381
Image Source

 

High Key: Brightly lit scenes like the opening of La La Land tend to be happier.

1-lalaland
Image Source

 

Soft Light: Few or poorly defined shadows create a sense of fantasy. For example, I find dragons and armies of undead more realistic than the fantasy world created in Letters to Juliet.

letters-to-juliet-christopher-egan-1-600x300
Image Source

 

Hard Light & High Contrast: Well defined shadows create a grittier look like Sin City.

MAC34_SIN_CITY_CAROUSEL01.jpg
Image Source

 

Warm Colors: Orange hues can evoke warm feelings and romance.

Limitless Orange
Image Source

Cool Colors: Blue is used to create a sense of cold or uncaring.

Limitless Blue
Image Source

The previous two images are both from the movie Limitless. You’ll notice the color difference the most by looking at Bradley Cooper’s skin tone. Skin tone one of the first reference points audiences latch onto.

Other colors can be used to create a sense of unease, like something isn’t right. The Matrix, for example, was decidedly green.

matrix
Image Source
Knowing the Toolkit: Skilled cinematographers, of course, can also use these techniques ironically or to create a new meaning with juxtaposition. One example is flash photography.

Flash Photo

You get this effect by having a harsh, bright light source right next to the camera (as you would on a disposable camera or smartphone). Things close to the camera (like faces) get washed out, while the background is underexposed. It also creates harsh shadows. (Look at the distinct black line on the left side of the red coat). It looks amateurish. But if you want your project to look like it was made by amateurs, this is your ticket. Thank you, Blair Witch Project.

blair_witch
Image Source

One Final Thought

When you think about it, all movies, all TV shows, all video games, every viral video you watch on your phone is really a manipulation of light on a screen. The realty – the screen – remains unchanged. The meaning comes from your mind’s interpretation of that light and the story it creates. Controlling that light is the difference between staring at static and touching an audience.

 

*The “key” in “low key” and “high key” refers to the “key light” or main light source.

Lights, Camera, Action! (Part 1)

What is it with films and lighting? It’s right there, number 1 in the most cliched of all Hollywood phrases*. And if you’ve ever been on a film set, you know that the vast majority of time is spent lighting and relighting the scene. But why?

Visual Medium

At its basic level, film (and the modern equivalent of digital video) is actually a way of recording and preserving light. So while we think of films as epic stories with huge budgets and special effects, it’s really a specific kind of recording device. On a technical level, you could compare film to taking a plaster impression of an animal print or recording tremors with a seismograph.

person holding camera film
Photo by Luriko Yamaguchi on Pexels.com

For those of you old enough to remember developing photos from 35mm film, this is “film” at its most basic. Photons bounce off an object, are focused by a lens onto the film, activate photo-reactive chemicals, and leave an impression. The film is then developed into a negative, and the negative is used to produce the positive picture.

Pozytyw_i_negatyw
Image Source

The thing is, that 35 mm film from your disposable CVS camera is actually the same film used in movie cameras and was essentially unchanged for 100 years. Today, the process is digital, but essentially your iPhone and a professional movie camera work the same way. And it’s all about capturing light. 

What are we waiting on?

So why does it take so long to light a film set? There’s a problem between seeing and seeing. Your brain plays probably about the same role in sight as your eyes. Yes, your eyes perceive light, but your brain needs to decide what to focus on (literally and figuratively). In addition to actual focal distance, it must determine proportion, patterns, colors, shapes, and movement. Are you looking at a cat or a lion? Or a toy lion? Or a picture of a lion? 

Our brains concentrate on only one small part of what we actually see. But when a filmmaker sits you in a darkened room and projects an image 30 feet tall and 100 feet wide, they need to direct your attention. They have to be sure what they are showing you is more interesting than the exit sign in the corner of the theater or that weird haircut on the guy two rows in front of you.

1007530832-Dipson-McKinley-Mall-4-Cantillon
Image Source

Just to be clear, your director of photography and the camera are taking the place of the two most complex organs in the human body. This involves framing shots appropriately, dressing and costuming appropriately, and making sure the correct subject matter is in focus. And it also involves a LOT of lighting. 

Consider this shot from Citizen Kane.

Citizen-Kane
Image Source

It’s both simple and incredibly effective. The main sources of light come from the back of the room, creating a hard silhouette of both subjects. They are both easy to see while also being obscured. The light beams add a sense of depth, while the desk lamp fills in some of the void. That way we aren’t merely looking at silhouettes.

This shot required critical placement of the lights, camera, and actors. The lights had to be focused and balanced in intensity so the desk wasn’t too dark or the beams too bright. Smoke or fog played a critical role in creating those beams, too.

Physics of Photography

Whether you’re taking photos at a wedding or shooting a film for Spielberg, everyone’s playing by the same rules. Light and film have certain properties and limitations. A lot of cinematography is understanding these limitations and finding a way to get the look you want with the tools you have.

Concentrate! 

There can be a lot of stuff in a scene: actors, props, sets. How does your audience know where to look? Light, of course. Here’s a very messy photo from my office.

IMG_7228

Now where did I leave that fine point black Sharpie? When I change the lighting, it’s a little easier to see. sharpie

Exposure

Ever had an annoying camper shine a flashlight in your eyes in the middle of the night? First, it’s painfully bright. Then it takes your eyes a few seconds to readjust to the darkness. Your irises make minor adjustments constantly, and because it happens so fast, you don’t usually notice. What happens if you’re filming a scene that includes very bright and very dark parts?

Here, Phillies Bear and Phanatic Pillow Pet are very excited to watch the Phillies (currently number 1 in the NL East!), but the problem is, the window behind them is grossly overexposed.

Philly overexposed

If we set the exposure for outside, it looks like they are sitting in a black hole.

underexposed

The solution is to add light inside the room or take light away outside (most likely by putting filters on the windows). Either way, it’s going to involve some lighting.

Inverse Square

The intensity of lights decreases as you get further away. Not by a little, but by a LOT. In fact, light follows the inverse square law. That means if you double the distance between your subject and your light source, but want to keep the same light intensity, you need to quadruple your lighting power. Bigger shot? More lights.

How many lights did it take to illuminate this post-apocalyptic set from Bladerunner: 2049?

 

BladeRunner2049_Framestore_ITW_06A
Image Source

Well, start counting.

BladeRunner2049_Framestore_ITW_06B
Image Source

You’re out of your depth. 

Lighting adds a sense of depth to a scene. After all, we see in three dimensions, but movie screens are flat. If there were no depth, everything would look like a cartoon. In the shot from Bladerunner: 2049, you can see the silhouette in the foreground, our actor in the middle distance, and hazy ruins in the background.

This shot from The Third Man, shows the depth of a tunnel. Even as a still frame, you get a sense of the dimension.

the-third-man1
Check out this image source for a great post about depth with lighting

Today, many movies are shot in 3D, but lighting still plays a critical role in creating the space for a scene.

Next Time…

That covers some of the technical hurdles that go into lighting a scene. In the next post, I’ll cover some of the other things a director of photography considers on a set.

 

*The phrase “lights, camera, action” was probably never used to get a film crew rolling. Even today, lights take a while to warm up. There is also some dispute about the origin of the phrase. In reality, the dialogue before a take goes something like this:

1st Assistant Director: “Let’s roll.”

Production Assistants: “ROLLING!”

Sound department: “Sound speeds.”

2nd Assistant Cameraman: “101 take 1. A mark.” (Hits slate.)

Camera Operator: “Set.”

1st Assistant Director: “And… Action.”

Putting the Assist in Video Assist

A while back I wrote an article about “grips,” one of the first film credits the average movie goer puzzles over. But gripping isn’t my day job. I’m usually a video assist operator. Most people, even my coworkers, don’t really know what video assist entails. Well hold onto your BNC*, because you’re about to find out!

s-l500

The Basics

Do you ever watch the bonus features at the end of a movie and see the director watching a television monitor? That’s video assist. The video assist operator sets up that monitor. Pretty basic. More broadly, it’s the video assist operator’s job to get the image from every camera (no matter how many there are or where they are) to the director. In a nice, studio setting, it may look something like this.

IMG_4104 2

This is a control room we built for Comedy Central. On a stage, with one camera, this may be as simple as connecting the camera to a monitor (with some BNC). Things get a little more complex if you’re riding around the street at seventy miles an hour or if one of your cameras is in a helicopter. In those situations, we use transmitters to get the image from the camera to the director.

IMG_4887
This is the “picture” car with the actors and camera. Our transmitter is on the roof.
IMG_4932 2
This is the follow vehicle for the director and assistant cameramen.
IMG_4933
And finally you have a follow van for the producers.

History

Once upon a time, we filmed on something called “film.” This was a roll of cellulose coated with a film of photo reactive chemicals that had to be developed before you could see what you shot. Each day, someone would run the film to a lab where it was developed. Then they would race back with “dailies” for the director and producer to review. It worked great for about 70 years.

But in the 1950s, television and video came onto the scene. Television captured images electronically and recorded them onto magnetic videotape so there was no need to “develop” film. Although the quality was far inferior to film, people could now watch what they filmed immediately after they filmed it.

In 1960, Jerry Lewis used this new technology for his film The Bellboy. As both the director and the star, he couldn’t watch his own performance. He connected a video system to the film camera allowing him to instantly playback what was just recorded and make adjustments to his performance. This is generally considered the invention of video assist** and the first use of video playback.

Jerry-Lewis-Creator-Of-Motion-Picture-Video-Assist-Although-we-have-been-he
Jerry Lewis with his contraption.

 

Playback

In addition to getting the image to the director, the video assist operator records the action, allowing for immediate playback. This allows directors to re-watch takes without waiting for dailies. Directors may call for playback to check performances, look for continuity errors, see if a piece of gear was in the shot, check to make sure a stunt or effect worked, or watch something back in slow motion.

The video assist system is also used to record rehearsals. When we shot on film, this was a critical way to practice camera and actor movement without wasting thousands of feet of film. Today, most film production is done on digital video, but film crews still use rehearsals to practice shots before all of the lights, makeup, and effects are in their final places.

On a one-day shoot for a thirty-second commercial, playback may not be critical. But on something like a feature film that may take months and is generally not filmed in chronological order, playback can play a major role in keeping a film’s continuity. One intern I worked with said that video assist sounded like the film’s library. I kind of like that. (To be fair, the script supervisor is the film’s librarian, the video assist operator is the guy who has to go to the basement and find all of the old reference books…)

It’s important to point out that video assist is for reference only. When we shoot on film, the video feed or “tap” doesn’t have nearly the depth of field or exposure range of a piece of film. Video playback is not a completely accurate representation of what you filmed. In fact, oddly enough, video taps only record what isn’t on the film. The light entering the camera either exposes the film or goes to the video tap. Most things happen so quickly it doesn’t really matter. But several years ago, I was working with a producer to film gunshots for an action movie. When I played it back for him, he didn’t see the muzzle flash. I had to explain to him that was a good thing because it meant the muzzle flash was on the film. (Ugh. Tech talk.) And while that’s not the case with modern digital cameras, the video feed is a compressed version of what you’re actually recording. Much to the dismay of one producer whose intern deleted their favorite take, video assist operators do not record full resolution video files.

One other note is that for decades, video was recorded on tape. This led video assist operators to also be called video tape recorders or VTR. And while many ADs will still stay “let’s roll video” we moved on to computers about ten years ago.

Sync

In addition to the image, the video assist operator works with the sound department to get audio to the director. It’s critical to make sure all of the cameras and the audio are synchronized. Like watching a badly dubbed film, it’s disorienting and distracting to watch things out of sync.

This may sound like a simple thing, but let me give you an idea of what it’s worth to a production. A few years ago, shortly after we switched to high definition signals, quality, affordable transmitters were still in development. The transmitters we used, while great at long distances had a noticeable delay. (It takes some time to crunch down all of that video data, shoot it through the air, and unpack it back into an image.) This isn’t an issue if you’re filming across a football field. It is if you’re sitting ten feet from your actor and the actor on your television is moving noticeably slower. So after one frustrating day of filming on a new television show with transmitters that were less than a year old, our company shelled out $40,000 to upgrade to the latest transmitters.

Gak

Gak is a technical film term for stuff. And video assist operators have a lot of gak: monitors, switchers, speakers, cable, transmitters, hard drives, HDMI, BNC, barrels, patch bays, UPS’s, power conditioners, batteries, power cables (OSHA cords, P-Tap, Lemo, Hirose, 4-pin XLR), 3-pin XLR, mini XLR, intercoms, video printers, and on and on and on. Again, all of this is to get the image from the camera or cameras, synchronize it with the audio and give it back to the director for live viewing and playback.

IMG_6062

IMG_6065

It’s a lot of techno-wizardry and it can be kind of mesmerizing. Maybe that’s why everyone likes to look over your shoulder. Or maybe they just like watching TV. Whatever the case, the video monitors always attract a crowd. The director, assistant director, and script supervisor are generally at the monitor. The producers will often get their own set of monitors. Then there are the hair, makeup, and costume people who need to be sure the actors look their best. And you’ve got the additional assistant directors cueing background actors, the electric and FX department standing by for cues, the humane society looking out for their four legged actors, and a couple of groupies for good measure.

And of course, all of these people need chairs and coolers and tents and heaters if it’s cold and, frankly, before you know it, you’ve got a regular village on your hands: a video village, which has become the industry term for, well this.

IMG_6109

One of the biggest positives about video assist, however, is that you always want to be near the director. So if it’s raining, you’ll be dry. If it’s cold, you’ll be warm. If it’s warm, you’ll be cool. And you’re generally the first person to know if craft service is serving something better than hot dogs.

24 Frame Playback

 Video assist operators have one more vital function that isn’t related to anything else I’ve mentioned so far. We are responsible for every “on-screen” video display. Every time you see a smart phone or a television or a computer monitor, a video operator has provided that image. Here are a few of the riveting images I’ve put on screens.

IMG_0559IMG_0576

Amazing stuff, right? The video operator works in conjunction with the director, props department, and editor or visual effects department to make sure the right images get on the right monitors. Sometimes, that image doesn’t exist yet, so we just put green or grey screens on the monitors. That helps the VFX department create the image in post. Green makes it easier to remove the image. Grey allows light to fall on the set and actors like a real television.

Video assist operators need to be careful about what they put on screen. One of my coworkers was troubleshooting an issue with his personal phone and forgot to delete the number when we went to shoot. Months later when the DVD came out, he was inundated by calls from curious fans who wanted to know who Jason Statham called in the middle of the film

jason phone
Source

The name, “24 frame playback” comes from the fact that film cameras used to film 24 frames or individual pictures a second. Televisions, however, display 29.97 pictures a second. So for every 1 picture a film camera takes, a television displays 1 ¼ pictures. (Oh God. Math…) Because the television is projecting light, that means ¼ of the television screen will appear brighter than in every picture the film camera takes. That’s why in old movies, you will see television screens flickering in the background. To fix that, playback operators force the television to display images at 24 frames, not 30.

Modern TVs and flat screens like your phone don’t display images the same way older TVs did, so we don’t actually need to create a 24 frame video. That’s why it’s anachronistic to call it “24 frame playback” when it’s no longer 24 frames and playback actually has another meaning. Sheesh.

For some more information about how a television actually works, check out this fascinating video from The Slow Mo Guys.

Roll the Credits

There you have it. Another one of those hundreds of names that scroll by at the end of a movie explained. While it’s true that most of the work of the video assist operator doesn’t end up one screen, the movie – and filmmaking – wouldn’t be the same without them.

 

 

*BNC, the ubiquitous tool of the video assist operator, is a type of connector for video cables. BNC is also refers to the cable itself. Many people erroneously think it stands for “British Naval Connector,” a rather shoddy backronym for an American invention. It actually stands for “Bayonet Neill-Concelman,” the connecting mechanism and the name of its inventors.

**Parts of the video assist system existed before this and several technicians improved the system over time. You can read more about that here: https://www.cnet.com/news/video-assist-predates-jerry-lewis-patent/