Smoking Pipes to Smoking Guns: The Deepfake

Two years ago, I wrote a post about some new technologies that were allowing video editors to create realistic videos of things that never happened. Today, we call them “Deepfakes.” I think it’s worth a re-read. In it, I argue that education–particularly arts education–is the most effective way to combat the Deepfake. After all, Deepfakes aren’t really new. We’ve been dealing with “fake news” and propaganda for hundreds of years. How would you have responded to this story from February of 1898? The U.S. went to war.

USS_Maine
Image Source

But it did have me wondering. What do we do with a Deepfake? How should we treat some kind of shocking video or audio revelation? I don’t exactly have a checklist, but I think we need to look at scandalous revelations holistically. What else supports this shocking news? Do the facts add up? Where did the evidence come from?

The problem is (and I acknowledge that Hollywood is largely at fault for perpetuating this myth), we are obsessed with the “smoking gun.” There will be some single, irrefutable piece of evidence that proves our point and wins the day. But as I say repeatedly on this blog, movies aren’t real life. “Smoking gun” evidence rarely exists, and when it does, it’s usually not enough to prove much of anything.

Strangely, Trump’s infamous “Access Hollywood” tape offers a perfect case study for Deepfakes. Here we had a presidential candidate caught red handed saying absolutely repugnant things about women, about mothers, daughters, sisters, friends, and loved ones. The Democrats thought this was it for his campaign. They finally had hard evidence of what a narcissistic, sexist pig he was. But it didn’t matter.

Everyone already knew all of that. They knew he didn’t respect women. They knew he was vulgar. They knew he was lewd. It didn’t matter. Trump didn’t even try to deny it. He could have decried it as a Deepfake, but he didn’t need to. The smoking gun was just that, all smoke and no fire.

Is it possible for some video of dubious origin to crop up at an inopportune time in efforts to sway the public’s mind? Yes. Should the NSA stay abreast of Deepfake technology? Of course. But I think the situation is overblown. McKinley didn’t need video evidence of the Maine explosion to invade Cuba in 1898. Bush didn’t need video evidence of WMDs to invade Iraq in 2003. The question is less about what kind of evidence we have and more about what kind of sources we gather that evidence from. Keep that in mind as the conversation continues.

Lights, Camera, Action (Part 2)

Psychology of Lighting

Like all aspects of filmmaking, lighting has subtle, psychological effects on the audience. Knowing what these effects are and knowing how and when to use them will greatly enhance any film you make.

Low Key*: Darkly lit or low light scenes evoke a sense of mystery and danger, like this shot from The Shape of Water. 

shape-of-water-online-678x381
Image Source

 

High Key: Brightly lit scenes like the opening of La La Land tend to be happier.

1-lalaland
Image Source

 

Soft Light: Few or poorly defined shadows create a sense of fantasy. For example, I find dragons and armies of undead more realistic than the fantasy world created in Letters to Juliet.

letters-to-juliet-christopher-egan-1-600x300
Image Source

 

Hard Light & High Contrast: Well defined shadows create a grittier look like Sin City.

MAC34_SIN_CITY_CAROUSEL01.jpg
Image Source

 

Warm Colors: Orange hues can evoke warm feelings and romance.

Limitless Orange
Image Source

Cool Colors: Blue is used to create a sense of cold or uncaring.

Limitless Blue
Image Source

The previous two images are both from the movie Limitless. You’ll notice the color difference the most by looking at Bradley Cooper’s skin tone. Skin tone one of the first reference points audiences latch onto.

Other colors can be used to create a sense of unease, like something isn’t right. The Matrix, for example, was decidedly green.

matrix
Image Source
Knowing the Toolkit: Skilled cinematographers, of course, can also use these techniques ironically or to create a new meaning with juxtaposition. One example is flash photography.

Flash Photo

You get this effect by having a harsh, bright light source right next to the camera (as you would on a disposable camera or smartphone). Things close to the camera (like faces) get washed out, while the background is underexposed. It also creates harsh shadows. (Look at the distinct black line on the left side of the red coat). It looks amateurish. But if you want your project to look like it was made by amateurs, this is your ticket. Thank you, Blair Witch Project.

blair_witch
Image Source

One Final Thought

When you think about it, all movies, all TV shows, all video games, every viral video you watch on your phone is really a manipulation of light on a screen. The realty – the screen – remains unchanged. The meaning comes from your mind’s interpretation of that light and the story it creates. Controlling that light is the difference between staring at static and touching an audience.

 

*The “key” in “low key” and “high key” refers to the “key light” or main light source.

This Is Not a Pipe

Photo manipulation is nothing new, and for anyone growing up in the digital age, Photoshop has morphed from a proprietary digital editing program into a verb. Now, with a few intuitive swipes on a smartphone, you can turn your mediocre vacation photos into vintage “instagrams,” crop, distort, color correct, add text, and even add special effects to your videos. While most of this computing power is used for cheeky fun, everyone holds in their hands the ability to make photo-realistic, doctored images. “It’s only true if there’s pictures.” Not necessarily.

Until recently, audio and video manipulation were only possibly if you had a large team of special effects professionals at your disposal. Doctoring video and audio recording simply took too much computing power. Now, that’s no longer the case. Two new technologies will allow you to manipulate video and audio recordings with the same ease as Photoshop.

The first allows simple drag and drop editing of audio files. The most of advanced of these, Adobe’s VoCo, will seamlessly enter typed text into previously recorded audio files. Without recording any additional audio, you can generate a new file of Neil Armstrong’s famous moon landing that reads, “That’s one small step for a man, one giant leap for mankind.” Or, “That’s one small step for a woman, one giant leap for mankind.” Or, “That’s one small step for man, one giant leap for your mom.” As long as you have twenty minutes of previously recorded audio file, the program can generate whatever new audio you want. This is particularly troubling for public figures who have days worth of audio files easily accessible to the public. In theory you could make an actor, athlete, or politician say whatever you want. The repercussions should be obvious and disturbing. For a more thorough explanation and example, listen to this podcast from RadioLab.

The second technology (also discussed in the podcast) is a motion capture technology that allows for real time manipulation of video images. This is similar to the motion capture technology Hollywood uses to create anthropomorphic creatures like Gollum or King Kong. Here you can see a sample of an edited video of President Obama. If you look closely, you’ll notice some irregularities where the program could not satisfactorily meld the two videos. But the bigger point is that this technology is becoming cheaper, better, and more accessible.

Is this the end of trust as we know it? Can we no longer believe anything we see with our own eyes? That would be the case if you accept film as reality. But film isn’t real life. I’ve said it before, and I’m sure this won’t be the last time I say it. Usually, I’m pointing out that theatrical feature films are more exciting, prettier, simpler, and more entertaining than real life. But for this post, I’d like to look at things from the other angle. No film, no matter how realistic or “unedited” it looks is reality. It’s just a film. Film isn’t real life.

In the commercial and theatrical world, professional filmmakers intentionally alter reality. They use all kinds of tricks to make things intimidating, pretty, ugly, or endearing. Makeup, costumes, and lighting turn a perfectly charming Emilia Clarke into the Mother of Dragons.

But these tricks are still in play even if you don’t use them intentionally. Here are two pictures of a friend of mine at a mud run.

IMG_8313IMG_8322

In the first, you can clearly see the falling rain. (No, those aren’t orbs…) In the second, you can see that it is still raining if you look at the water. But the change of angle and focal length means you can no longer see the falling rain. (This is the same reason why it sometimes looks like it’s barely sprinkling at rained out sporting events.) I didn’t hide the rain intentionally, but that’s the reality of the situation.

The important thing to remember is, consciously or not, all media is an interpretation of reality. When you’re on vacation and take a picture, it captures some part of the moment, but it isn’t a recreation of the moment. You’re limited by the abilities of your camera. You choose to photograph the Grand Canyon not the parking lot next to the Grand Canyon. You crop out the guy wearing that ridiculous Hawaiian shirt. And what about the people in your photo? Are they like my nephew, who for two years refused to smile any time someone pointed a camera at him? Or do they ham it up for the camera in the hopes of becoming an internet star?

This is nothing new. Film has been an interpretive art since its inception. Below are two of the earliest war photographs ever taken. These are from the Crimean War in 1855, twenty-three years before the first movie was made.

canonball2canonball 1

Both show a scene of desolation strewn with cannonballs. But it’s the second one that would make photographer Roger Fenton famous. Simply put, it’s a more striking photograph. Fenton wanted to show the horror and destruction of war, but he was restricted by his cumbersome film equipment. His solution? Move the cannonballs onto the road to take advantage of the high contrast. Although it was much more labor intensive than Photoshop, it’s the same basic principle. He altered his photo for effect. He sought a deeper “Truth” that wasn’t reflected in “reality.”

In the late 1920s, surrealist painter Rene Magritte created this thoughtful painting called “The Treachery of Images.”

the-treachery-of-images-this-is-not-a-pipe-1948(2)

If you don’t know French, the text reads, “This is not a pipe.” Of course it’s not a pipe. It’s a painting. It represents a pipe. Our brains conceive of it as a pipe, but it is not a pipe. The same holds true for all media.

A definition might be helpful here. Media is the plural of medium. The two definitions that come to mind are medium (size), as in the size between small and large and medium (fortune teller), as in someone who communicates between the living and the dead. Both of course share the same Latin route, medias, which simply means middle. It shouldn’t surprise you that our news sources are generally referred to as “the media.” They are the middlemen. They transport ideas from the source to us. But along the way, they must interpret it.

Media is also used in the art world to describe the material that an artist uses. You may see the phrase “mixed media on canvas.” This medium might be oil paint, latex paint, clay, canvas, silk, steel, analog audio recording, digital video recording, computer programs, or even food. The point is, the artist interprets the world through this medium. Film, as an artistic endeavor, is its own medium. But never forget that the six o’clock news, the news radio traffic report, and the Wall Street Journal all operate in artistic mediums. To explain in more detail, I’d recommend listening to Malcolm Galdwell’s Revisionist History podcast from a few weeks ago.

In it, he discusses this famous photograph from Birmingham in 1963.

footsoldier

It seems to show a police officer unleashing his dog on a black protester. But it doesn’t. The man in the photograph wasn’t part of the protest. He wasn’t a “Foot soldier,” as the Civil Rights activists called themselves. And the police officer hadn’t unleashed his dog on him. If you look closely, both men seem surprised and the police officer is leaning back, trying to pull the dog away. But that’s not what the nation saw.

This photo highlighted the brutality of the Jim Crowe south. It represented the discrimination, the institutionalized hate, and the lynchings. It shifted public opinion to the side of the Civil Rights movement, and it was all done on purpose. Bill Hudson, the photographer, chose this picture over the hundreds of other photos he had taken that day. The editor of The New York Times chose to put this story above the fold rather than any other news of the day. This is what a medium does. It takes the raw data, curates it, interprets it, and disseminates a cohesive message. In doing so, a medium must disregard data that fails to support its message or obfuscate its position. In its quest for “Truth,” it must necessarily deviate from “reality.”

To further clarify, look at Ronald S. McDowell’s statue inspired by Bill Hudson’s photograph.

footsoldier2.jpg

The figure representing Walter Gadsen, the student, is considerably younger and shorter than he was in reality. The police officer is emotionless and inhuman, reminiscent of T-1000 from Terminator 2: Judgment Day, which came out four years before the sculpture was dedicated. The police dog’s mouth is wide open, bearing vicious, anatomically improbable fangs. But remember this is a piece of art. This is not a pipe.

The push and pull between film as art and film as documentation will never end. But as a filmmaker and consumer of media, it’s important to acknowledge that film is a medium. It is not reality. There is no magic bullet, no enforceable code of conduct, no ten commandments of filmmaking that will ever make film purely objective. Reality is reality. Film is film. The best thing you can do as both a filmmaker and a consumer is educate yourself.

It’s important to learn about technology, to learn what is possible and how to spot a fake. But it’s also important to learn about art. You know—art, that thing that gets cut when we want to tighten budgets. Learn about artistic conventions. Learn to read the meaning behind how a frame is composed, how set decoration reinforces the theme, and how story arcs are constructed. Understand that models and movie stars are just people, too. Appreciate the complex and tragic story of Aaron Copland and Fanfare for the Common Man (examined here in another great podcast). See the allegory between Game of Throne’s White Walkers and climate change. Learn how Proust’s understanding of memory preceded neuroscience. Discover the 1920 play that introduced the word robot and the idea that robots are out to kill us. And, above all, recognize that just because you see a video of something doesn’t mean it’s reality. In fact it’s not reality. It’s just a video. And while there will be a degree of objective truth in it, there will always be a degree of the artist’s truth. After all, it is a video. It is not a pipe.