Ideas in Sound Design: Semiotics and Language – Part 1

Cross-posting from my personal blog.

Let’s start off with a disclaimer.

I am no expert in linguistics and semantics, nor would I consider myself truly conversant in the many critical models employed in film theory/criticism. Semiotics was something waved in front of my eyes a couple of times during undergrad (where I did not studio audio or film, by the way), then explored in much greater depth during a course in my graduate program. For this article, I’ll be falling back to some of the general concepts of Semiotics. They are the points that have truly stuck with me over the years, and they easily apply to many facets of media. They can help explain the significance of events in a narrative, the choice of words in dialog, or the functions of shot composition and sound design.

So, what the heck is Semiotics? To be honest, it’s a lot of things…

This article will be focusing primarily on some of the concepts developed by Ferdinand de Saussure. Specifically we’ll be dealing with the idea of “Signifier” and “Signified.” The notion of common historical acculturation and habituation will also come into play (though I can’t say I know if Saussure is responsible for that idea as well).

At the most basic level, you could think of the signifier as the pointer file in your project that tells your DAW which audio file to play. In this case, the audio file is the signified. The signifier/signified relationship is a code system we use everyday to convey and understand meaning. It’s language. Your DAW understands that a pointer file really represents the physical location of a set of data. The process is the same when I tell you to imagine a chair. The word “imagine” tells you to picture in your mind’s eye the appropriate piece of furniture indicated by the word “chair.” Notice in that example that I’ve really only used signifiers to communicate abstract concepts.

You might argue that a chair is a physical object, but “chair” is really more of a category. It’s highly unlikely that any two people reading this article imagined the exact same chair. The chair you imagined is probably a physical object you encounter in any of the number of spaces you occupy in your life. “Chair” calls up a string of properties in your brain that associates the word with an item that fulfills those properties. So, here we all are…sharing an understanding of the word “chair” and the function it implies, but holding discrete interpretations of its physical meaning…which brings us to our next point.

The connections that exist between signifier and signified are culturally dependent. If you were to read the word “voiture” with no prior understanding or exposure to the French language, you would make no connection to the signified…much the same way “car” would confound someone perfectly isolated from exposure to English. But this concept of cultural connection extends down to a more granular level. Your physical location and the people around you exert just as much influence in the connections between signifier and signified. [Did you just make the connection that it affects the language you speak?] Consider some of the cultural language differences between Britain and the U.S.A; both English speaking countries. One extreme example is the word “fag.” In Britain it’s something you smoke. In the U.S.A. it’s a derogatory and discriminating term used by insecure idiots.

So the connections, and there can be many, between signifier and signified are context and culturally dependent. The extension of that idea is that those connections are also continuously in flux. Additionally, meaning is not an asset exclusive to written and spoken language. Actually, language isn’t even necessarily restricted to written and spoken words. Meaning can be conveyed through actions, gestures, images, sounds or even situations. What gives these different signifiers meaning, a connection to the signified, is the shared cultural history of a society. Signifiers point to additional meanings or take on entirely different ones, while new meanings and concepts in need of signifiers are continuously developed.

Hopefully, you’re starting to see how this applies to sound design.

I’m of the belief that sound design is an informed process. Let’s pause to look at a definition.

Design
verb (used with object)

1. to prepare the preliminary sketch or the plans for (a work to be executed), especially to plan the form and structure of
2. to plan and fashion artistically or skillfully
3. to intend for a definite purpose
4. to form or conceive in the mind; contrive; plan
5. to assign in thought or intention; purpose

[from dictionary.com]

Without the thought process, the planning, the meaning behind it…a sound remains nothing more than a random auditory effect placed against a visual. It is far more appropriate to think of sound as a form, or an element, of language in a piece (though I admit that this is far easier to coordinate in early/pre-production than to start in post).

There are two ways to deal with signification in sound design.

The first is to work with signification that is already inherent in a culture. The oh-so-frequently heard red-tailed hawk used to convey a sense of open and wild spaces, the sound of a heart-beat to convey tension or concentration, a minor 3rd chord to suggest melancholy or sorrow. Notice that these are conventions that have been used many times over the years. To use them without a sense of self-reference now is considered somewhat lazy or trite, but they are easily understood thanks to their historical usage. Society and culture (at least some of those around the world anyways) have been habituated to them as signifiers and can readily derive their meaning. More artful uses reference these signifiers, implying the same meaning without the use of those same tired signifiers. There are dangers with using existing, well known, signifiers though.

Besides the aforementioned “overused” issue, this approach presents a potential problem…it may pull in additional unwanted meanings. Because these significations are accepted thanks to historical habituation, it means they may have acquired additional specific meanings beyond the general/original meaning. The red-tailed hawk can detract from the grandeur of the shot in a dramatic film if it causes one to think of all the times it’s been used in comedies recently. [A crude example, but it illustrates the point.] This additional meaning problem is one reason why most productions opt for new custom music, rather than using well known pop music. These new meanings bring in outside associations that can cause a shift in perspective for the viewer. On the flip side, early trailers for new films often use music and sounds from older, popular, films in the hopes of creating that connection to generate more excitement. What works well for advertising rarely works well for narrative, though. And working with existing signifiers requires a careful balancing act.

The safer, though perhaps more difficult and satisfying, method for working with signifiers in sound design is to develop a new “language” for each production. Remember that signification is a function of exposure, a collective agreement by a culture or society that connects signifier and signified through repitition over time. This means that it is possible to attach meaning to sounds exclusively with the audience during the course of a piece. The more instances an audience has to observe the connection, the more ingrained it will become. In his book, Thinking, Fast and Slow, psychologist Daniel Kahneman states that stimulus in our environment affects the way our brain processes information (in immediate ways). He also says that our minds have the predilection for finding patterns (frequently, even where they do not exist). [The guy has a Nobel Prize. I’d like to think he knows what he’s talking about.] This gives sound designers the opportunity to subtly push the audience, twist their perception, and add or clarify meaning through the use of sound.

The difficulty is in the fact that the signifiers have to be used enough for the brain to make the connection, but without creating repition that causes the viewer to disregard them. Habituation to specific sounds makes us stop attending to those sounds; unless they occur in a situation where they normally wouldn’t. That’s the trick, finding ways to habituate the viewer to the signification without habituating them to a specific sound.

Many people naturally begin to apply this process when brought on to a new project. We think in terms of sound symbolism, try to reveal facets of characters by the classes of sounds we attach to them, affect mood and tone, and look for ways to distract and enlighten the audience. My only goal for this topic is to help you understand that you are constructing a language of sound…and maybe push you to run with that idea.

Part 2 of this topic will look at a few examples soundtrack semiotics in action. Keep an eye out for it.