Cross-posting from my personal blog.
Let’s start off with a disclaimer.
I am no expert in linguistics and semantics, nor would I consider myself truly conversant in the many critical models employed in film theory/criticism. Semiotics was something waved in front of my eyes a couple of times during undergrad (where I did not studio audio or film, by the way), then explored in much greater depth during a course in my graduate program. For this article, I’ll be falling back to some of the general concepts of Semiotics. They are the points that have truly stuck with me over the years, and they easily apply to many facets of media. They can help explain the significance of events in a narrative, the choice of words in dialog, or the functions of shot composition and sound design.
So, what the heck is Semiotics? To be honest, it’s a lot of things…
This article will be focusing primarily on some of the concepts developed by Ferdinand de Saussure. Specifically we’ll be dealing with the idea of “Signifier” and “Signified.” The notion of common historical acculturation and habituation will also come into play (though I can’t say I know if Saussure is responsible for that idea as well).
At the most basic level, you could think of the signifier as the pointer file in your project that tells your DAW which audio file to play. In this case, the audio file is the signified. The signifier/signified relationship is a code system we use everyday to convey and understand meaning. It’s language. Your DAW understands that a pointer file really represents the physical location of a set of data. The process is the same when I tell you to imagine a chair. The word “imagine” tells you to picture in your mind’s eye the appropriate piece of furniture indicated by the word “chair.” Notice in that example that I’ve really only used signifiers to communicate abstract concepts.
You might argue that a chair is a physical object, but “chair” is really more of a category. It’s highly unlikely that any two people reading this article imagined the exact same chair. The chair you imagined is probably a physical object you encounter in any of the number of spaces you occupy in your life. “Chair” calls up a string of properties in your brain that associates the word with an item that fulfills those properties. So, here we all are…sharing an understanding of the word “chair” and the function it implies, but holding discrete interpretations of its physical meaning…which brings us to our next point.
The connections that exist between signifier and signified are culturally dependent. If you were to read the word “voiture” with no prior understanding or exposure to the French language, you would make no connection to the signified…much the same way “car” would confound someone perfectly isolated from exposure to English. But this concept of cultural connection extends down to a more granular level. Your physical location and the people around you exert just as much influence in the connections between signifier and signified. [Did you just make the connection that it affects the language you speak?] Consider some of the cultural language differences between Britain and the U.S.A; both English speaking countries. One extreme example is the word “fag.” In Britain it’s something you smoke. In the U.S.A. it’s a derogatory and discriminating term used by insecure idiots.
So the connections, and there can be many, between signifier and signified are context and culturally dependent. The extension of that idea is that those connections are also continuously in flux. Additionally, meaning is not an asset exclusive to written and spoken language. Actually, language isn’t even necessarily restricted to written and spoken words. Meaning can be conveyed through actions, gestures, images, sounds or even situations. What gives these different signifiers meaning, a connection to the signified, is the shared cultural history of a society. Signifiers point to additional meanings or take on entirely different ones, while new meanings and concepts in need of signifiers are continuously developed.
Hopefully, you’re starting to see how this applies to sound design.
I’m of the belief that sound design is an informed process. Let’s pause to look at a definition.
verb (used with object)
- to prepare the preliminary sketch or the plans for (a work to be executed), especially to plan the form and structure of
- to plan and fashion artistically or skillfully
- to intend for a definite purpose
- to form or conceive in the mind; contrive; plan
- to assign in thought or intention; purpose
Without the thought process, the planning, the meaning behind it…a sound remains nothing more than a random auditory effect placed against a visual. It is far more appropriate to think of sound as a form, or an element, of language in a piece (though I admit that this is far easier to coordinate in early/pre-production than to start in post).
There are two ways to deal with signification in sound design.
The first is to work with signification that is already inherent in a culture. The oh-so-frequently heard red-tailed hawk used to convey a sense of open and wild spaces, the sound of a heart-beat to convey tension or concentration, a minor 3rd chord to suggest melancholy or sorrow. Notice that these are conventions that have been used many times over the years. To use them without a sense of self-reference now is considered somewhat lazy or trite, but they are easily understood thanks to their historical usage. Society and culture (at least some of those around the world anyways) have been habituated to them as signifiers and can readily derive their meaning. More artful uses reference these signifiers, implying the same meaning without the use of those same tired signifiers. There are dangers with using existing, well known, signifiers though.
Besides the aforementioned “overused” issue, this approach presents a potential problem…it may pull in additional unwanted meanings. Because these significations are accepted thanks to historical habituation, it means they may have acquired additional specific meanings beyond the general/original meaning. The red-tailed hawk can detract from the grandeur of the shot in a dramatic film if it causes one to think of all the times it’s been used in comedies recently. [A crude example, but it illustrates the point.] This additional meaning problem is one reason why most productions opt for new custom music, rather than using well known pop music. These new meanings bring in outside associations that can cause a shift in perspective for the viewer. On the flip side, early trailers for new films often use music and sounds from older, popular, films in the hopes of creating that connection to generate more excitement. What works well for advertising rarely works well for narrative, though. And working with existing signifiers requires a careful balancing act.
The safer, though perhaps more difficult and satisfying, method for working with signifiers in sound design is to develop a new “language” for each production. Remember that signification is a function of exposure, a collective agreement by a culture or society that connects signifier and signified through repitition over time. This means that it is possible to attach meaning to sounds exclusively with the audience during the course of a piece. The more instances an audience has to observe the connection, the more ingrained it will become. In his book, Thinking, Fast and Slow, psychologist Daniel Kahneman states that stimulus in our environment affects the way our brain processes information (in immediate ways). He also says that our minds have the predilection for finding patterns (frequently, even where they do not exist). [The guy has a Nobel Prize. I’d like to think he knows what he’s talking about.] This gives sound designers the opportunity to subtly push the audience, twist their perception, and add or clarify meaning through the use of sound.
The difficulty is in the fact that the signifiers have to be used enough for the brain to make the connection, but without creating repition that causes the viewer to disregard them. Habituation to specific sounds makes us stop attending to those sounds; unless they occur in a situation where they normally wouldn’t. That’s the trick, finding ways to habituate the viewer to the signification without habituating them to a specific sound.
Many people naturally begin to apply this process when brought on to a new project. We think in terms of sound symbolism, try to reveal facets of characters by the classes of sounds we attach to them, affect mood and tone, and look for ways to distract and enlighten the audience. My only goal for this topic is to help you understand that you are constructing a language of sound…and maybe push you to run with that idea.
Part 2 of this topic will look at a few examples soundtrack semiotics in action. Keep an eye out for it.
Robert Reilly says
This article was wonderful. Its a beautiful perspective on the aspects of culture and sound and really opens the door to new opportunities for creative design. Can’t wait to read part 2!
Graham Donnelly says
Great article and I like the direction this topic is going. Thanks for sharing this detailed view and insight. I look forward to the follow up.
Quick note though, here in Britain we do not smoke Faggots, that word has the same meaning here as in the USA, we do refer to cigarettes as Fags though :)
Shaun Farley says
Thanks for the correction, Graham. Updated the piece to maintain accuracy. ;)
Thank you for the interesting piece. I like that you emphasize the importance of having an underlying idea or direction that control the sound. I work with game audio now, and a major part of my work is figuring out how to add nuances and subtext to what is already there in the story and graphics, which sounds and how they are placed. Look forward to pt. 2 :)
Ro Shaw says
I really enjoyed reading this, thankyou for sharing. I attempted to use the second technique in micro and macro forms across the story arc of Alice: Madness Returns.
If you’ve played Mass Effect 3, it utilizes some musical motifs effectively to add significance without habituation, you can see it coming but its smoothly executed.
To use considered and crafted material, be it audio, use of specific words, art etc, to push audiences perception is, to me, the tip of what makes ‘Signifier’ and ‘Signified’ important. You mentioned subtlety in pushing an audience, and I think behind this is where a huge creative battle is waged. Arguably the best written descriptions, at least in fiction, provide those signifying directive cues, but leave enough for the readers imagination to fill in the gaps, in some % forming the signified themselves, in my experience this can make for a more powerful and engaging experience.
In terms of a ‘language of sound’, its similar to whispering in someones ear, and it can be used with literally every sound that a player hears over the entire course of a game, be it 2 or 200 hours.
I’m looking forward to Part 2!
Error in title–spelled Semiotics wrong….
Shaun Farley says
So it is, Karen. That’s awfully embarassing. Thanks for pointing that out, I’m a horrible typist.
what brand is that portable recorder on the pic?
Shaun Farley says
Richard Gould says
Couldn’t agree more. Wonderful topic and article. Sound Designers should always consider the audience’s relation and preconceived feelings towards certain sounds. In fact, it’s a wonderful asset to have. Music shares this but the harmonic and melodic language differs from culture to culture. Pure sounds meanwhile I believe are more universal in their connotation. What sounds cute, scary, happy? These are more universal and evolutionary traits from shared experiences. Can’t wait to read more.
Pawel Andryszczyk says
Thank you for this article. I find this kind of approach (philosophical, language-wise, cultural) to sound design equally, if not more, important than all technical, DAW+GEAR centered discussions. Looking forward to the second part.
Alexandre Saba says
I really enjoyed this perspective, great article Shaun! Conditioning players and listeners can be a very useful tool if you intend to surprise them with the opposite or point at the obvious. The cultural aspect is also something we need to be aware of and research before hand. Looking forward to the next one!
Tom Todia says
This was a very elegant way of explaining the potential use of a signifier. Especially when it comes to undesired consequences of its cultural baggage.
This article has been very useful, I’m writing a paper about the characteristics of the sound design of a videogame based on a semiotic model. Great article by the way, I’d like to contact you!
Shaun Farley says
you can e-mail me at shaun @ this website