Aaron Marks Special: Disciplines of Sound Design, Required Sounds, Sound Design Thought Process and Development Cycle

This is an excerpt from Game Audio Development. It’s used by permission of Delmar/Cengage Learning.

Chapter 3 – Disciplines of Sound Design

There is a variety of disciplines requiring proficiency for a successful game sound design career. Since games utilize an assortment of sound styles, a good sound designer will need the knowledge to work expertly within each area to create sounds that are appropriate and of high quality. In the film and television industries, sound design disciplines are highly specialized and require specific talents and years of experience to be considered respectable. We’ve discussed previously that a career in sound design is within reach of those who have a good ear, engineering and production chops, and are proficient with audio editing software. The next step is to refine these skills further and apply them directly to a specific discipline.

Foley

Foley (named after Jack Foley, one of the earliest practitioners of the art in the Hollywood film industry) is the art of creating general movement and object-handling sounds that are in sync to onscreen actions. In film, this is used extensively to create or sweeten existing footfalls, clothes rustles, body falls, hits, weapon handling, and other general sounds a character would make while moving through a scene. Since the dialogue is the most important sound being captured at the time, these other movement sounds are missed or purposely deadened during each take with the knowledge they can be recreated later during foley. To accomplish this task, foley artists will gather suitable props and set up insulated “pits” with the appropriate surface types and make a few run-throughs before an actual recording. When the artists are fully practiced, they will then “perform” the scene and recreate the missing sonic elements in time to the visuals. Foley work is like working in “stealth mode” because it works best when it isn’t noticed.

In games, “foley” refers more to the type of sound effects rather than the methods used to create them, although they can be captured similarly. Many games developed today require sounds such as footfalls, body hits, or object-handling noises for the purpose of adding a sense of realism to what the player is experiencing. The main difference between film and game foley work is how the sounds are “performed” and how they are ultimately implemented into the game medium.

Game cinematics are the most similar to film with regard to foley. The onscreen action is defined, happens predictably, and is the same no matter how many times it is viewed. Sound designers are tasked to create each sound within this mini movie—from simple footfalls to massive explosions to everything in between.

The foley sounds can either be “performed” to the pre-existing cinematic or they can be recorded and edited individually and later synchronized during the audio post-production process. Either way, the end result is nearly the same. The lines may be a bit blurred between the “types” of required sound effects, but this is where being familiar with each discipline of sound design really comes into play.

Footfalls are by far the most performed sound in film and a good example for discussion. Since they are recorded live while the foley artist is viewing the scene, each footfall is naturally distinct. The pitch, intensity of the step, and variation of the surface will change each time—whereas no two will ever sound the same. Within the game environment, memory constraints often limit the amount of sounds available—and for sounds such as footfalls that may be repeated continuously, there are sometimes only a few variations that are actually implemented.

Repetition is not the most preferred use of a sound effect, and every effort should be made to ensure that players never notice they are hearing the same sound repeatedly. For footfalls, having several individual sounds to trigger randomly is one way around this pitfall. The sound engine can also be programmed to alter the pitch and volume of each step or add effects processing, such as reverb, as the repeated sound is triggered. Either way, the production team should keep repetition of sounds to a minimum and strive to keep the playback of each sound fresh.

Remote Recording

Obtaining sounds remotely—outside of an acoustically treated studio and in uncontrolled conditions—is the extreme end of sound recording and something that game sound designers do quite frequently. Not only do they need the patience and skills to accomplish the job effectively, but specialized equipment is required to make the most of what can often be the absolute worst of circumstances. Game projects, like any other form of entertainment, rely on the sound professionals to give them their own “audio” identity; since everyone has access to the same sound libraries, the only way to ensure originality is to grab the remote gear and hit the road to capture previously unrecorded sounds. Not only does this fulfill the need for fresh sounds, but it also ensures that the copyright ownership of the recordings can be transferred to the game developer if required.

The game project will dictate the type of sounds needed and the preferred method for obtaining them, with budget constraints and contractual points further defining the path required by the sound designer. Large budgets will usually allow for field recordings to capture unique sounds; in this situation, the developer usually insists upon full ownership of any recordings the sound designer makes while performing work for the contract. Smaller-budget projects might rely more on sound library recordings or will allow the recordist to retain the rights to the sounds captured. Many options are possible to meet the needs of all parties involved.

Remote recording typically encompasses sessions that can’t be achieved within a recording studio due to safety, health, or size issues, or even scheduling conflicts. It’s extremely impractical to record a race car engine, cannon shots, or smashing debris in a studio for many reasons; the only reasonable solution is to take the studio on location. With battery operated recorders and microphone pre-amps—and a variety of sturdy field microphones, stands, booms, cables, and bags of necessary extras in hand—the sound designer is fully equipped to ensure that at least the technical requirements are met.

Even with the right equipment, recording in the field can be fraught with unpredictability. The location of the objects or environment being recorded can also make a huge difference in the process. It is entirely possible to capture solid, useable sounds in between jet arrivals at the local airport, but this situation would be totally unacceptable for quieter, ambience effects. For those, selecting a better time of day or night might be the best way to avoid unwanted interruptions.

Remote recording can be quite a challenge regardless of the experience level. Rain or high winds can devastate recordings and equipment. Blistering heat or piercing cold not only affect the gear but the operators as well. Mechanical issues with the recording devices and the objects being recorded often bring the session to a grinding halt. Uncooperative bystanders can interfere at the most inopportune times, rendering takes unusable. Animal subjects almost never perform on cue—and when they do, the sounds they make aren’t the ones needed. In the end, remote recording is a test of patience. Since the variables are great, almost nothing is under your control and it often takes an extraordinary effort just to be there in the first place. The sound designer has to be ready for anything.

In-Studio Recording

In-studio recording is the most desirable method for capturing quality game sounds. Not only are the conditions controllable and predictable, but studio gear is often of higher quality than smaller, portable equipment. Most large game developers have spent thousands of dollars on acoustically designed and well-equipped recording spaces for their in-house audio teams. Third-party contractors spend a large part of their budgets on recording equipment and also take great care with their own personal spaces. Even those who don’t have defined areas understand the importance; they typically ally themselves with commercial recording studios when the need arises. Recording spaces are an important tool in the game development process.

Beyond sound effects such as car crashes, jet engines, and explosions, studios are useful for practically everything else. From the subtle sound of an expended shell casing hitting the ground to the dynamic strike of a crowbar forcefully colliding with an old car hood, the controlled conditions of the studio will make the job easier and give better results. Random outside noise is seldom a factor, and the sound designer can focus on creating the perfect sound instead of having to rely on luck out in the field.

The best part about working in the studio is the abundance of quality tools close at hand. Whether sounds are being recorded into a top of the line Pro Tools system or through an analog board to an analog multitrack tape deck, the chances are that the gear being used is far better than any remote gear available. Outboard processors offer choices of preamps and real time effects that can be invaluable during the creation process. Let’s not forget the security of having backup equipment available in case something becomes inoperable. Studios are definitely the first choice when recording game sound effects.

Original Recording

Most creatures, vehicles, weapons, and alien planets depicted in games don’t actually exist in the real world. Due to this conundrum, the simple task of sticking a microphone in front of something and recording it won’t come close to producing the results a game may require. Using sounds that are associated with familiar objects in real-life risks breaking the immersive effect of the audio—something that must be protected at all cost—so it’s not prudent to even consider that route. Instead, game sound designers turn to another one of their many skill sets to provide the answer: original recording and development.

There are thousands of ways to create “make believe” sounds, but it takes a measure of skill to ensure that it is done believably. The type of sounds needed will drive the initial creative direction, whether it is based on something real or something completely fabricated. For example, we can assume an alien spaceship must have a power source to propel it through the depths of space.

Creating the engine sound could easily be based on a pre-existing rocket or jet engine sound, or even sound from a vacuum cleaner—anything that gives a sense of “power.” These sources could be driven though various effects processors or plug-ins to give the appropriate “alien” feel to them and then layered to create the final sound. These types of sounds, based on real objects, work well because they give a sense of familiarity but are different enough to be believably alien.

Contorting pre-existing sounds isn’t the only route the sound designer can take in the original development process. Sounds can be generated entirely from scratch through a variety of sound sources such as keyboards, sound modules, virtual instruments, samplers, or tone generators and then processed and edited to create the needed effect. This method is often preferred in instances where pre-existing samples are unavailable or when the overall audio theme prescribes a more electronic feel. Using the previous alien spaceship engine sound as an example, this can also be created believably using this method. A simple low synthesizer note could be linearly pitch shifted (some audio editors refer to this as “pitch bend”) and edited in such a way to replicate an electronic sounding “power up” effect to serve the purpose. There are also times when layering “electronic” with manipulated pre-existing sounds will produce the perfect sound effect. The key is to be open to trying whatever it takes.

Sound designers also have the option of utilizing specialized software or effects processors designed specifically to create unique, one-of-a-kind sounds. Sound-mangling software in particular generates its own signal, adds effects processing, and morphs it into the direction of the user for some often spectacular results. While the sounds created are unique, the disadvantage is that the amount of experimentation often needed to create anything useful can slow the creative process. Knowledge of the software’s capabilities will streamline the effort and maintain productivity.

Sound Libraries

Sound libraries can be lifesavers when creating sound effects. The advantage of sound libraries is that they provide a wide selection of sounds in many formats (e.g., DVD, CD, or download). These high-quality recordings save massive amounts of time and place the expense of creating them on someone else for a change. The prime disadvantage is that everyone has access to the same libraries, and it can be tough to maintain any kind of originality. Many times an overused sound will actually distract players as their minds associate it with something else they’ve heard in the past. Breaking the spell of the game can be avoided by staying clear of such overused sounds.

Using sound libraries and being familiar with their content is a skill acquired with experience. Good sound designers can match particular sounds to their associated libraries (usually after hearing them on a movie or television program), and this familiarity translates well when knee-deep in a project of their own. Grabbing the exact library and file you need saves enormous time and keeps the creative process rolling, something which is appreciated with a rapidly approaching deadline. Knowing how to effectively use sound libraries is an essential skill.

Sound libraries typically contain regular audio or digitized files (e.g., .wav) and are also available in CD, DVD, and occasionally hard drive formats for the large collections. While sound designers have the option of auditioning and saving audio tracks straight from disk, most will save each library as digitized files for easier archiving, faster auditioning, and streamlined use. Since sound for games is digitized as part of the sound design process anyway, doing this step beforehand can save an incredible amount of time over the course of a production.

Being familiar with each library is essential and will help make the creation process more efficient. A general knowledge of the sounds available—their quality, usability, and how they are labeled—will simplify the process. In order to manage literally millions of separate sounds, it’s common for sound designers to make use of a variety of search tools—utilizing keywords such as “stab,” “zap,” or “whoosh” to find a specific sound. These functional search tools can be found online at vendor websites or packaged with each library.

Determining Required Sounds

“During the preliminary planning phases of game creation, the sky is the limit. The game will have brilliant 3D graphics, intuitive gameplay, a full orchestral score, Star Wars quality sound effects—and it will all be done in six months for under $100,000.” If only the reality of game development were that optimistic. Plans change and compromises are made due to many stumbling blocks that present themselves along the way.

On the sound front, many developers feel that everything that looks like it can make a noise has to have a sound created for it. A sound asset list is fleshed out, and the sound designer is given the daunting task of producing each one according to this initial assessment. However, as the development cycle progresses, it becomes painfully obvious that this first plan is overly ambitious and adjustments have to be made. Initially, the game platform’s memory and processor restraints become apparent, as artwork and other assets compete for a defined capacity. Not everything will fit through the pipeline efficiently, so the number and size of the assets are reduced. Disk or cartridge space on which the game is to be stored also has a finite volume—and, once again, assets are reduced or compressed. Unfortunately, the audio is usually the first to fall victim—and drastic steps are taken to make it work.
Sound effects that were presented in the first sound asset list are reevaluated with the idea of reducing the amount needed. It’s often an unpleasant task, but there are obvious choices that can be made. Footfalls, gunshots, explosions, and any other sound that may be heard multiple times can be strategically cut back. It may have been a great idea to reduce repetition by having a dozen different footfalls for each surface, but this could easily be cut in half if needed. The same could be done for gunshots or explosions as well. Since these types of sounds can be coded to trigger randomly, the development team can get away with using fewer of the same types of sounds in order to save space.

A simple test can be constructed to determine how many variations will give the illusion without being repetitive. During chaotic gameplay, typically found in action games such as FPSs, a player can easily suffer from sensory overload. With explosions, gunfire, screaming, ambience, environmental sounds, dialogue, music, and everything else playing at once, the soundscape can quickly become a nightmare. Even if everything has an associated sound to it, this doesn’t mean they have to all play at once!
Developers select sound effects based on their entertainment value and those that best convey what is happening in the game at that moment. When sound effects are implemented, they will be prioritized in order of their importance with these ideas in mind. In an FPS, gunshots, explosions, voice commands, and character sounds will have a higher priority than sounds which occur in the distance or off screen. As a game moves further into its development, sounds that are consistently tagged as low priority and serve no specific purpose might be dropped in order to save processing power or space. There’s a delicate balance between creating a believable soundscape and confusing the player with the bombardment of sound. This is an issue definitely worth considering for the sake of a game’s quality.

:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Sound effects serve a very specific role in video games; they entertain, provide feedback, present important information, and breathe life into the virtual world. Without them, the experience would be far from satisfying. Sound designers have a huge responsibility to the development team, the game itself, and the player, to create and implement an effective soundscape using a wide variety of unique skills and tools. Game sound design is not an easy job by any stretch of the imagination, and doing quality work takes effort. By establishing a solid knowledge base—and having the appropriate equipment and the skills to do the job well—the sound designer can be a powerful force on the game development team. While past stigmas still haunt the audio side of this business, players are pleasantly surprised and impressed with the utility and quality of today’s sound effects. All it takes of the sound designer is well-directed effort and the desire to do the job well.

Chapter 4 – Sound Design Thought Process

There is a certain mindset the sound designer will employ as the actual creation process begins. Understanding the purpose of sound effects will serve as a good start, but going back a little further into the human psyche will shed additional light on the subject. What distinguishes average sound designers from great ones is their use of sonic elements to actually influence how players feel when they hear the sound.

A little understanding of human psychology can allow the sound designer to create sound effects with emotional impact. We have been conditioned all our lives to respond to sound—such as answering the telephone when it rings, going to the door when there is a knock, and running out of the classroom when the bell sounds. A siren, scream, or loud bang elicit a specific response: to immediately evaluate any threat and our well-being. Other sounds cause us to react physically—such as fingernails scraping a chalkboard, a snarling dog, or the grinding crunch of a breaking bone; these can cause discomfort or fear. A great sound designer will take advantage of these pre-conditioned responses, creating intense and emotionally charged sound effects.

As the sound designer reviews the sound asset list, the next step is to formulate an approach to creating them. Professional sound designers are always listening to the world around them. This simple act hones their creation skills and helps them determine how to go about capturing or recreating any game sounds from scratch. Can the needed sound be captured, or will it have to be manufactured? Is the sound of the object being portrayed “real” enough, or should another sound be used instead? Unfortunately, not all actual sounds are interesting enough for entertainment purposes and will often need a little enhancement to make them work. Either way, these types of creative decisions need to be made early in the process.

A gunshot in a film is typically a loud, forceful sound that portrays the urgency and power of the situation; it grabs your attention and makes a very influential statement. In reality, a gunshot may carry the same strong statement in the action itself—but its sound is nothing more than a simple, unexciting “pop.” A game that is trying to engage a player and evoke a strong emotion can’t do it with a toy gun that goes “pop, pop, pop.” Instead, it has to convey the fury with something much bigger such as a “BLAM!” However, if the goal of the game is to be as realistic as possible, then maybe other sounds or musical accompaniment will be designed to provide the emotional statement.

Creating a gunshot that provides guttural satisfaction every time the trigger is pulled requires a little creativity. It would be logical to start with an actual gunshot, but this is not always necessary. Finding or creating other elements with the same sharp, explosive characteristics is all it takes to make an exciting and satisfying sound effect. One or several of these can be layered appropriately to create an original sound effect that will do the trick.


When faced with any sound effect, it is a good idea to strip the sound to its basic elements and create with these in mind. A gunshot is a quick, explosive sound that has a loud, fast attack and then decays rapidly. What other more available sounds contain these types of characteristics and could be used to create or enhance the sound? Consider the crack of a high pitched snare drum, the hit of an oak axe handle, the burst of a balloon, the smack of an empty tennis shoe on a concrete floor, the slam of a wooden cabinet door, or a pre-recorded explosion pitched higher all as acceptable elements. Layer these sounds, apply appropriate equalization (EQ), add a little reverb—and it’ll start to sound pretty good. By shifting the focus away from just simply recording the gunshot, experimentation will lead to better, exciting sounds that really make a statement. This fundamental thought process will lead sound designers down some interesting roads throughout their careers.

The best way for a game sound designer to get into the proper mindset is to be completely immersed in the game—to live and breathe it. The sound effects creator needs to fully understand the game’s intentions, characters, and environment in order to create sounds that are appropriate; being totally consumed by the virtual world they will portray is the best way to do this. Obtaining copies of any artwork, storyboards, rough versions of the game, and completed music will serve this purpose well and is highly recommended. Most sound designers find this look into the world they are creating gives them much needed inspiration.

Game Sound Development Cycle

With the boundaries fully established, it is time to get to work. This phase is highly subjective due to the variety of opinions that saturate the production team. If the sound designer is lucky, the producers will recognize the talents of the audio experts they hired and let them create unencumbered. The pressure is already immense, but having a nervous producer constantly breathing down one’s neck can really turn up the heat. Since the producers are ultimately responsible, it is understandable why they are so concerned—but if the sound designer remains professional and does some extraordinary work, there will be more room to do the job.

Sound designers approach the creation of each sound effect differently, based on their personal working styles. Some will record or gather sound elements that they feel can be used during the creation process and use these as a foundation for most of their sounds. Others will approach each sound individually, confronting the logistics as they appear and deciding on the more purposeful approach instead. Others will have a good idea already worked out in their minds and manage it methodically, while some prefer to accidentally stumble across a workable idea. There are as many approaches to the process as there are sound designers; it’s just a matter of finding what works best and leads to the best results.

Not everything can be planned—and even if it is, there are no guarantees that the final sound effect will even work within the confines of the game. It’s a good idea to remain open to doing whatever it takes to create appropriate sound effects, even if the methods seem unconventional or are a bit outside of the comfort zone. A sound designer’s identity is not only about how well the job is performed but by the flavor and style of the designer’s unique personality and working methods.

Sound Recording

The most common technique of creating sound effects is to record them directly from their source. This ensures their originality and increases the chances of capturing the exact audio the sound designer hopes to achieve. Available sound libraries fulfill this role as well, but there is always a necessary compromise in having to settle for a sound that is “close enough.” This method is a logical approach to sound design, and maintaining creative control is a definite advantage. The main rule to respect when pursuing this particular option is to make recordings incorporating the perspective in which they will be played back in the game. It’s tempting to record everything close up to capture cleaner recordings, but the end result is that everything will ultimately sound just that: close up. Volume, panning, and reverb effects are tools that may be used to add air and provide additional depth—but nothing expresses the intent of the sound better than capturing it from the viewpoint of the player.

Background sounds and ambience should be captured from a distance as a stereo recording to take advantage of any natural spatial interaction. When heard within a game environment, these properly recorded tracks will provide a more realistic atmosphere and make the entire experience more believable. These can also be taken a step further by recording them in surround, assuming they will be played back in the game in this format. Any games can benefit from this type of treatment, especially those that depict a living environment.

Sound effects that are part of an onscreen character’s immediate vicinity are best when recorded from a medium perspective as either mono or stereo. First-person shooters (FPSs) such as Medal of Honor or Half-Life 2 make heavy use of sounds such as weapon handling, footsteps, body falls, and injuries that directly concern the player—and their realistic representation is necessary for the immersive effect. These sounds can be recorded from a closer distance and adjusted with volume, EQ, and reverb if necessary—but in order to produce sounds of the highest possible quality, this should be avoided.

Close miking is a technique that serves its own specific purpose in sound effects production and often yields interesting and sometimes unexpected results. A sound that is recorded close to the microphone brings out exaggerated elements that under normal circumstances wouldn’t be noticed. This practice not only allows for a cleaner recording—but the recorded object can be used for something entirely different, resulting in a unique presentation and a previously unheard sound.

Listening Up Close

Operating a retractable ball point pen from arm’s length produces the standard “click click” sound associated with a standard pen. However, when held against the ear, the simple “click” becomes more complex—and the sounds of scraping plastic pieces and the “ping” of the spring is very apparent. This closely miked sound could be used as a toggle switch or, if lowered in pitch, as a weapon loading sound. This method can completely change the approach to creative sound design and is well worth the effort.

The book is available at Amazon.com.

Game Development Essentials: Game Audio Development
Aaron Marks & Jeannie Novak
© 2010 Cengage Learning Inc. All rights reserved.