The April’s issue of AudioMedia is available on-line and includes an article on the sound of “Submarine“. Paul Mac talks with the crew behind the sound the film, including re-recording mixer Nigel Heath, sound mixer Martin Beresford, sound engineer and music mixer Jake Jackson and director Richard Ayoade.

“The director wouldn’t say to the actors, ‘walk four and half paces, stop for three and a half seconds, and look, and then remember your lover, close your eyes slightly’. He’ll tell the emotional story and the actors interpret it.” – Nigel Heath

Continue reading…

Medal of Honor has been one of the most important franchises of warfare video games. Most of their titles have really good sound work, and Medal of Honor (2010) is not an exception of it.

Below is an interview I had with the audio team of the game, talking about the different challenges they had and the techniques/processes applied on recording, designing and implementing the sound of this adventure.

Danger Close Games Audio Team:

• Audio Directors: Erik Kraber and Paul Lackey
• Sound Design Leads: Tyler Parsons and Jeff Wilson
• Dialogue Lead: Joshua Nelson
• Sound Programming: Eduardo Trama

Desinging Sound: With all those great Medal of honor titles already in the market, what was your approach on this title to make it as good as the previous ones but also new and different?

Erik Kraber: We come away from every project with a laundry list of things we wish we could have done better, but were unable due to time, resources, or technology. It is often hard to get past the frustration of knowing what “could have been” on the previous project, but it just fuels us to keep pushing it further each game. For this latest Medal of Honor we had quite a challenge, because it was an entirely new setting and decade for us to create within. So our massive library of sounds that we had recorded and built of every World War II weapon, vehicle and foley suddenly became dated.  MOH 2010 was about reinvention on all fronts, and audio was a big part of that. Fortunately, we had a very talented senior team of sound artists and programmers who have all worked on multiple MOHs before, so the franchise audio aesthetic sensibility and quality bar is now part of our DNA.

Our focus was to take our learning s from the previous games and go deeper on all the things we try to do on each MOH – focus on authenticity and the personal story of the soldier. We want to give the player the feeling of truly being in that world, living that experience. While not every sound in the game is necessarily realistic, they are intentionally not hyped to the point of feeling like a Hollywood blockbuster. Guns need to represent their real-life counterparts accurately but still provide distinct character and interact with the world in believable ways.  The voice acting needs to feel more like a documentary than a feature film and represent, as accurately as possible, the foreign languages of the indigenous people. The music needs to be current, exciting and fresh, but true to MOH’s orchestral roots and its focus on scoring the humanity of the soldiers throughout the story and not just support action.

DS: How was your collaboration inside the audio team and also the relationship with other development crews?

EK: Within the audio team, communication was easy, as many of us have been working together for years. We also collaborate with many external contractors for music, dialogue, and sfx and with other studios, especially DICE, who handled the development of the multiplayer portion of our game. Fortunately, as part of a big company like EA, we have the opportunity to share and learn from all the other great sound people in the numerous studios across the world.  Paul Lackey organized a Gun Summit, where we had EA audio teams who were working on weapons-based games come to Los Angeles for a week-long weapons recording session and knowledge sharing session. The result was a great exchange of ideas and approaches to sound recording, design and technology and some of the most comprehensive weapon field recording sessions ever done.

Tyler Parsons: For the single-player portion of the game, Paul, Jeff, and I each owned individual levels as well as global sounds (and focused on our own shares through the bulk of production), but we also held group play-through sessions as we got closer to final in which the entire audio team would gather and really scrutinize each level in turn. This made sure that everyone got “ears on” everyone else’s work and had the chance to offer feedback to improve the game.

Our sound designers interact with every other discipline in the game (design, background art, animation, visual effects, et al). We had a very strong relationship with the level designers – we would have frequent meetings and spotting sessions to discuss the latest changes in the levels, how we could best use sound to tell the story and enhance the experience. Implementing sounds throughout the game required us to constantly work with design scripting, the VFX and animation tools, and so forth (as well as collaborate with software engineers to draw up new audio features).  Luckily, the other disciplines were staffed by rock stars who appreciate the impact that sound can have on the player’s experience. We got excellent support.

DS: Could you tell us about your approach and designing the sounds of the weapons?  There’s a wide variety of guns and firing methods.  How this affected your decisions on the sound?

Paul Lackey: When I started authoring our weapons my initial goal was simply to be authentic and to strive for the highest fidelity in a game. I wanted guns that leapt out of the mix but with no distortion or heavy limiting so they could pop without hogging up too much space or fatiguing the ears. Fidelity came with outstanding source recordings and having a lot of options. The pop required setting the overall game volume pretty low to give the required dynamic range. However, once a few of the guns went in and were meeting my initial fidelity goals… I really wanted to reinforce that the primary way players interact in the world is through their weapons.  So gun shots needed to reflect off walls, echo thru canyons, distort thru air and be delayed by distance. Envelopes were added to provide subtle air distortion effects. Band pass filters that shifted based on distance were also used to tweak tonal qualities across the battlefield.  We sort of cheated on the acoustics and interior spaces by authoring the effects per zone rather than using the CPU to ray trace for collision. We had the tech but were already pushing the CPU so left it out. Hopefully next game. Additionally, the heft, power, and reach of particular weapons was conveyed by how much they interacted with the space. I couldn’t always make big guns louder but I could make them seem like their sound traveled further by boosting their tails and slap.  So how the weapons excited the air was used to tell something about the weapon in the player’s hand or the one firing at them. Finally, and I think a really important point, was that the weapons allowed players to reach out and “touch” the world. Of course we already had lots of bullet impacts to be heard when in close proximity to the player, but early on I found myself pulled out of the game when I shot distant rock faces but felt like my bullets passed through the geo because I heard nothing. So I worked on some very distant bullet impacts that smack the rocks, echo, and arrive via speed of sound well after you see the puff of dust. These are probably my favorite sounds in the game because they sound super cool and really do a good job of making me feel like I can interact with the world pretty much anywhere I can see.

DS: And what about the ambiences and warfare backgrounds?

Jeff Wilson: We designed our ambient backgrounds as realistically as possible, utilizing many audio recordings we captured ourselves. Wind, dirt debris and sand were components that were often mixed near field, giving the player the tactile sense of the rugged Afghani terrain. Our story often dictated that the player was a certain distance away from an ongoing battle, so we then chose distant explosions, weapon fire and aircraft to fill out the rest of the soundscape. We developed the tech to ‘anchor’ a quad ambience to the world, so we would mix the distant battle assets either stereo front or stereo rear, and then set the quad ambience file facing a certain direction in game.  This way the ambience would stay fixed to the world as the player moved in the 3D space, keeping the sound of the distant battle eminating from a specific direction.

TP: Backgrounds involving combat usually featured very distant battle elements to set the scene (i.e. distant engagements near Bagram airfield or in the Shahikot valley) with the bulk of near and mid combat sounds like weaponfire, explosions, bullet whizbys and impacts generated by actual NPC (non-player-character) events using sounds we’d previously created. To make certain battles more intense, I would create multiple versions of their areas’ backgrounds with extra layers of mayhem and then either swap to them (at an opportune moment) or layer them on top of the existing backgrounds if streaming bandwidth permitted.  I also created positional sweeteners like extra whizzes, impacts, and debris and scripted them to play at random points around the battlefield.

There are a couple of places in the game in which there’s a ferocious fight going on but the NPCs are of an inexpensive type that doesn’t generate any sounds in sync – weaponfire, foley, or dialogue.  Those moments had to be covered completely with chaotic backgrounds combined with positional sweeteners going off around the player – not unlike for a film, but with no way to sync it to the action. I think those moments ended up working pretty well despite the fact that they were completely smoke-and-mirrors,

DS: How was your workflow and process on the cinematics?

EK: For the pre-rendered cinematics, we handled them the same way post is handled on most animated films. We recorded the actors interacting with each other either on a stage or in a studio and assembled the dialogue into a cohesive radio play. That was delivered to the art department to use as their guide track for the mocap actors and animatiors. There were certain situations where rewrites or re-recordings had to be done to accommodate changes to story or technical issues, but mostly we were able to get the feel right on the first pass. Once the final animations were done they rendered a final movie file for us to work on in post. The sound editing work was divided between me and Earbash, whom we love to work with and have for many years. We use Nuendo here at EALA, and it has excellent mixing automation tools to handle complex multichannel mixes.  All of the mixes were done in 5.1. Our biggest challenge was trying to create an experience that never pauses. If you noticed, we have no loading screens in the game. To have that seamlessness, we split the music out as a separate track from the beginning and ends of the cinematics, so we could pre-cue the music for the cinematic before the end of the level and have the music continue on after the movie into the beginning of the next level.  Getting the mixing just right at the 11^th hour was really tough, but we are really happy with the results.

TP: Many cinematics were handled in-engine as opposed to cutting to pre-rendered movies. I would do a video capture of these sequences when they were as close to complete as possible, then create a one-shot bed of sound covering everything needed for a sequence. Depending on the way the scene was being handled (for example, whether or not the player has control of his movement or view, whether the player’s actions can change the length of the sequence, etc.), I would bounce specific elements of the cinematic separately so they could best be fit into the scene: positional elements in their correct places regardless of player movement, elements that need to persist playing as loops, and so on.

DS: All games, specially warfare stuff like Medal of Honor have constantly big amount of loud sounds.  How you dealt with dynamic range?  What challenges you found in the mix?

PL: We have a very rich mixing environment so I could modify groups of sounds based on lots of simultaneous game events. Much like ducking for dialogue but used when the player shot, sprinted, was hit by bullets…etc. This allowed key sounds to stand out without having to make them overly loud. This was ideal for sounds that occurred a lot, but for big moments… The headroom was there to let things punch up. One of the more challenging things we went after was seamless audio transitions between our levels. On paper this was a one line task, but to accomplish required lots of crazy logic to trigger and set volumes for music that bridged the gaps between levels and cut scenes, set cut scene volumes, cleanly handled fading out game play audio while pulling in assets for the next level …all while the levels that initiated the transitions got unloaded. This turned into a fairly big headache during our final days but was a bit of polish no one on the audio team could live without. Seriously a dedicated team.

TP: Some of the space for dynamics was provided by the script – the story features lulls as well as peaks, so we weren’t tasked with creating a neverending battle that’s always cranked up to 11.  The mix system we created for Medal of Honor allows extensive control of dozens of different buses and was used to make detailed, deliberate mix changes on the fly based on nearly any game event.  It let us easily trigger global or level-specific changes to carve out space for punchier explosions or weaponfire – or suppress or filter what would normally be loud in order for quiet details to peek through.

DS: How you deal with perspectives and space from the recording and also in the design and implementation processes?  What’s the approach on getting spatial realism in the game?

PL: Our approach to perspective was to record from all the distances we needed and then layered them in game. I didn’t rely too much on distance based low pass because distant sounds already sounded distant in the recordings.  I actually applied a good amount of high pass based on distance to keep things from getting muddy, and then as described above for weapons…implemented envelopes, delays, and filters to simulate air distortion and reflections. The other thing we did was to record our weapons in two locations. One shoot was in the mountains and was very clinical about capturing the authentic voice of each weapon plus we got nice long mountainous tails. The second shoot took place in an urban setting and was focused on capturing reflections from inside buildings, courtyards, alleys, and on rooftops. A trick I learned a few years back from a colleague at DICE was to add the same environmental gun tails to every weapon in a space. This really serves as a sonic anchor by putting each gunshot into the same location.  I would vary pitch a good deal on these but at their core…every gun shared sets of interior and exterior tails.

JW: While recording weapons, we set up our microphones at multiple distances to capture the weapon fire with a built in worldized effect. These assets were then edited into three distance models:  Near, Mid, and Distant. Then in game, we dynamically blend between these three perspectives to create a realistic sounding weapon at any distance.

TP: Many sounds (particularly weaponfire, but also a lot of foley and hard effects) were recorded from multiple perspectives to allow us freedom in creating our final game assets. Character foley usually had a close mic and a distant mic; vehicles were recorded from onboard and third-person perspectives as the gameplay required; weaponfire had as many as 72 channels of audio being recorded per gun, miked from close, medium, medium distant, and extremely distant locations. We also worldized dialogue assets in various canyon locations to give some of our distant dialogue a realistic sonic character.

Beyond the recording phase, we added spatial realism using our mix system. To handle environmental acoustics, we set up a series of plug-in chains whose parameters can be updated by any game event (such as entering a specific location).  Combined with the ability to modulate bussing to and from these effects, this let us adjust reverbs, delays, filters, and so on to match any space (or condition) the player was in.

DS: In an interview for Sound & Picture magazine, Tyler Parsons talked about using worldizing techniques in the dialogue.  Did you use that technique for other sounds as well?  How was the reverb used to reinforce the environments?

PL: We did not run anything besides dialogue through speakers , but we did do a lot of Foley and dirt/debris recordings in some quiet desert washes where we were able to capture the space of the place in our assets.  The results are so much better than close micing in a studio and then trying to simulate outdoors. Reverb was used in game for interior spaces.  Two slap delays were dedicated for the exteriors.

TP: We also worldized “Purple Haze” and a few other songs while we were out there, but somehow they didn’t end up in the game. We did of course record some sounds (weaponfire being a major one) in outdoor spaces like canyons or mock urban environments that gave us the sound of the space as well as the direct sound.

Several of our plug-in chains were set up with particular canyon or hillside spaces in mind, containing combinations of reverbs and delays that would create the aural characteristics of those areas.  We also created plug-in chain templates and mix setting templates for the different interiors in the game (caves, warehouses, huts, vehicles, various rooms, etc.).

DS: And what about the implementation process?  What were the challenges you found when integrating sound in the game?

TP: Despite our tools and overall pipeline being pretty sturdy by this point, we still had to regularly repair or re-implement a handful of sounds that would get accidentally disconnected by other departments’ last-minute changes (especially as things got busier toward the finaling phase).

Eduardo Trama: On the technical side, we had three major systems to balance between the three platforms – PS3, Xbox 360 and PC. They were memory limitations, CPU usage and streaming bandwidth. As we started adding/integrating sounds into the levels we found issues within these 3 major areas and worked on tuning each sound per platform to accommodate their varying restrictions. In some cases it was as easy as changing the compression rate, whereas, in other cases we had add extra technical features to reduce memory, CPU and/or stream usage. The challenge was that those system resources were shared with everything else in the game (e.g. physics, AI, graphics, animation etc) so we had to be clever and vigilant about keeping everything balanced and sounding right within these restrictions.

DS: What were your goals on blending music and sfx together in order to reinforce story, emotion, and other elements?

EK: Music has always been a very special part of the Medal of Honor franchise, with incredible scores written by composers Michael Giacchino and Christopher Lennertz. Moving to modern day we realized that the classic orchestral score was not going to be the right approach but we wanted to retain the heartfelt emotional quality that reflected more than just “action”. For this game we found Ramin Djawadi, an extremely talented composer who had many film and television credits – including Prison Break and Iron Man – but had never worked on a game. He had a great sensibility about how the score could fit emotionally between all the density of gunfire and explosions. We decided early on that we didn’t want the score to be a constant presence, as it had been in some of the previous titles (and with scores that good, can you blame us? J), but rather an element that comes in at the right moments to support action or stir emotion. We felt that with more areas where you just heard the detailed work of the combat soundscape, it would feel more real, more documentary-like, and then when the music came in it had more weight because of its absence. While at the end we had more music covering the game than we had originally thought we would (Ramin’s score turned out amazing) we still have much more of a range of music presence than before.

Because the score was a combination of electronic, solo ethnic and orchestral instruments, we had many layers of elements to work with. During the orchestra recording sessions on previous MOH games, we would have certain instrument groups not play on multiple passes ( e.g. have the entire orchestra play once, then have the strings not play, then have the woodwinds not play, then the brass, etc). This gave us several submixes with specific groups of instruments allowing us to build the intensity of the score with both off-line editing and in-game dynamic switching. But even with those submixes we were still limited to playing entire groups of instruments and not individual instruments. With Medal of Honor 2010, we were able to pick just tiny individual instruments or voices and build textural pieces that would subtly underscore the scene and foreshadow the full piece later in the level. The result was a score that enters the back door rather than the front.  The score creeps up on you more.  At the end of the day, while it is not as thematic as previous scores, I feel it has been the best at really tying in to the on-screen action and story by melting into the rest of the soundscape in a way that feels natural and cohesive.

Lastly, we wanted to go against the grain of the action in a number of spots in the game. While the default expectation is to score the action, Ramin and I looked for moments when you would score the emotion of the story, rather than the combat. An example of this is during the end of the Belly of the Beast level. With you and your fellow soldiers pinned down at a defend point, Tech SGT Ybarra, who has been desperately calling in for some support for some time, realizes that the enemy force has overwhelmed them and that they aren’t going to survive the attack. He radios back in to call off the air support, wanting to make sure that more American forces don’t get killed trying to help them. Within the hail of bullets from both sides, rather than score the intense action of the moment, the music scores the feeling of accepting death and acknowledges their quiet heroism. The SFX start to slip away and the slow string piece rises as the enemy forces push in to surround you and your men. This approach gave the game heart and connected the player with the tragedy and triumph on a more personal level.

DS: How challenging was the dialogue production?  How it was done?

Joshua Nelson: We were intent on crafting engaging, realistic and memorable game moments for Medal of Honor, so our team, including Writer Mike McCarthy, worked closely with military consultants throughout production. Our Tier One contacts shared how they communicate with one another and how they react to different situations in the field. When we shared game scenarios with them, they offered advice on how they would handle those scenarios. Mike did a lot of research and took much care with getting phrases and lingo correct for the dialogue. I think this helped tremendously with the feel, authenticity and coolness factor of our MoH characters because we didn’t want them to feel like “Hollywood.” That said, there are some great catch phrases that characters, like Voodoo, have in our game. Many of them come directly from Tier One Operators who tell us that’s just what they say when in the field.

Over the course of production we recorded much temp placeholder dialogue to test different combat scenarios and storytelling moments. We did multiple iterations of temp recording for each level and noted things like intensity and whether a character is winded from running. During each iteration, we worked to more accurately convey scene context and character realism through the dialogue. Later during final recording with the voice actors, it helped to have specific notes about intensity and contextual requirements, because it is easy to overlook whether a particular “Yes Sir” in a script of 9000 lines needs to be delivered calmly, stealthily, or over the noise of combat.

We recorded about 6000 total lines of Arabic, Pashto and Chechen dialogue for the foreign characters in the single player campaign. This included randomly triggered contextual dialogue for combat, and also many short conversations between 2 or more persons for use with enemy patrols and camp positions. We worked closely with translators who coached our actors in the recording studio.  They also served to change common English idioms to their Pashto, Arabic or Chechen equivalent. Literal translations sometimes don’t make sense, so they told us when something else would be said. Getting the foreign dialogue into the game is a fun process because the in-game characters begin to come alive, and co-workers in the EA office start shouting memorable enemy catch phrases at me when I pass them in the hallway.

In most levels of the single player campaign, the player receives communications through a radio com, which meant we would be radio processing a large number of the dialogue assets. I worked with Paul and Erik to get the right combinations of static, fuzz and background noise for characters. Erik wanted the radio transmission’s processing to reflect the sender’s proximity to and level of involvement with the player position and current situation. Squad members near the player had a clear channel, while characters who were not immediately near the player or who might be in trouble had more interference and background elements. A character in a dire situation might send a transmission that makes the player think “wow…that character could really use my help right now; maybe I can get to him later in the level,” so we looked for ways to reinforce this with the processing.

Another effect that we worked for was a radio doubling of character voices in close proximity to the player. In single player, when one of the squad members is nearby, the player can hear the character’s voice in stereo or surround sound space, absent of radio processing but effected by the local environment. The volume of the character’s voice would vary by distance to the player. The player will also hear the line through the radio com at a constant volume, processed and with a short delay. The combination of both sources together produced a nice effect, especially when the distance between the player and character varied over duration of a conversation. Since we did the radio processing on the sound assets themselves instead of dynamically processing them during gameplay, we actually had two WAV files for every dialogue line that had a radio component. This was a lot of work to edit, process, and get working in-game, but the result was worth it.

The following is an exclusive interview with Supervising sound editor Mark A. Mangini and Sound Designer Dave Whitehead about their work on “The Rite”.

Designing Sound: What ideas did Director Mikael Hafstrom have about the sound design for the film? Did he reference any past exorcism movies for inspiration or homage?

Mark A. Mangini: Mikael did not come to the film with any preconceived notions about the sound of the film. He was open to hearing new ideas and felt that our work would be an exploration; constantly evolving as we discovered together what worked.. We knew we didn’t want to sound like any other films so it was understood that a great deal of experimentation was in our future.

All of us wanted to avoid exorcism movie cliches like overzealous pitch shifting or devilish sounding voice replacements. It was clear from the beginning that the performances, especially Anthony Hopkins’, were quite remarkable and needed little if any work, as they played quite convincingly on their own.

The charge from Mikael was to heighten tension and create dread with sound, where-ever possible, while never allowing it to challenge Father Michael’s skepticism. i.e. if the audience sees or hears something that Fr. Michael didn’t or didn’t acknowledge, what could he be so skeptical about? If he saw a “spinning head” or heard demonic sounds, what was keeping him from believing? As such, up until the final exorcism, everything done in VFX and sound is fairly understated allowing the audience to maintain the same doubt as our protagonist. Everything you see and hear could have a real world explanation. This made our job particularly difficult as we were always having to play “devilish” or “eerie” sounds with just enough believability so as not to beg the audiences credibility and investment in Fr. Michael’s quest for the truth.

Dave Whitehead: As I was working from New Zealand I was only able to talk with Mikael once. Mark is such a great communicator, the notes from Mikael were clear and concise. The most important challenge was the arc in which the demonic experience was delivered. It had to be slowly drip fed and not shoved in your face from word go.

DS: What sound design motif’s were established to hint to the presence of demons/the devil?  Did they evolve as the story progressed?

MM: Dave really took the lead here as he created 90% of all the creepy atmospheres and demonic weirdness for the film. An amazing body of work. He made several signature sounds and atmospheres that repeat throughout the film. I had three favorites that he made (though it’s like picking which of your children you like the most): a sound we called THE POPE LAUGHS that opens the movie. A bizarre, quacking echo repeat that we re-used in several other spots to indicate simply that these events (possessions) sprung from the same source, a lovely repeating bell chime that served as the recurring (and re-appearing) bracelet signature, and an amazing and haunting musical presence made out of bicycle spokes used in the girls death in the rain.

DA: The possessions had to build gradually as Father Michael was a non-believer and we had to play into his skepticism. The actual vocal design had to be more plausible yet still disturbing in Rosaria’s first possession scenes and they became more demonic or hard to explain as the story progressed. I liked the notion that being possessed was like falling. I used lots of pitch bending to help give the feeling of falling and developed simple motifs such as the ones Mark mentioned above.

Visual motifs were obvious catalysts for palette building. The nails, the wooden beams in the rectory, the bracelet, crucifixes, the mortuary, the confirmation card and Baal (the demon) himself. The direction regarding Baal was very open, I researched the history of this demon and found a wealth of material online. I didn’t delve there too long as its pretty dark stuff, but it confirmed the direction Mikael had gone down with the presence of cats, frogs, the mule and the overall feeling of dread.

My overall approach on treating darker subject matter like this is to shift normality and the comfort level of the audience. As the demon was presenting itself, the wood in the room could moan, the air can be sucked out of a space and leave only darkness and weight, the demon could talk and the dialogue is around you. Sounds were related to what we saw and enhanced the dread Mikael wanted us to feel, but rules did not have to apply. Ambience can be cut hard in and out in the middle of a scene, dialogue can be in  the surrounds. Paranormal spiritual experiences like this, cannot be explained by us, therefore there should be less sonic rules.

The other side of this are the religious motifs. I used choral source material and bells to create ambiences for the rectory and a palette of stingers. Where there were demons, there were also angels. My take on the film was a struggle between good and evil and there had to be a shifting balance between these two.
My favorite scene to design was the bicycle sequence. I recorded bicycle spokes and made stingers, hits, tinkling rain, washes and rhythmic structures. The spokes (from a design perspective) were what drove the scene.

When I was 7 years old I was hit by a taxi and I remembered only a few sounds. The sound of the woman passenger screaming and crying above me, the ambulance doors closing, the siren and the sound of the scissors cutting my clothes open. I liked the way such a traumatic experience gave you only a small handful of sound images to take away, nothing else was important.

I also liked the notion that in this woman’s dying moments, the sound of the bicycle spokes, the rain and Father Michaels words became her requiem.

DS: Rome is such a beautiful city to photograph, how did you establish it sonically?

MM: Well, sadly, we didn’t get to go to Rome to record. That would have been nice. Our producer, Beau Flynn, asked us to bring a fresh approach to the sound of the city. I don’t think we spent enough time in Rome to really establish it sonically. Rome, like many in Europe, is a city of a 1000 churches, and, spending as much time as we do at the Vatican and with priests, it was important to get the bells right.

I did a film in Prague many years ago and I spent three months recording in and around the city where I got the best and freshest church bells I could find and used many of them for the film. The trick was understanding and using the right bells for the right occasion and time of day. They bring so much atmosphere to the Vatican locations and the city. Otherwise, Rome sounds modern, replete with scooters and traffic and the Euro Hi-lo siren (appropriately pitched at an augmented fourth – otherwise known as the Devils’ Interval).

DS: Was any production dialog used for possessed human vocals? If you did shoot group, what were you looking for in the voice talent for the demented characters?

MM: A great deal of Production dialog was used for the possessed vocals. Especially with Anthony Hopkins. Marta’s (the young girl, Rosaria) voice was augmented and replaced with the most ADR, maybe 50% of it but, even then, we used her to ADR herself for effect and to improve her English. Marta is an Italian actress who speaks English quite well but with a discernible accent. The first big exorcism shock is meant to come when she speaks “as if” she’s been possessed by the dead American teenager from the opening scene. We did ADR with Marta and a dialect coach to improve these performances from the production. Her voice was then replaced, selectively, as she got deeper into the exorcisms and to make her sound more devilish and possessed.

Hugh Waddell (who co-designed the demon voices with Dave) and I did several casting sessions in Los Angeles to come up with what would become our “demon” voice for Rosaria (and for Fr. Lucas). We knew we had to replace her voice with not only perfect English but a more demonic sound for key moments in the exorcisms. I can’t say that Hugh and I were looking for anything in particular rather, it was more what we were trying to avoid that drove our selection process: gravelly and cartoon-y evil that you had heard hundreds of  times in bad horror films.

During these casting sessions we would have each actor sync record two sequences for both Rosaria and Fr. Lucas. Men and women auditioned for both characters. All would get the same direction from Hugh and I; simple instruction on the kinds of voices to avoid and what we were trying to achieve dramatically. Hugh would then fine cut these recordings and send them, unprocessed, to director Mikael for review who acknowledged a decision we had made during the auditions that Susan Silo was marvelous. She’s an actress I had cast 30 years ago as a Gremlin in the original Gremlins film for Joe Dante. She has this strange combination of vocal qualities and acting ability. Kind of like a possessed Smurf.

All of the subsequent ADR recordings with our newly cast voices were fairly experimental as we searched for sounds and performances that fit the image. Hugh would assemble our select takes into sync tracks and ship them to Dave for further work. Dave would then process, embellish, sweeten and mix these recordings into separate 5.0 stems for the dubbing stage. We separated all the “processed” voices into individual stems as there were great non-language sounds that Dave came up with that wanted to live in the International mix. In the end we had five, 5.0 “demon voice” stems on the Dialog mix console so that we had bussing and mix control over each component.
The big epiphany for Father Lucas was the decision to have Anthony do his own Demon voices. Dave had done some initial voice processing and embellishment for the first temp dub and it was a great success. Everyone knew we wanted to take it a little farther but no one dared admit to Anthony that another actor might replace his voice. It was at this point that I realized we had the most gifted voice over talent in our very midst. We needn’t look and farther. Why not try and utilize Anthony’s consummate acting skills and put them to solving the biggest sonic challenge in the film? Funny how, often, such an obvious solution can avoid discovery for so long!

Anthony came in for one ADR session and did a remarkable job of replacing his own voice with variations of what we had in production as well as creating ghostly whispers and echoes that were meant to swirl around and mimic his sync performance on camera.

Dave did an amazing job of taking those organic echoes and transforming them with bizarre reverbs, delays, pitch shift, and panning. Again, Dave would “process”, for lack of a better term, all of Anthony’s production track and ADR and re-deliver 5.0 mix-downs of his work for the dubbing stage.

What’s fun for me is that, to an uncritical ear, Dave’s work and Anthony’s performances appear to be simple electronic echoes of the source but they are all new performances with subtle voice intonation changes as well as fanciful line readings.

DA: Rosaria’s sequences were an exercise in micro editing. Along side the ADR, I developed a non human palette, that were close to the characteristics of human vocalizations. Her screams were mainly baboons which when pitched down sounded like a demented human, her breaths consisted of asthma attack material from Marks own asthma attack, dogs, horses and mules. Her moans were lion cubs, domestic cats, dogs and a baby crying. The shift between production dialog and all of the ADR provided was constant. Susan Silo’s ADR was definitely great material to work with, she had such and amazing voice and it was the perfect counterpoint to Marta’s amazing ADR and production. Hugh would cut the initial version of each actors ADR and send them to me then I would cut my version incorporating all material, human and non human and send them back to Hugh to get feedback. Rosaria was more about editing and less about effects chains and augmentation.

Initially for Anthony’s scenes we experimented with pitching and adding non human sweeteners. We toyed with the idea of voice replacement,  but the fact of the matter is only Anthony could enhance Anthony.

Hugh recorded and cut versions of Anthony whispering and growling lines. We wanted to have Baal be in and around Father Lucas, he did not always have to be attached. It was to make him seem like more than a mere man, more frightening and unhinged. We would have whispered lines playing in the surrounds at the same time Father Lucas was talking on screen. We would add verb to individual lines to accentuate certain points of the scene. For all of these treated dialogue sequences I also delivered the dry version so they could decide how much of the treatment to use.

Again with Anthony it was a constant shift between production and ADR. We did pitch down his material very slightly. The more difficult process was the panning, verb and delayed treatment of his voice. I recorded multiple versions of the same line through various chains and edited the treated 5.0 stems. I then premixed 5.0 splits of dry dialogue, verbs, non English effects/breaths, panned delays etc…  and sent these straight to the dubbing stage.

DS: Were any of the recorded exorcisms played during the lecture hall scenes from real incidents? If you did design them, how did you approach/research that?

MM: No. Dave and I listened to a number of recorded examples and they just were not that interesting. What I found was, that if you divorced yourself from the knowledge that they were real exorcisms, they weren’t all that unusual or frightening. In fact, they often sounded cornier than a bad Hollywood horror film; crass, raspy, gravelly voiced amateurs overacting and trying to be scary. Yet they were compelling because you knew they were real.

In other words, it’s all about the context. If a scene is working, is truly scary, just about any sound you use could work. We struggled a great deal with the lecture hall scenes and tried exhaustively to create what became illusory goal: making what was meant to be ‘real’ exorcisms on screen sound frightening. I think this was our sonic “Waterloo”. By that I mean, there is always one sound, one elusive sound, on every film where an inordinate amount of time and resources are spent in trying to achieve a goal that will never be achieved and failure is inescapable, for whatever reasons; the filmmakers don’t know what they want or can’t decide, the action on screen doesn’t carry it’s weight dramatically, etc. We fiddled with these ‘real’ exorcisms a great deal and never satisfied ourselves completely. As is typical in many post-sound endeavors, we accepted what we had as the best we could do and called it a day.

DA: I cut a version of one of these exorcisms and the degradation of the recordings seemed to be the key. Worldizing through a handy cam seemed to be the type approach that would work. To be honest, I haven’t even seen the finished film yet as its not yet released in New Zealand, so I’ll see what they ended up using in a few days.

DS: There’s a cool sounding phone call in the film using some kind of static flutter, how did you guys approach it?

MM: Interesting you should ask about this one sound beat as the director and editor both loved this sound yet it was a simple static sound from my library that I threw in for the first temp mix just hoping to give the call a sense that it was coming from a greater distance than America. (Not sure how far away Hell is). It never changed from that day forward. It’s a crappy mono radio static that Mike Chock cut so that all the pops and clicks were highlighted in the right places. The father’s voice (Rutger Hauer) was also a very raw recording done on-set that we had always meant to redo in ADR but it’s sonic shortcomings added to the desired effect. At the end of the call, we put Rutgers voice in a little reverb as if to say it was coming from a different place than the hospital.

The following article contains excerpts from the “Future of Sound Design” lectures at GDC, VFS and Dutch Film Festival originally presented in 2006. Please note that certain expressions are personal opinions, and cannot be read as “fact”.

In our endless passion to make games have a similar, or exceeding sound-scape experience in comparison to other media, we constantly try to find new ways and techniques to obtain this. Some people ask “why are we comparing ourselves to film sound design, we’re very different”. Other say “Film sound experiences are the ultimate goal”. I say both are right. But to really figure out what the future may hold, we have to first learn from the past to enable measurement of missing objectives and goals.

To answer, we have to being by asking ourselves some questions:

• What’s been done in the past?
• What’s broken/missing?
• How does this compare to Visuals?
• What about Emotions ?
• Is there a future for Audio?
• What about everything else ?

The Past – Evolution in Numbers

Technical hinderances, ever since the X360 and PS3, have been much less of a hurdle for a sound designer to create engaging soundscapes. Lets look at the history, based on the most popular game machine/console during each period.

As you can see in Fig 1 & 2, the amount of sound-data currently storable on the console is so much, that in comparison the old consoles barely show up on the graphs. Memory isn’t really a technological barrier anymore.

What’s broken/missing?

The obvious one to look at would be the Sound Designer/Artist. Are the requirements of creative vs technical understanding still too high? Are they a hurdle we still have to overcome? In fig 3 I’m showing my estimates of the job requirements of a sound artist/designer working in the video games industry, when looking at the biggest selling platforms. Funny enough, the industry seems to repeat itself. Consumers are now using phones and other small devices to play games. These devices seem to be equally powerful to game consoles 4-5 years ago, which brings back the same technical hurdles, all well known and documented.

The 2nd obvious one to look at is the “no boundary” story telling experience. During the years I’ve noticed that for some sound folks who grew up in the technically restricted era, its very hard to cross-over to new platforms. As an example this is one the reasons I originally stopped doing music in the mid 90’s. I was pretty good at making small processors like the C64 and SNES do things they weren’t meant to do, and therefor got an edge on making enjoyable music. With the introduction of Redbook (CD) audio, the playing field was open to everybody, and I was no longer able to take advantage of any hurdles others hadn’t overcome yet. It’s critical that these folks find ways to move to the story telling, un-inhibited way of thinking. They have to gain this experience, or they’ll be left behind.

So what does this lead us to? To me, the biggest general missing link in making games have equal or better sound experiences than film is an investment in emotionally believable audio followed by treating the player smart, in both gameplay and audio. Lets focus on the first one for now.

Too often when playing games (including our own), I still feel “disconnected” from the experience due to sound. Some games make a great attempt at it, but in the end, there’s always something happening which causes the de-focus from the experience. During the remainder of this article we’ll touch upon what causes these disconnects. To understand this better, lets also look at how audio and visuals work together

Visual Media

Look at the below pictures. You’ll probably have a different instant feeling or emotion about each. You can tell that approaching visual realism isn’t always a good thing (the “uncanny valley” effect). It distracts from the believability, and the connected emotion you’re supposed to have. We’ll see later that Dialog has a similar issue.

Another interesting effect is that feelings generated from visuals can be interpreted different from person to person and feelings created by visuals are culturally relevant at times. Images generate a feeling, a response that we learned during our life.

VISUALS and AUDIO –  THE MARRIED COUPLE

Now, lets do a quick exercise to see the relation-ship between visuals and audio (note: due to copyright, we can’t put this music here).

• Pick one of the above pictures. Look at it carefully. You’ll notice the feeling you had initially withers away quickly.
• After a few seconds, cue up your favorite rock piece near the chorus… Did the meaning of the picture change?
• Now cue a film-score (I like to use “seven pounds” as an example). How dramatically did the meaning of the picture change, but importantly, did the feeling it generated in combination with the picture sustain?

What can you conclude out of this? How does Audio fit into this picture?

• If picture gives you the instant feeling/reaction, audio maintains this feeling over time.
• Audio cues can change the expected emotion a picture generates
• Audio can enhance picture in more than a support role and change the emotional outcome
• Audio emotions take (usually) time to establish

A quick word about Feelings and Emotions

We have to understand when to say “Feeling” and when to say “Emotion”, as both are pretty different. It’ll also help us understand how audio plays a big part in this.

• Feelings are a learned response of the culture and your surroundings in which you grew up.
• Feelings are a subset of all your mind-body states (i.e. disappointment, hunger, hope etc.)
• A Feeling is the response part of the Emotion. (“I feel disappointed”… a resulting emotional reaction could be “I’m Angry”)
• Emotions are cross cultural – the same meaning all over the world
• Emotion is a chemical state in our brains. Those same chemicals inhibit our capabilities and limiting what we call rational thought

So how can a game use this ? If visuals and audio work together to establish feelings and emotions, you can use this to a certain degree to influence game-play:

• More emotion = less judgement
• If you want to remember something, get into the emotional mood you were in when you first experienced it.
• You’re likely to come back to a “liked” emotion. Some emotional states can be addictive.
• Person’s mood tends to follow that of the situation presented in front of them.

OK, so lets get back to Audio…..

The main ingredients of Audio in Game:

Everyone reading this probably knows the 4 main “technical” ingredients of audio in a game: Music, Dialog, Sound Effects and Mix. During the many lectures on this topic I always asked the question to the audience: “What is the most important element to a believable and emotionally engaging soundscape”? “What is the top contributor”, and “What is the top damager”. The answers usually ranged across the board, each picking their “favorite” one. Composers would pick music, sound editors would pick sound effects. Repetitive footsteps was often mentioned. Dialog was the most picked damaging ingredient…. Seldom the answer was “all”.
It’s probably obvious to you that every single ingredient is of equal importance to create an emotionally believable soundscape. You can’t approach a single ingredient in a lackluster way. Believability is key.

Believable Dialog

Dialog is still the #1 offender in believability area. I’ll get probably flamed for saying this, but I’ve yet to hear a single game which makes me believe I’m listening to the characters on-screen for an extended period of time. None have captured it as truly “believable” yet. Space, placement, acting, story, odd breaks, visual discrepancies etc. all contribute to dialog flaws. We’re running up against the same “uncanny valley” effect as visuals. We’re approaching reality and the human ear will now pick up every flaw, and is no longer forgiving. Yet if we’re very far from reality, we’ll believe it. Have you noticed you can watch a saturday morning cartoon and believe the characters?

What also isn’t helping us is that some characters in-game on-screen still look robotic (or don’t even exist), making us having to work even harder to make that certain voice believable in contrast to the visuals.

One issue I’m hearing quite a bit is the recording method used. Lots of dialog for games is still recorded in the traditional “music” way of placing a U87 (or similar) close-up to the actor. Often, introducing movement, space, air, body shocks is totally negated. 

Game Example: "NBA Street"

In 2005 we did a quick test for a game called “NBA street” for which no longer accepted this type of recording. We rigged up a lot of players with wireless lavs, and had them play for several hours while feeding them scenarios and lines. The resulting effect was a much higher degree of believability. Following is an example with the “U87” version back to back with the “Lavalier” version. On purpose, similar hokey lines were picked to illustrate the concept (this was not a concept of good acting). Which one is most believable to you?

Example: nba.mp3

#1 is the old U87 version, #2 is the Lavalier version, separated by a beep

Believability Gap

Believability “gaps” are the #2 offender. Everytime a player is jerked away from the game’s believability, it makes him realize it’s a game, and intensity lessens. When this happens the intensity buildup has to be restarted to a certain degree. The game will never be able to reach the full potential of engaging, emotionally believable audio. Some examples of these gaps are:

• Awkward forced Loading screens (i.e. “silence screen”)
• Repetition on dialog, sound effects, or anything else noticeable
• Un-natural imbalance (Vol / EQ / Space etc.)
• Non believable Dialog
• Anything which goes against “learned” sounds, if not on purpose (more on this later)

Audio Mix – “The Glue”

This is a large topic, one that’s too big for this article to cover, to which we’ll come back later. It’s the #3 offender to create sustaining believable soundscapes. Too many games still ship with the “wall of sound” approach. A player can only take this so long. There are many solutions to this problem. One of the causes still seems to be “producer X listening on his TV in a noisey floor area wanting to hear every detail”, which is a hard one to overcome.
With any of the solutions, the mix shouldn’t make a user notice what’s happening, yet get more engaged. If your producer is asking for the “wall” approach, he’s in reality not asking for this. He just couldn’t hear something he wanted to hear. It’s now up to you to refocus the mix constantly to allow him to hear what he wants to hear, yet doesn’t take away from the rest.
Much more to come on this topic as there are many tricks to accomplish this.

Excuses

One of the other blockers in achieving an emotionally engaging soundscape is excuses. Yeah, the ones that every Sound Designer or sound-sup makes when he can’t achieve the needed result. We’re all guilty of it (including me). Yet, it’s one of the hurdles we have to overcome if we ever want our industry to excel above the film media’s level of engagement. If you know you can’t mix for an emotionally engaging soundscape, don’t do it. Don’t pretend you can learn it overnight. If you don’t have enough money to achieve the result, scope down, sell your ideas to execs or whatever you have to do. Don’t use’ em as an excuse why you couldn’t succeed. There are many reasons, many which are direct blockers, and many which can be overcome with creative solutions. I often hear “well, we don’t have the tools that some others have”…. Tools are a means to get to a result, but not the only way. Come up with creative ways to get the tools you need.

Wrap up

So what is the future of sound design ? It’s not some sort of new tool. It’s not a new console, it’s not a new “cool way” of creating sound in real-time. It is purely us overcoming our hurdles to find new ways creating emotionally engaging and believable soundscapes. Breaking out of traditional ways, learning what the human reacts to. How feelings and emotions tie in with sound is a must-know Content is no longer king, technology is no longer the queen. The combo of all of it, and the stimulating, game supportive result is what the player will be experiencing and wanting.

–end of part 1– We’ll look at “ear deficiencies” and “experienced sound”, including methods on how to use those to build an emotional soundscape, in the next few days.

Written by Charles Deenen for Designing Sound

Thanks go to Re-recording mixer Steve Maslow for taking time out of his busy dub schedule to answer some questions!

Designing Sound: Comedy, being so reliant on timing requires a great dialog mix for jokes to be heard. What challenges exposed themselves during the dub and do you as mixers get to go to any of the test screenings to see what jokes get buried by laughter?

Steve Maslow: The biggest challenge for the dialog mix for this film was the quality of the production sound in many of the outside scenes and that of the sound quality of the dialog. I had three different mics at times for a character: boom, lav, and a plant (one that was hidden in a prop). The boom on some shots had a lot of noise associated with it (wind and traffic), the lav was very chest heavy and didn’t have the air around the dialog I usually like, and the plant at times was off mic for the character. Any loops associated with these conditions were a very difficult match.

Now when the picture editor edits a scene he’s cutting, he may use a combination [or comp] of the mics and then that sound quality is what they become used to during picture editorial. I may also get this combo for the temp dub, and now I’m stuck with what they have been listening to for weeks. When I start to pre-mix the dialog, I now have the three different mic channels on separate tracks and choose one of the these tracks that I think sounds the best, which will sound different to the director because of what he’s been listening for months in the cutting room. This forces me to go back to the combo of mics used in the temp, which may have a noisier track than I’d like.

I like to go to the test screenings to see what has worked, and if the jokes get buried from laughter, so be it. I won’t go back on the stage and raise the lines because this has the effect of stopping the laughter because the dialog is so loud that the audience stops to listen to the next line.

DS: I know external factors motivate usage on set, but what are your thoughts on boom vs. lav?

SM: I will always try and use the boom mics over the lav. The boom mic has a much more natural sound quality. I have rarely liked the way the lav mic sounds because of the close proximity effect it has, giving it a chest heavy sound and making it difficult to get the right perspective in the scene.

DS: You’ve worked on a lot of great films Steve. I actually just watched “The Thing” again last night(admittedly one of my favs). What film(s) has been one of your favorites to mix?

SM: Films that stand out would be “The Raiders of the Lost Ark”, “The Empire Strikes Back”, “The Last Waltz”, “Blue Crush”, “U571″, and “Speed”.

DS: How the hell did you guys mix without automation? What was the longest time you had to rehearse a reel before printing back in the day?

SM: Typically we would rehearse all morning until lunch. This was when you had to have three guys on the desk (dialog, music and effects). We would take copious notes and or make cue sheets to know where dialog started and ended in a scene so that we could make our moves to push and pull music or effects. This type of mixing was truly a performance. Usually after rehearsing, we would find that something would be missing, so we would have to pick up additional units, making for a longer rehearsal. Sometimes it would take a couple of days of rehearsing to get it right. After lunch we would load up the recorder, which depending on the year, was usually a four track. This meant that all three of us would record to one recorder, because that’s all we had. One big stereo mix (left, center, right, surround). If one of us made a mistake during the record pass, we would stop and roll back to find a place where we could all three match the output of our console with the record level of the recorder. Once we were satisfied that the three of us were at the right levels, which was done by an A/B switch between the console and the recorder, we would punch in and hope the level was spot on. If it wasn’t, we would have to roll even further back and try again.

DS: Ben Burtt implied recently that having a traditional adventure film score in “Star Wars” really brought legitimacy to the film. I read that the score in “Evan Almighty” is very grand in scope too; How does it help the story and emotion? What are your favorite cues?

SM: I thought it fit the film very well for its emotional impact. Also a lot of tunes were used to compliment the various themes throughout the film. “Are You Ready For A Miracle”, “Revolution”, “Sharp Dressed Man”, “Have You Seen The Rain?”, and “Everybody Dance Now” were some of the tunes used to further compliment the comedy throughout the film. Rascal Flatts recording of “Revolution” stands out for me as one of my favorites.