Symbolic Sound has published on their blog called “the eight nerve”, an interview with sound designer Sylvain Lasseur talking about his use of Kyma system and several aspects about his work.

Sound designer Sylvain Lasseur is not just bi-coastal; he’s bi-contintental, working part time in Paris and part time in Los Angeles!  We recently had a chance to ask him a few questions about how he uses Kyma for 5.1 sound design and to explore some of the differences between post production work in Paris and Los Angeles.  By the end of the interview, the discussion turns to food, wine, and the Marx Brothers.  Read on!

In this exclusive SoundWorks Collection video profile, producer Michael Coleman sits down with the oscar nominated sound team of HUGO including co-supervising sound editor Philip Stockton, co-supervising sound editor Eugene Gearty, and re-recording mixer Tom Fleischman.

Martin Scorsese’s adaptation of Brian Selznick’s award-winning novel The Invention of Hugo Cabret stars Asa Butterfield, as an orphan boy who lives in a Parisian train station. Sent to live with his drunken uncle after his father’s death in a fire, Hugo learned how to wind the massive clocks that run throughout the station. When the uncle disappears one day, Hugo decides to maintain the clocks on his own, hoping nobody will catch on to him squatting in the station.

His natural aptitude for engineering leads him to steal gears, tools, and other items from a toy-shop owner who maintains a storefront in the station. Hugo needs these purloined pieces in order to rebuild a mechanical man that was left in the father’s care at the museum — the restoration was a project father and son did together.

SoundWorks Collection

Andy Farnell – a familiar name in computer audio – is a computer scientist, sound designer, author and a pioneer in the field of procedural audio. He is a visiting professor at several European Universities and a consultant to game and audio technology companies. His book, ‘Designing Sound‘, is a bible for procedural sound and should be on your bookshelf, if it isn’t already!

He was very kind to find time in his busy schedule when I visited London, and we talked about what procedural audio is, where it stands now and what it can be in the future. This article is a transcription of our conversation, which he was again very kind to edit along with me. It was no easy task because there was so much good content!

Thank you Andy!

DS: Where does Procedural Audio stand now? Would you say it is comparable to where CGI was in the 70s/80s, when computers weren’t powerful enough?

Andy: That is a central mythology – that the computers aren’t powerful enough to do it. This is often brought out as a straw man argument against Procedural Audio by skeptics. One of the things I did with my 2005 demo was to make all of the sounds (they weren’t very high in quality) that you would need for a first person shooter game – fire, water, wind, rain, some animals, some footsteps, some guns, some vehicles. This was 2005 and I had them all running on a 533 MHz processor generating a realistic-ish sort of soundscape to prove that if you had 1GHz processor and if you used half of it for the graphics then it would be quite possible to synthesise all the sounds using the remainder. Six years after doing that people would still come to me with this straw man argument, they would say, “You know Andy, we love this Procedural Audio stuff but there’s just not enough CPU available”. But we now have two to the five times more CPU than when I did my 2005 proof-of-concept demo. So, what’s behind that? Why are they saying that? It’s not true. What happens is the internal politics of resources. The requirements always expand to fit the resources available. The game worlds get bigger and bigger and the graphics get more and more demanding. The audio team will always have the least amount of CPU allocated to them as an afterthought, because in the current structural model of production sound is “post production”, and no body wants to commit to giving audio that much CPU bandwidth. I feel that is the real reason behind the argument. You often get these straw man arguments that enter in to a culture and just get recycled. People know that there is an argument and it comes to their tongue very quickly and they say “Yes we could do it but there is not enough CPU”. With the left over CPU on a modern games console I could provide you great procedural sound.  On an eight core architecture, we would need one or two CPU cores to give procedural sound. Even more interestingly is what happens when we run models in GPU, and many Procedural Audio models are inherently parallelisable. So, yes, Procedural Audio is somewhere in that era before the Tron movie, or before the Pixar CGI revolution, its possible, but not yet seen as viable, perhaps the shift is too painful for big companies to make.

DS: Have you tried doing a similar demo using today’s technology?

Andy: No. I just don’t have time at the moment. Life is moving so quickly and I’m involved in so many other interesting projects. Some of them to do with computer science, some of them to do with philosophy. Interesting times, but in the future I want to have a research department and have a bunch of guys, really smart guys who are just on this and want to do it and I can help direct their research. Because, when you look at what Procedural Audio breaks down in to, its actually very deep specialisations – just like CGI. There is room for intense talents within the area. Let us make use of CGI analogy again – if you are a real good texture artist you are great at looking at skin and saying, “That is the skin of a 40 year old, that is a skin of a particular kind of salamander, look at the way the bone structure moves underneath it, look at the way the light hits it”. You get specialisations within Procedural Audio which would be people who are very good at fluids, they are great at doing water falls and drops of water and boiling mud and lava. They understand that sound. They are able to model it and come up with great sounding objects and great processes that do it.

Before I took up this umbrella term, this banner of Procedural Audio, and tried to make a focussed idea out of it, I had mentors – I mean people I looked up to, leaders with ideas that nobody else was doing in industry or academia people like Perry Cook. He is like the grandfather of Procedural Audio. He was doing it in the early 90s, when the argument that there wasn’t enough CPU really was a good argument [laughs] and then after him came Dinesh Pai and Kees van den Doel and they worked on impacts and fluids. They did that as very narrow academic work. I don’t know if they saw (the generality of the possibilities) that the water could be taken and integrated with a glass so we get an object that could be filled up or emptied, or become raindrops in a particle based weather system that interacts with different objects the rain falls on… The object-object interaction based idea of “sounding objects” really came out the North Italian schools, otherwise we just have event driven sample playback . But, they did extremely good work and a lot of my stuff is just interpreting their work and generalising, extending it, and making a coherent philosophy of sound as process rather than data. We must always be mindful of that background to it. It didn’t just pop out of the air. It is a project that has been in the background (since Mathews in the 50′s)  growing slowly. If anything, I have  been a very vocal advocate of these ideas applied to the general case of everyday sonic simulation, and been instrumental in defining what procedural audio is.

DS: So the obstacles aren’t purely technological?

Andy: I don’t know what the “real obstacles” are now. I’ve said in another interview before, that around 2006-2007 it dawned on me that there weren’t any fundamental obstacles to radical technical progress. We could do this. The obstacles were structural and political. How do you introduce a new technology? How do you get people to take risks on that? One of the weaknesses of it, it’s a weakness but a very deep philosophical strength (and this is quite subtle), is that sound as data fits in to a capital model. Intellectual property allows you to own a sound asset. So if you record or create a sound, it is an asset that you own. You can trade assets. But procedural model breaks with an ownership model because what you are doing is you are substituting general sounding objects for something we can make million sounds in the future. There is no redundancy built in.

DS: Although, if a game developer spent time and resources building a procedural audio engine, as they would spend time building an audio engine, wouldn’t it be an asset that could be owned?

Andy: The observation there is that the code is the asset. But the code is useless without a group of people who understand how to make it sing. We move the value from residing in the thing itself to how it is used. I see procedural audio as an art to be practiced, not just an application layer to be built.

DS: So even with Procedural Audio you will need a sound designer to understand what it can sound like and how it can impact a player/end user?

Andy: I think this was on the game audio forum or something years back. Someone raised the accusation at me that, “Guys like you put people like us out of business and you are making technology that is going to replace our art”. I took that on board as a very valid point. Being a sound designer myself, the last thing I want to do is put other sound designers out of work. I see it as a liberating step – you have your sound samples and you have this. I always see it as a complimentary technology and not a replacing technology, that is point number one. Point number two that is more important is that every new technology that comes along generates a new requirement for skillets which the talented people in that business become really good at. So every Procedural Audio team would need a good sound designer.  I wouldn’t leave it to the programmers, I want somebody who has a great set of ears and I would actually put them in a higher position and get them to direct the programmers and say, “No its more like this, listen to these examples. I want to get this emotion across”, and they can direct it aesthetically. It’s not really putting sound designers out of work and it is not a totalitarian project. This is why I worry that the bean counters, the alienating/asset-oriented capitalists, are seduced by this kind of technology because they just think, “Well we plug that in and we get rid of the sound department”. That’s not what I want to see happen. One of the great advantages is that it gives 90% of your assets for free. You just put your objects in the world and you get default sounds. What that means is that you don’t have to worry about an asset-event matrix any more. You don’t have to worry that somebody has forgotten to put a sound on something because everything will be covered by default and now the sound designer is liberated not to be thinking up every single little rock sound but to go and focus on the emotionally significant sounds – the hero’s sword, the getaway car, the gun sound. They can put all their time and energy in to getting those right and not have to worry about the other stuff. That’s another argument for Procedural Audio. It raises the bar from where you start from.

DS: So, Procedural Audio is just another tool in the arsenal of a sound designer? A combination of different techniques?

Andy: Procedural Audio is a philosophy about sound being a process and not data. In its broadest sense, if I were to say it is the philosophy of sound design in a dynamic space, and the method is as irrelevant to Procedural Audio as whether you use oils or water colours is to painting. If you use papier-mâché and glue or whatever is to sculpture – the end is in the artist and not the method. So you can mix and match the methods, they exist separately. In the industry now, all the successful Procedural Audio is mixed methods of samples – granular methods with the exception of as far as I can see of Nick Fournel’s work which is basically a similar kind of resynthesis but it is phaselet or PVOC type re-synthesis. First lets see why that happens. There is obviously a clear bridge there between existing technologies and the direction that Procedural Audio can go in. It gives you an immediate start. You can use your existing sample libraries and your guys out in the field. I incorporate this in to my understanding of Procedural Audio as: your sound guys now do analytical recording, not for the purpose of using those as final products but for exposing and analysing the sound underneath so you can build your procedural model. Granular methods are a very direct way of doing that. You just take the input sound and bust it up in to its component waveforms and then you re-synthesise them as grain clouds in different ways. Or in the phase vocoder or linear predictor sense you split them up into transient-exciter components and resonant parts. In that sense it is a direct re-synthesis. Now these approaches that have a method and analyses part, in effect, have a one to one mapping. So you are doing re-synthesis but you can fiddle around with the parameters in the middle. I call this a shallow or phenomenal approach. What it means is that is that the way in which you can change the sounds is limited mostly by your understanding of the parametric interface of the re-synthesis method. The re-synthesis method doesn’t capture any of the physics, it doesn’t capture any of the process of the sounds. Whereas building a procedural model, where you got a model and a method which compliments the model, you are  interested in the behaviour of the sound that is built in to your model. That is not phenomenal. That is not surface, I call that essential or deep. Guys who are doing that kind of thing are like Zheng and James, they are using really accurate models. They are basically doing what computer scientists mean when they say computer modelling. They are using fluid dynamic models to model fluids. They need a rack of computers that take days to process a few seconds of sounds, its not practical Procedural Audio, as would be used in games, because it does not meet the real time criteria for a start. At least not yet.

But having said that I also think there is another side of (non  real-time) Procedural Audio which is not in computer games. It’s in animation, where the idea is that once we have introduced the sounding objects as models, into the scenes, sound ceases to be post production. You re-arrange the objects and their behaviour in your scene to do your visuals and the sounds come out for free. You can even change the location of the microphone virtually after the fact. This is the future of cinematic sound , CGA or computational audio. The best thing about this is that we can drop the real time constraint and  trade speed for quality.  Why not have thunder rendered with one with a million N-waves, which sounds more like the real thing but which takes a long time to compute because you have got a render farm and you are a million dollar Pixar type company? You need good model programs and lots of computing resources but you don’t care about the real time thing. I think Procedural Audio encompasses that as an art.  Skywalker once kindly offered me computing resources, I think Randy Thom set it up, but its hard to work remotely out of context, I would like to put a masters student or PhD on that one day, I grew up with that rather dangerous Radiophonic workshop ethos of Oram/Derbyshire  (two original women sound designers),  creative research and commercial production combined, it is an iconoclasm of experts, expectations and traditions. Much too risky for todays world.

DS: Much about sound design is about achieving hyper reality and not reality. You wouldn’t want a gun to sound like a real gun. So is Procedural Audio about creating realistic models and building on them to achieve hyper realism?

Andy: Amongst the many good sound designers that I have met, they all turn out to be really quite well rounded smart people. They aren’t purely phenomenal,  see the world not only through an artists gaze, but an informed, worldly apprehension. Sound is about going in to the world, what happens inside and outside things, its about deep knowledge about how things work and what does that mechanism mean to your emotions. I think the role of what a sound designer is becomes somebody who’s language is not about computers but understanding the mechanisms of sounds. And the  natural progression for sound designers, when they run out of all the shiny plugin technology in the world is Procedural Audio because it lets their art develop, connecting with sound and its causes and its propagation and reflection. Deep knowledge of sound is what a sound designer has, even when they can’t express or vocalise it. People who have become really talented at that often have hidden knowledge, ineffable knowledge, they don’t have a way to vocalise it or make it explicit but they understand things intuitively and you see them in the studio just do stuff and if you ask them how they did it, they would go “I don’t know, I knew thats what was needed” and it gives them, I don’t like the word, but it gives them a “rationale” to explicate their knowledge about sounds.

DS: How does Procedural Audio fit in to this? Most sound designers aren’t mathematicians or scientists, most of them have a good understanding of how objects react in the world but not necessarily the science behind it.

Andy: In the very long run and I’m talking about – I hope to see it in my life time – ten, twenty, thirty, forty years in the future I think all of this that we are talking about today will be well understood as part of the discipline of sound design. I think the philosophy of Procedural Audio will be a part just as much an artist now can talk about textures and different kinds of lighting. In a way sound is a real throwback, right now. Some people have said to me, “Sound technology is fifteen-twenty years behind graphics technology”. That’s crazy, all of the algorithms that made visuals possible came out of sound. One dimensional signals first and then two and three.. it comes from radio and radar. And, somehow culturally, sound got left behind. Because, we are visual beings and we put all our energy in to manipulating and creating ways to have a visual reality. Twenty-thirty years in the future all of what we are saying now will be a part of the language of sound design. But right now, theres a fork in the road between my understanding of it – the philosophy and the way it should go and the way it will go practically – and what is happening. So part of my philosophy in education is principles not products. It’s a reaction against the commodification of skills and people, techniques, arts being reduced to products. So if you look on the sound design list when somebody says, “How do I make such and such a sound”, and somebody else says, “Oh you need the zzaaaq plugin, that does that”, and they completely abdicate any desire for knowledge. They don’t even care (to pay for it) because they can get it off bit torrent anyway. Somebody else has packaged that capability and knowledge and given it to them. And by doing that they have robbed them of the knowledge, (Zarathustra says: be careful what I give you as a gift, because I may take something away from you) the knowledge is useless by itself but as for an artist, for your career, your development, for your ability to do things, that knowledge is important, its part of it. Both the commercial approaches to Procedural Audio at the moment, and this isn’t dissing these guys (in Audio Gaming and Sony… I am in touch with Amaury quite a bit and try to help them out recruiting and seeing the way ahead), what they are interested in producing is products (that is business). They want Procedural Audio models as drop in objects in the game. With this model, given to the current middleware developers, you’ll probably have opaque Procedural Audio objects, you cant see inside them, they have a few exposed parameters. Say you have a car and you can choose four or six cylinder engine, a bunch of different configurations for the exhaust, you can choose the body material, the tyre and that would be it. That is good! That is how you want your end user to see the object. At a certain level. So one approach would be to sell these as closed objects, with their functionality is hidden. But, for me as an “academic”, as a pioneer I am much more interested in enabling (my relationship is with the sound designer, with people), you should get in to this stuff. To me, the good stuff will be toolkits like the kind Dylan Menzies and others have proposed  (and using the pluggable physical components using FEM/discrete numerical difference schemes like Stefan Bilbao explicates) Now you have an engine model and its a part of the car and you can replace the engine – real world analogy here – you can tinker, you can take it apart and change the the way the camshaft and the pistons work. You can replace the method that is used for that engine with a subtractive method that you wrote in C++ and drop it in because there is a well defined interface. And the kind of well defined interfaces that work are like data flow interfaces – like Max/MSP and Pure Data type interfaces where you can just plum these objects together very quickly in the studio and test them in-world while playing. Ultimately, open ended data-flow user interfaces with efficient JIT /bytecode compilers are the future for creative Procedural Audio. Then you will have a very vibrant community, a vibrant ecosystem of programmer-sound designers and sound designer-programmers and people who work in teams and Procedural Audio will be a vibrant technology. But I think first it will go through what basically becomes a plugin (mystified) culture. Procedural Audio in my (wishful) philosophy is open and based around knowledge [laughs]. That will come with time after these products have driven a path and we will think about sound differently. The construction of sounds will be more technically informed and richer than saying, “Oh yeah, Hollywood Edge track 6 number 4, that’s the one you need”.

DS: Would you relate the questions raised about Procedural Audio to people talking about motion capture replacing actors?

Andy: I never made that connection, that’s really good. Mo-cap in relation to CGI is the same as what I am calling analytical recording for Procedural Audio. You go out in the field and try and look at the behavioural features and try and capture them and then use them as data.

DS: So would that be the important point to make then? To use the analysis as data and not exactly copy it, just as how mo-cap is used in CGI?

Andy: Yeah, I was talking to some guys who are in to this mo-cap and human face stuff recently and they were telling me some amazing things about how Tom Cruise voiced an animated character and they got all his face expressions – the eye brows and everything. The trouble was it was a dog character or something and when they played it back it looked too much like Tom Cruise [laughs] – and this was how you could extrapolate to hyper reality where you just scale everything so that the eye brows doubled the distance. So they got this super hyper real version of Tom Cruise in the character but somehow you still knew it was him. This is built in to procedural technologies and data analytical technologies. You can create models based on interpretations of real data and then extrapolate that off in to hyper reality and thats really powerful in film and games. You want that capability built in. They were doing something else really weird, like they were morphing characters, they were doing something like Sigourney Weaver and Tom Cruise and you make Sigourney Cruise. Like you do with hidden Markov models in composition where you can hybridise them and you can kind of have Beethoven and Shostakovich and get new ones, Shothoven or Beethovich (??) [laughs] or what ever your new composer is.

DS: The obvious use of Procedural Audio is gaming and animation, as you mentioned. Where else do you see it being applied? There has been some talk of it being used in electric cars to simulate engine sounds.

Andy: I can see it as a real interesting safety feature in a car. I would also worry about this becoming a real nuisance – imagine a car running along the street sounding like a clown’s car and the person is drunk and its 3 in the morning! I can see a lot of social tension about objects having arbitrary sounds. Sound can be very intrusive and it has very different implications across different cultures and age groups.

DS: I came across a Harmon owned company called HALOSonic that promises to deliver technology that seems to simulate engine sounds for ‘cars of the future’. While there is very little information on the actual technology being used, I wouldn’t be surprised if it used Procedural Audio.

Andy: Let us analyse that as a product and say why Procedural Audio is really powerful for something like this. On the face of it you would think of using samples. I am the CEO of the company, I want a cheap product and I want to get it out. RAM and disk space is so cheap. I’ll  get a bunch of sound designers, get them to make loops, load up my product with a thousand loops and plus I can now sell you extra ones. Why the hell would I be interested in Procedural Audio? There were two really powerful reasons – technical computer science reasons. Number one: If you haven’t got memory space like greeting cards, watches, mobile devices – one of the biggest errors of judgement in mobile/casual gaming and the whole mobile technology industry has been over estimating the available bandwidth. And what’s a good technology when you haven’t got any bandwidth? This is what is your biggest asset when you deliver procedural content. It occupies 4kb and plays for six hours. It has a million things that you can change all the time or download another one. So, as a technology procedural content is very powerful in situations where you’ve got limited bandwidth. With cars, another place where procedural technology is very powerful is where you want the sound to encode a large vector of changing parameters. Why is it useful to have a sound on a car? By listening to a car engine I can tell a lot about it – is it slowing down, speeding up, is it a large car or small car. I can localise it pretty well. So to replace a completely silent car engine what you want is a procedural sound object which behaves like the car (that is familiar to peoples expectations viz a viz reality – and hence safety) with engine, with tyre sounds, with exhaust simulation to delineate rear and front approach  In fact you could encode all kinds of other information about the car as a safety feature which people would quite quickly get used to. If it is a bus – it could be a bigger noise, if it is a bike it’s got a lighter sound. That would be difficult to do with a sample. So the procedural object would be more versatile and able to to encode more information. That would be argument number two. Argument number three might be that to develop a library of a thousand different car engines would be very expensive. But once Procedural Audio technologies mature I should be able to buy an engine model as a one piece of software and adapt it – I could commission it as a one of piece of software or buy it on a license, put in to my product and I have all the versatility of it. There comes a point I think where the code becomes cheaper than the recording, for a limited use case. Maybe. I’m not sure about the economics of that. I am interested to see how it turns out. If on the other hand there are a lot of people out there recording things and there is a very buoyant market in recordings….

DS: Do you see Procedural Audio code becoming a commodity that is built and sold?

Andy: I think so, I see that happening with the apps market. Production models have changed, and this is very much down to Apple. To their credit, I’m very anti-Apple at the moment (for ethical reasons), but to their credit one of the really good things that Apple has done is (by standardising experience and lowering expectations) accelerate the tool chain to the point where the production of an app is a half a day’s work sometimes. Cookie-cut template things that go out in the store and you download them. That is code. It’s the entire application. Whereas before, code was something that was produced over weeks or months, that still happens with bespoke apps but we’re seeing the trajectory towards commodity code like that. So yeah, if you are a great procedural sound designer why could you not be a guy who specialises in engines and you sold two to Volkswagen and one to Mercedes and you are working on a couple of other ones for some company? People come to you because they know you are the ‘engine guy’. Because its the standard API for interfacing with the procedural code you know at the end of the day you are going to get six floats or something – four rotational velocities for your wheels and you are going to get an engine speed and you are gonna get a bunch of stuff and you plug that in. I would like to be optimistic about the futures for these market places, I would like to think they they might work. That is very much my (humanist) philosophy as a person, I think that these things should create work which should create opportunities and they should create markets and they should create things that people can do and involve their talents and this is why I say you balance technology, art and business . And any one of those can become an overbearing dominant thing. The culture can override the technology and the business, the technology can become the “oh technology oh technology!” and stamp on the business but more often these days it’s about the business running out of control. If the business dictates the technology and the art, if any one of those three gets out to kill the other its bad. As for Apple, who want to commodify and control your creative experience for a profit motive – the absolute antithesis of their “nineteen eighty four position”. So what I would fear in the case that you are saying is that commercial attempts at pushing procedural audio initially turn out quite watered down,  neutered and over-packaged like supermarket food, in which case it will be procedural audio only in name, only in marketing speak. But then I’m a dreamer, 20 years ahead with this stuff in my head, disappointment with present reality is built in, its what keeps us pushing. Computational audio, more intelligent structuring of audio in games and cinema, will be a new creative frontier. But we need to get past a crisis of purpose with technology. The idea that easier is always better, that more is always better, dumbing down, disabling and concealing rather than opening up, enabling and enhancing. To paraphrase Laing who says “The emphasis is more and more on communication, but people have less and less to communicate”, we don’t really need more ways to do things, but better, more thoughtful and reflective ways to do things, to replace brute force and dizzying excess of choice from the previous epoch with more focussed and elegant ways. I hope some of the philosophies of procedural audio will shape sound design in a wider sense.

Set of 9 videos featuring sound designer Ben Burtt talking about sound and his work on “Super 8″.

‘Pugs Luv Beats‘ is a hilarious music composition game for iOS devices developed by Edinburgh based studio Lucky Frame. It’s about guiding pugs (in costumes) around a galaxy of worlds, whilst creating an endless variety of music. It sounds fantastic and runs on a generative sound/music engine developed in Pure Data.

Lucky Frame is Yann Seznec (artist, musician and sound designer), Jonathan Brodsky (artist, designer, musician, coder) and Sean McIlroy (illustrator and print maker). Jon and Yann were kind enough to make some time right after the release of the game to talk about the sounds and technology behind Pugs Luv Beats.

DS: How did Pugs Luv Beats come together?

Yann: Jon and I made an app called Mujik a couple of years ago. A lot of people downloaded it and there were a lot of good reviews. It was basically a different approach to music on a mobile interface. After playing around with that for a while we started thinking about how much further we could take the idea and Jon started getting into the idea of making games. So, we started thinking about how we could really bridge that gap between music and games. What if you could use a game interface to create music rather than to play music that is already there? That was the starting point. Jon made a demo which we dubbed ‘Space Hero’. The idea was that you were controlling a little ship that was shooting enemies. As the enemies came on screen they made a sound and as you destroyed them they made a sound, with the twist being you could edit how the enemies came after you so it was like a piano roll hybrid drum sequencer. It was more of a proof-of-concept than anything else. We took that to Channel 4 and to make a very long story short they ended up eventually telling us that they liked the idea and that they wanted to invest in it. Interestingly they told us, ‘We want to invest in the concept but don’t make that game’ [laughs]. So we started making various different prototypes for what became Pugs Luv Beats.

Click here to view the embedded video.

DS: What is Pug Luv Beats about?

Yann: We were just prototyping with lines and dots on the screen and I think we went through two full prototypes. We ended up with a little bug and you had to tell the bug where to go, and once it did it would create a sound and then you would have a second bug and you can tell that bug where to go and it would instantly start creating these interesting rhythmic loops. That was the core of the game. Once we had a fun-to-play instrument, we tried to figure out how to make a game out of it – which was quite a challenge. Eventually the bugs became pugs and we designed this whole crazy world around them and decided the sounds that they made would be dependent on what terrain they were on.

We then made a bunch of different terrains and terrain sound banks and each of them react slightly differently. We then had the idea of the pugs being slower on certain kind of terrains, and you can speed them up with costumes – if you want. If you don’t, then it means that they are slow. If they have a costume its going to be 25-50% faster which is really interesting because something that has a slow melody and rhythm will speed up.

The way we have it set up with ratios is quite cool because there are eight terrains and each of them has a slightly different ratio of ‘slower-ness’, which means you can get a ratio of 66% between two of the same distance and if it’s 4/3 or 5/4 or 7/4 you can get all these interesting rhythmic ratios going. The other level that we’ve added on top of it is that where the pugs are hitting are given a value. So as the pugs are running around generating all this music, they are raising your in-game currency and allowing you to explore other places and find new terrains and new costumes and new planets. I’ve just described the full game right there – Pugs Luv Beats.

Click here to view the embedded video.

DS: So it’s a generative music game which was designed in Pure Data? Why Pure Data?

Yann: That was a decision we made at the end of the prototype phase, it was almost too late.

Jon: LibPD (embeddable PD core) came out some time in 2010 and PD has been around for god knows how long. The embeddable core is so great for us.

Yann: It’s really amazing because before that we didn’t really have a proper way of doing sound at all.

Jon: I was writing the entire DSP engine and right before we got in to LibPD I had a chain-able DSP system that was defined through code. But, it wasn’t twenty years of audio research [laughs] that we could just drop in to our program.

Yann: I know very little code. Whereas, my training, if I can call it that, is in graphical programming with software like Max. I felt our time wasn’t being used effectively. As soon as we were able to integrate LibPD in to the whole system, it made our sound development strategy thousands and thousands of times better. Pure Data, being a brilliant and occasionally frustrating thing, was overall really good. The integration through LibPd and in to Objective C is surprisingly painless once its all setup.

Jon: A lot of people have done a lot of good work. The toolkit that we are using for running the game is called openFrameworks, which like Max or PD came out of the art and code movement. And a guy made a wrapper that takes the nice parts of LibPD and none of the nitty gritty things and it was probably less than a day’s worth of work to drop PD in to the game engine I had been building for a few months. LibPD itself was made by Peter Brinkmann with help from Peter Kirn. We have a lot to thank them both for.

DS: How is PD integrated with the game engine?

Jon: I basically send messages, just like how you would send messages in PD or Max/MSP between your different sub patches. I send those same messages from my game engine about what’s happening in the world in to LibPD. I construct lists that say, “There’s a bug on this tile, he’s picking up a beet and the BPM is whatever and the tile is this type”. There’s really not much integration, I can just tell PD what’s happening in the game at any point in time and then PD/sound designer deals with it.

Yann: Yeah, it literally was me making Pure Data patches as if I was making Pure Data patches for anything. The only difference being I have a little receive box in there that Jon is making the sends for and it outputs a giant list.

Jon: There’s one list for sound effects, there’s one for what happens in the world and there is one when the world is setup – I use a random number generator to generate the world.

Yann: It’s really great. It’s an amazing system. I get these three lists. The last list Jon described is the first thing that happens.

DS: So the world creation is completely random?

Jon: The nice thing about random numbers on computers is that they are totally predictable. The way random number generators work is that you give them a seed value and then every sequential random number is based on that seed value, so I don’t store the worlds, I just store seed values. When you go back to the world, I see the random number generator and regenerate the whole thing. That is also how we communicate about what world is on screen – just through those seed values.

Yann: We never even see that number.

DS: So it is a sort of identifier for the world?

Yann: That is exactly what it is. I use this number to do things, like pick a sound library for the beats.

DS: How many libraries does the game use?

Yann: At the moment there are only five. We may add some later on.

Jon: But that’s partially true. Only half of the sound engine is the beat sounds.

Yann: So beats are what happens when a Pug hits a beet [laughs]

Jon: The spelling of beats varies between vegetable beets and musical beats throughout the code. Sometimes the Pugs are still called rats for historical reasons [laughs]

Yann: Going through the code is hilarious! There are 5 beat libraries, which are each comprised of: a kick equivalent, a snare/clap equivalent and sound for when the beet is harvested. Otherwise there are libraries for each terrain as well. When it chooses a random number, it uses that random number to select which beat library it will use. It also sets a couple of other things – like there’s a little synthesiser you can play along with the planet. It will choose an attack and a decay on that synthesiser as well, that way each planet will have a lightly different sounding synthesiser. Another important thing that it does is it picks a scale or a mode (depending on how you want to think of it) and it applies that scale or mode to all the sound libraries and the synthesiser as well (the synthesiser is a semi-tuned synth). That way whenever you got to a planet it will be one of eight scales/modes. The way I’ve done that is for each library I have made a thirteen file sound-set for a chromatic scale, but that random number will select which look-up table it’s using and will select only eight of those thirteen. It does that across all the sound libraries. Another part of that list is the BPM. Each planet sounds quite different, even on a global level – in terms of being slower/faster, minor, mixolydian, major, etc.

DS: Is the BPM random?

Yann: The BPM for each planet is random, but will be the same when you return to that planet each time. Jon generates it from the random number, but sends it to me separately so its easier. I use that BPM not to trigger the sounds, because I’m getting the triggers from the code, but to calculate the delay times so that they are synced. I’ve put a soft delay on virtually everything to give a ‘rounder’ feeling. This whole process happens in a tenth of a second. Once you are on a planet, you have all these pugs running around. If you set a node and the pug starts running out, every time the pug jumps on to a new square I will get a list of numbers from Jon’s code. The list will be an identifier for the pug, what terrain he is on, whether he is wearing a hat, whether he is wearing a skin, whether the node he is jumping on is a node you placed or if he is just jumping on it to get to the node (I use that to make two different sounds).

Jon: It easier to think of the world as a heightmap rather than separate pieces of terrain. So there is actually an even more specific number. You can say, “this is sand, but also sand 0.5”, you can get a number between 0 and 1.

Yann: In practice that number is giving us a separate value for each terrain tile, which gives us a kind of location. If we have a specific number for each tile, every time a pug goes to that tile it plays the same sound.

Jon: It makes it like an instrument you can discover. It’s like a piano where you don’t know what note each key is going to play, but if you go back to that key it will play the same note.

DS: Each terrain has a different sound bank then?

Yann: Yes. I tried to match a vague conceptual thing to each sound bank. For example, the mountain is a kind of far off sounding synthesiser. The desert is an Iranian dulcimer called the Santoor, which is quite cool. For water I have an African harp. The links between the actual sounds and the terrain are somewhat tenuous but I was prioritising on making everything sound good. It can get quite chaotic when you randomly get a planet that has got five terrains – but that’s part of the fun.

DS: Were there many technical limitations in creating sound for such a game?

Yann: It limited me in the sense that I wasn’t expecting it. I was very used to patching on computers, so I had loads of buffers and then at one point Jon asked me, ‘How many buffers are you running?’. And, I was like, “There’s eight instances of that and there’s four buffers in each, so there’s thirty-two”, and he was like, “YOU HAVE THIRTY-TWO BUFFERS?!”, and that was just in one of the sections [laughs]. In PD doing polyphony is much more round about than in Max, so I did have to learn that kind of stuff. Artistically I didn’t feel very limited by it. What was really fun about setting up this patch was trying to make it as flexible as possible so that I could develop it further down the line. One of the updates for example might have pickups that change how the audio is played, so you can have the sound in reverse for or you could have the sound engine going all wonky at some point or the other.

DS: Considering you can do anything you want, do you set up artistic limitations for yourself?

Yann: It’s really hard. I think for us the biggest advantage in forcing us to be limited is we have a real goal in mind – a released commercial piece of mobile gaming software. That’s a real interesting framework to try to place around everything. You could use the same data from this game and each formant of a sound can be applied to each pug and you can do crazy harmonic re-synthesis stuff. It would be awesome! Very cool! Who would want to play that? [laughs]

Jon: I think to a certain extent you are limited by time and complexity. There are a lot of defaults happening. The art style is very much like what’s in Sean’s sketchbooks. The African harp and Iranian dulcimer, that’s the most default Yann sample-set [laughs].

DS: What about the choice of sounds, considering it’s for iOS devices?

Yann: We made two choices very early on from a sound design perspective. One being we didn’t want to rely on people wearing headphones. Whilst I of course wish that everybody played all music games with headphones, I think it is unreasonable.

Jon: Or more preferably with studio monitors [laughs]

Yann: Yeah! “Please use medium field speakers with proper sound insulation” [laughs]. We decided very early on to make it sound awesome through headphones and awesome through the built in iphone speakers. It is a challenge. What it means is that you have to have mid-range in all your sounds and you need to make that mid-range not sound annoying when you listen on headphones. Stuff like the santoor cuts right through. And when you are dealing with generative sound like this you have to be careful about volumes. We haven’t put a compressor on anything, I haven’t really felt the need for one yet. The big sounds like bass drums were quite tricky, but it’s about putting a peak at the mid/high range to make stuff cut through.

DS: Considering most of the sounds occupy the mid-range, did you also try to create contrast between them?

Yann: The beats, the vegetable beets that create beats are the main percussive element. Otherwise I wanted to have a bit of a range. There’s a kind of a distortion guitar thing, there’s a sine wave synthesiser, there’s the santoor, there’s a xylophone/toy piano.. they need to be cohesive but also different enough so that when you land on new terrain it is really clear that a new sound is happening. It was quite challenging to make it sound like you were in the same game still. Also, everything runs at 22k but it does sound good and it’s funny how you much can get away with at 22k. It’s still in stereo.

DS: You mentioned about updates to the game. Anything around the corner?

Yann: Yes indeed! We are currently working on our next release, which is going to be a free app that is just the synthesizer part of Pugs Luv Beats, developed slightly to allow you to dress up the pugs in outfits and hats to create different synth voices. People really love playing with the singing pugs so we figured it would be a fun spinoff. That will be released in a couple of weeks.

Shortly afterwards we’re also going to release another update to Pugs Luv Beats to tweak a few things like the in-game beat currency. We’ve gotten some really great feedback on some small changes to make, so we’re going to implement a bunch of those.

After that we’re going to make a different music game entirely, but that’s all still hush hush!

DS: Pugs Luv Beats just got nominated for an IGF award for ‘Excellence in Sound’. Excited?

Yann: We are super excited! It’s absolutely amazing news for us. The other nominees are of such a high quality, it’s incredible to be associated with them. It’s particularly great to be recognised for the audio, since it is a music game and all.

http://luckyframe.co.uk
Twitter: @lucky_frame
The Amazing Rolo
More information on Pugs Luv Beats and LibPD integration on the CDM blog

Here is the first interview with this month’s special guest Elliot Koretz, talking about general aspects of his career.

How did you get started in sound design?

My first industry job was as an apprentice editor in the shipping room at Disney Studios. I was exposed to all types of editing (picture, music, and sound) but I was attracted to sound for not only what I saw as the ability to be very creative but for the autonomy of working independently of the director and producers who seemed to be always in the picture editors room. At Disney I met a sound editor who was also moonlighting at Neiman-Tillar, a leading independent sound house back in the day. He saw my interest in wanting to advance to editor a little quicker than what was the norm at Disney and offered to put in a good word for me there. I was offered assistant editors position and took it. While there I was first introduced to electronic editing. This was approximately 1980 and they had, as far as I know, the first system that was used for this, ACCESS. That’s really pretty amazing for so long ago. I think the first show I ever cut on electronically was a tv show, “Aloha Paradise” It was a kind of “Love Boat” on land and the sound needed was pretty straight forward fx. But I do remember one particular episode where the story line had a man who was interested in a divorced woman with a young child. The kid was opposed to this relationship and at one point bites the guy on the leg in kind of a comical manner. This lead to what I believe may have been the first “design” moment of my career. I layered a celery snap with some sort of other big crunch and………I was off and running as a designer.

After that I moved around landing at a number of post facilities for a while. I was an editor at Stephen Cannell, which turned out to be a great place to learn to cut action sequences. On shows like “The A-Team” you had a week to cut an entire reel (approx 12 min) of Dia, FX, BG’s and Foley. And inevitably you had a scene like this: Our heroes were in some sort of large vehicle, traveling pretty fast on a rough surface, being chased by a helicopter that was shooting at them. They meanwhile had constructed some sort of rapid firing gun that was shooting nails or some other projectiles……..and little to none of this could be created just straight out of the sound library.

These kinds of sequences needed multi-layered design and remember this was on film. Many units and also much of the final result of my work couldn’t be heard played together until the dub stage. On an old fashion film sync block you could only hear three or four “channels” at once. Anything wider than that and you had only your experience and imagination to visualize the combined sound.

I think doing this kind of design work way back then really helped me understand how to efficiently combine elements to get the sound I wanted.

I spent some time at Soundelux when the company was still pretty young and while there moved into cutting sound on features. (Still editing on film). I did return to tv editing and ended up working first as an editor then as supervisor on the show, “MacGyver”. It was another busy design show with the lead character always inventing something to beat the bad guys that required creative design work. After a successful first season the producers wanted to change to an all-electronic post. Soundelux at that time was not prepared for the huge investment in equipment and ultimately the show was moved to a newly created facility, Modern Sound. Over that summer they built a new mix stage, foley stage, and editing rooms using both Synclavier and 24 track editing systems. I was offered to continue as the supervisor of the show and accepted. After a very brief training period at the offices of New England Digital (the creators of the Synclavier) I jumped into the world of electronic post again.

The problems we faced were immense. This was 1986 and the technology was still in it’s infancy. There were not yet sound libraries that were “digital” and the decision was made to purchase a copy of the library of a leading sound supervisor at the time, Fred Brown. Then the issue was storage. The best we could do at the time was to digitize onto floppy discs. They could only hold a few seconds of sound each so you can imagine the challenges that caused. This was truly the bleeding edge of technology.

It was at times very exhilarating but often very frustrating to be at the forefront of this transition. There were times we struggled to achieve what was extremely easy to accomplish on film and other times we saw how cool it was to work in a non destructive environment with new tools to manipulate the sound.

After that season I moved around again to a couple of different facilities but then found what turned out to be a long-term home at Weddington Productions. The three owners at that time (Steve Flick, Richard Anderson, and Mark Mangini) were doing some of the most creative sound design anywhere. There is no question that was the turning point in my becoming a much more accomplished designer. Working with the talented people at Weddington constantly challenged me to step up my game and really think hard about what I could do to impact the movie sonically in every detail.

While there I made the full time transition to ProTools and it’s world of opportunities that cutting digitally has brought to all of us.

All these pieces of the puzzle have helped form what I do today. At Universal where myself and my crew have 5.1 editing suites and all sorts of plug in devices I reference all that experience from both the film and digital worlds when conceptualizing the design work I do.

How has been the evolution of your work and how your approach to sound has changed over the years?

Well, in some ways it’s changed dramatically and in others not so much. In a practical sense I mentioned the switch from film to digital. I really embraced it and all the flexibility it gave me while staying in my editing environment and not having to wait for a reprint of something or a specialist for processing. It’s just more efficient and much easier for me to experiment with sounds. In a more subjective perspective I think I grow after every film I do. I am a very hands on supervisor and I feel that one of the perks of being in charge is that I get to choose which elements of the project I will personally handle. I still try (time and budget permitting) to be very old school in my method. I like to pull and organize the fx and bg’s my editors will work from (I always encourage and give them the option of going beyond the pull) and give them a “cut list”. I think that method lends itself more to continuity and flow of the sound of the film. If I can’t do that then I meet with the editors, run the reels and give as much info as I can to them and review the work later. I think one of the bigger changes in my approach in recent years has been to make a concerted effort to co ordinate with the composer more. We all have been in the situation in a mix where we are fighting for the same sonic space with the music tracks. If I know where the music is working and in what frequencies and what type of rhythm I can attempt to compliment it and not fight it.

I wonder how sound design has changed the way you listen.

I think I listen and think about emotions. What are we trying to say to the audience? Like with music I don’t want to fight the dialog so I see fx, bg’s and design as a tool, sometimes very subtle other times not, to promote the directors’ vision. I listen for bridging opportunities to use sound to connect scenes. I always remember on the dub stage for the movie “Speed” Greg Landaker (not sure about spelling) the lead fx mixer suggested some great ideas to do just that. The frenetic pace of the film lent itself to fast whooshing elements to bridge cuts. It was just one more layer to make it a more finished and cohesive movie.

How has been your work with directors? any particular story on that?

You touch on a very important question. I think that just as important as my design work on the film or maybe sometimes even more important is my rapport with the director (and the picture editor as well). We as supervisors and designers need to be very politically astute and sensitive to the personalities we work with. Some clients like a “take charge guy” who they are counting on to lead the way in the sound post. Some want a person that gives them exactly what they ask for…and nothing more. I guess what I am saying is that we need to size up who we are working with and as early as possible give them what they need. As wildly creative as we are we can’t lose sight that we are a service. I don’t believe that one style will fit all.

I have been very fortunate to work with some amazingly talented directors. When people look at my resume they usually want to know about Michael Mann. In addition to working on and supervising some of his television shows I supervised and did the design work on both “Collateral” and “Miami Vice”. Michael is without question a creative genius and a visionary that has given us some amazing tableaus. The challenge is that he is so demanding of himself, often working 20 hours days for seemingly months on end and he expects his team to keep up with him at all times. I think he has his ideal of the visual and sonic harmony he wants and has little tolerance if you are not on board with him. If you understand that it makes your job less difficult. People always want to hear horror stories, the truth is that the hours were long and tough but as I was mentioning in the previous question when you understand who you are working with and what they expect of you then you as a supervisor can depersonalize challenging situations for you and your crew and keep everyone on point.

I did a film with the amazing Irish director Jim Sheridan. He was a very easygoing guy with me and my crew and regaled the dub stage with wonderful tales, as is the tradition for storytellers like him. His style was more to allow me the freedom to bring design ideas to the stage and then he would give input.

I love when a director really understands and supports what sound and sound design will bring to their film. I worked with Gavin O’Connor on the film “Miracle” a few years back. He wanted realism throughout his film. He wanted hockey players that could act as opposed to actors that could skate a bit and for sound he wanted the most realistic sounding sports movie ever. We did extensive recordings of skating and hockey crowds and then mixing with Mike Minkler and Myron Nettinga we got a terrific soundtrack. Gavin was so incredibly appreciative of the work we all did and that’s always refreshing and nice to have.

Another great collaboration has been with the director Thor Freudenthal. His name may not be familiar to everyone but I think it soon will be. He is a very talented young director. I worked on “Hotel for Dogs” and “Diary of a Wimpy Kid” with him. Both films presented unusual design challenges and Thor was very supportive to make that sonic space that we all hope for available in the final mix.

All these directors I just mentioned understood the value of doing field recording for their films. Whether it’s getting out to Miami and recording onboard speed boats at over 135mph (Yikes!), Directing a crowd of 5,000 people chanting “USA, USA” or dog ADR sessions (Story on that to follow), working with someone that gets the concept of what we can bring to the film by doing these things is always a bonus.

I wonder how you approach the different roles you can play on a film, such as sound designer or supervisor. Any preference?

I really enjoy them all. Unless a particular film is just too demanding a job for me to exclusively hold both titles I will try most often and handle those myself. To accomplish that I am fortunate enough to have worked with for almost ten years one of the best assistants (who also happens to be one of the best field recordists, great editor and also talented mixer) Bruce Barris. His wide range of skills allows me the freedom to be creative while he has handled some of the other aspects of the workflow. He has been an invaluable partner in the design process.

And speaking of that I do see the work we do as a collaborative effort. I am most definitely the point man with the client but it is the entire team that I count on. With the budgets so tight these days my crew is often small. Everyone has to be really capable. I try and spend quite a bit of time with each member keeping them up to date with as much info as I can.

On some of my films for one reason or another I have assumed the role of ADR supervisor as well. I do really enjoy getting the opportunity to work with the actors.

So I guess that although design is probably my favorite part of the job, as I like to say “it’s full service” and I’m good with hands on the other tasks as well.

What are your favorite tools to work with?

I think it’s rare that I use any effect out of the library without doing some sort of tweaking to it. I use quite a bit of the standard plug ins that are included in ProTools and also the Waves bundle, Izotope (particularly Trash), AltiVerb, and Speakerphone to name a few. Multiple layers of sounds addressing different frequencies are the key. I look for new plug ins and applications all the time as they are rapidly growing.

Do you have any special method for dealing with deadlines/creative challenges?

Well, that’s not very easily accomplished. It’s time management. I think one of the most important skills in that regard is having the dub stage experience to really understand what will play and what will be less important in the overall mix. Sizing up the key sequences and looking at how much time you have to spend on them is crucial. I find this does not come naturally to everyone and I help my crew know what areas to concentrate on.

Is there any of your projects that you remember for being the most challenging or favorites?

Which of your children do you like the best, eh? So hard to answer. I will pick out one but I probably could find examples in almost all my films.

A couple of years ago I did a family film, “Hotel for Dogs”. On the surface it seemed like a fairly simple movie to do. There were some Rube Goldberg like mechanical inventions to design but otherwise I didn’t at first see any challenges or possible difficulties. Was I off the mark…..by a mile. The movie had many dogs in it (it was titled Hotel for Dogs…..right?) a number of them feature performers. The conceit was that they sounded like normal dogs. Nothing comical or unrealistic in their performance. It turned out that in every single bit of production the tracks were filled with the sound of the various trainers urging on their dogs to perform with whistles, clickers, and other devices that basically made the original sound track unusable.

So now I was faced with the reality that I had to replace every single sound all the dogs made for the entire movie. There was no library in town that has such a variety and complete sets for all these dogs. I was in serious trouble until an incredibly serendipitous event occurred. Some of my crew members and I were walking to lunch. We were working at Universal and sometimes we would cut through the theme park to eat up above us at City Walk. As we walked through the park I noticed there was a stage with the sign that read “animal act”. There was a worker standing in front with a dog beside her. I told her I was a fellow employee and what I was working on and asked, “Do the dogs in the act follow commands to bark?” She assured me they did and led me to backstage to meet the trainers. Turned out they had worked on my movie and actually some of the same dogs were here in this live show. After discussing what was needed with the trainers we set up a date and brought the dogs down to the foley stage for a “doggy” ADR session. Each dog responded to silent commands and barked, whined, sniffed and growled as we recorded them. I now had my kits for each of the main dogs in the movie.

Cutting their tracks was like doing voice replacement for about eight actors throughout an entire movie. Dogs never stop making sounds. They are always panting and licking and doing something that required considerable thought. I would find the most evocative material while still “keeping it real”.

The satisfaction came that in the final product my work was truly invisible. The dog vocals fit perfectly (being from the same dog in many instances) and no one would ever suspect that what they were hearing was not production. The work did not call attention to itself but never the less was some of the best sound work I’ve done recently.

What are your favorite films for sound?

That’s another loaded question. There is such great work out there. I go all the way back to classics like “Shane” and “Forbidden Planet” as early examples. And certainly I used to try and destroy my speakers playing “Top Gun” and then “Days of Thunder” at dangerous volume levels. The work of Ben Burtt, Gary Rydstrom, Randy Thom, Ren Klyce….I could go on and on. When I worked with at Weddington the movies that we were doing, Die Hard, Apollo 13, Speed, all the Joe Dante films……..were so incredibly well done. And recently my colleague at Universal Scott Hecker has put out some of the coolest tracks (300, Watchmen, and Suckerpunch) with Chris Jenkins and Frankie Montano mixing. I thought Avatar was an incredible piece of work knowing the difficulties in having to conceptualize design when you may still be working against a storyboard. I love movies. Always been a film fan and it’s just too hard to narrow the field on my favorites.

This is a good point to mention mixers. To understand how to collaborate and help them do their job is huge. I can’t emphasis enough my belief that it’s a team effort and although I do plenty of premixing back in the editing room I love that another set of very talented ears listens to the material and can add their expertise to it. I always try and meet with the team as early as possible and include them in the process.

Is there any advice you’d like to give to other sound designers out there?

Well, I think the key is to think divergently. Outside the box. Don’t be confined by the laws of nature. That’s how little kids think and that ability seems to disappear as we grow up. I know that there are sounds that have to be exact and correct but emotional sound has a huge role in design. And the practical advise is to put your ego aside and listen to what the filmmaker is saying and present yourself in a manner that instills confidence that you are the right person for the job. One of my favorite stories that help bring that point home is this. Walter Murch and Randy Thom were participating in a forum about sound. When Randy was speaking he told a story of how when he meets with the director he regales him or her with visions of incredible design to come with all sorts of amazing nuance and the client is wowed. They know they have he right person. The the meeting ends, Randy goes into the privacy of his editing room and says to himself, “How the f*ck am I going to do it?”

So don’t let them see you sweat. Bring your best attitude to your meetings……and then go back to your room and start panicking!

Seriously, this has been a lot of fun. Thanks for the opportunity to share some stories. I hope this has been informative and a little entertaining.

DS: Like many other audio professionals that I know, you’ve got a background in music. Do you consider that the driving force behind entering an audio profession? How did that background in musical performance affect your first foray into audio production?

JS: Since musicians are attuned to their aural surroundings, either naturally or through training, they’re also instinctively aware of the importance and influence of sound in media like video games and film. This awareness led me to the field of sound design and that seems to be the case with almost every other audio professional I’ve met. It’s certainly helpful to work with other musicians because we’ve all inherited a vocabulary of articulation, dynamics and tempo that can be applied directly to sound design, editing and mix.

My sense of phrasing and dynamics in sound design is largely due to my background in music. Like many sound designers, when describing the design of a sound or scene, I often find myself “conducting” my way through a verbalization of what I’m hearing in my head. This is why sound designers don’t talk about their work in public; people might think we’re crazy. Most importantly, performing has given me a natural education in mix, especially my performances with orchestra and chorale.

DS: You’ve also studied computer science. What was the motivation for that, and how did you envision these two fields merging in your life?

JS: When I was preparing to enter the field of game development, I realized that many game audio professionals were expected to implement and troubleshoot their work at the code level. During my studies in programming, I concentrated on building a simple audio engine and then integrating it into a larger development project. Meanwhile, my friends in Berklee’s Music Synthesis program were using the Csound programming language and early versions of Max/MSP to create sounds unlike anything I’d heard before. I wanted in on the action and my studies in C/C++ and procedural programming helped me hit the ground running.

With the robust offerings in audio middleware available today, sound designers typically don’t need to dig down to the code level to get their work done. Still, game engines and tools are growing increasingly complex to meet the scale and scope of modern titles. I consider a strong technical background to be a huge benefit for keeping up with the constant changes and for interfacing with the tech team.

DS: How would you describe your transition from music into sound design? It seems like there’s been a fairly linear progression through the games you’ve worked on.

JS: To be honest, I was concerned that moving into sound design would be more of a fork in the road than a transition, almost as though I’d have to sacrifice music to pursue sound design or vice versa. This has proven true to some extent because on large-scale projects, no one person has the bandwidth to do it all. The good news is that the modern aesthetic in game audio and cinema is blurring the lines between sound design and underscore. It’s exciting for me to hear the fields of sound design and music complementing each other, playing off one another, and arriving at a soundscape that wouldn’t have been possible given a more traditional separation of the two disciplines. I’ve embraced this aesthetic entirely.

Day-to-day, I find that I create individual sounds, compose scenes, and mix in a very musical fashion. When I look back at the work I’m happiest with, it’s always a place where I’ve developed the sound or scene “musically” in terms of voicing, articulation, and timbre.

DS: What, or who, have been some of your biggest influences sonically? What do you bring from those influences to your work?

JS: Wow, there are so many personalities and projects that have shaped my sonic style. My strongest influences come from the worlds of music and film sound. On the music side, Björk and Amon Tobin have had a huge impact on my sound design and personal aesthetic. I love the way Björk’s songwriting and vocal performances drift between haunting and visceral. If I can capture that same emotional range in my work, I know I’m doing something right. I consider Amon Tobin to be equal parts sound designer and musician. Deconstructing his music is some of the best sound design education you can get.

In the field of film sound, I’ve been heavily inspired by the work of Ben Burtt, Walter Murch, and Randy Thom. All three of them are incredibly committed to detail and quality, but more importantly they design, edit and mix thoughtfully, always complementing the overarching narrative of their films. Another thing that draws me to these three is their ability to articulate how and why they arrive at their conclusions and practices. Their lectures and articles have proven more valuable than any other resource in my career.

DS: Is there anything you do to find new influences and inspiration? What do you prioritize in continuing to hone your craft?

JS: One of the biggest challenges for an in-house game audio professional is staying current in popular culture and in the trends of audio itself. In an age of unprecedented and convenient access to games, film, music and literature, that challenge gets a lot more manageable. I draw a lot of inspiration from music and film. Spotify and sites like nme.com help me keep tabs on the latest developments in music. Netflix is my go-to outlet for discovering independent and foreign films that are hard to come by in the theaters. Most importantly, my friends and colleagues who share those same influences keep me informed of their latest discoveries.

One thing I keep in the back of my head while working on a multi-year project is that the tastes and trends in audio are always changing. I know that the sonic expectations of the entire world are being shaped by the latest releases in gaming, film and music. If I hear something unique in a new game, for instance, I immediately ask myself “Could I create that if I needed to?” because chances are, that sound will be referenced by someone on a future project. If the answer to my question is even a “maybe”, I get to work trying to recreate the scene or individual sound effect. It’s not just a good exercise professionally, it’s the type of discovery and experimentation that keeps things interesting, challenging and fun.

DS: In your career thus far, what has been your favorite challenge?

JS: My favorite challenge has been developing and adapting my sound design to meet the needs of multiple game genres. So far I’ve worked in the genres of real-time strategy, city building, sports, fighting and first-person shooter. Each genre has brought with it a tremendous amount of discovery and a new set of audio challenges. The perspective shift from top down to first person, for example, has been a huge eye opener in terms of new avenues and constraints for audio.

One thing that I always enjoy is deconstructing the legacy of each genre and the expectations from core players of that genre. I like to identify those expectations early on to ensure I’m meeting, and hopefully exceeding, what fans have come to expect from the aesthetic and player feedback of each genre.

DS: What kind of things do you do to advocate for audio throughout the rest of the development team?

JS: I learned early on that without advocacy, audio can easily fall off the radar for the rest of the studio. It doesn’t help that the audio team is typically working behind closed doors. I’ve made a concerted effort to promote the audio team within each studio by reaching out to other departments and developing the mutually beneficial relationship that can, and should, exist between them and the audio team. The key here is that talking is not enough. I show artists and designers how audio can develop their ideas with prototypes rather than trying to explain it with words or on paper. And I’ve yet to meet an artist or designer who isn’t excited to see his or her work come alive after an audio pass. Many times those prototypes lead to iteration on the original artistic or design concept resulting in a stronger end result.

I’m also a strong promoter for audio being included in the concept and pre-production phases of a development cycle. Most modern game projects are too large for audio to be treated strictly as a post-production department. I feel that the most memorable and best sounding games are those where audio is a contributing voice from the start.

DS: What developments in game audio make you excited for the future of that industry?

JS: Two things: The availability of incredible middleware and the independent game development scene. With industrial-strength audio middleware freely available to professional sound designers and students alike, new discoveries are being made all the time in terms of implementation. It means that students are going to come into the game development field already trained up on the tools we use day to day. Middleware still comes with its own technical challenges, but for game audio professionals, I think the scales are now weighted much more heavily on the side of creativity.

The independent film scene has brought us some of the most experimental and creative audio in the field of cinema. I believe independent games are headed in that same direction. Just as independent film sound is informing the sound of major motion picture, I believe the sound of independent games will influence the work of game audio professionals everywhere.

DS: What are you currently working on?

JS: I’m working on Bioshock Infinite at Irrational Games.

DS: Do you have any advice for people interested in entering game audio as a profession?

JS: First and foremost: No reel, no deal. I’m consistently surprised when I meet people who want to get into the audio field, game-related or otherwise, and haven’t created a demo reel of any kind. You live and die by your latest work in this industry and a reel is required for even most internships. There are free audio/music apps available for download and there are plenty of places to host your content once it’s ready.

Next, and this might seem obvious, I advise aspiring game audio professionals to play games! Knowing both the pitfalls and the paths to success that have come before you is invaluable. More importantly, a sound designer well-versed and well-played in video games holds a common vocabulary that can be used to interact with all team members at a game studio.

And finally, I encourage aspiring game audio professionals to use the copious amounts of information and software available on the net to learn the tools of the trade. The same tools we use every day, both audio middleware and game engines, are available to anyone with an internet connection. Download them, start learning, and start experimenting!

TONSTURM‘s first ambience library Mountain Air is available now at $99 (Introductory offer until Dec 24th. Regular price is $119)

This ambience sound pack features surround sound recordings which were captured during an extensive field recording trip in the beautiful Alps of Austria and Tyrol. You get 7.12 GB of clear and wide sounding mountain ambiences full of air, birds, brooks, crickets and cowbells in 5.0 Channel Surround HD Audio @ 24 Bit, 96 KHz.

Below is a q&a I had with Tilman Hahn, who runs TONSTURM with Emil Klotzsch.

What was your inspiration for the library?

Right from the beginning when we started Tonsturm, we had planned to create effects and ambience libraries. The ambience libraries just needed a little bit more of research as we definitely wanted to recorded them in surround. We both have a lot of ideas and try to gather more information and experiences until some of these ideas seem to materialize into a topic for a sound library. Our ideas are also influenced by our work as sound designers and sound editors. This year I was asked to work on a film that is set in the Alpine Mountains. I have been hiking and recording in Austria two times before but these recordings were all done in stereo and not for library purposes. So this seemed to be the perfect moment to start what we were already planing since last year. From what I have seen and recorded on my previous trips we knew that Austria and Tyrol would be a great place for recording our first ambience release.

How was the trip and what places were you looking for?

I went on this trip together with Bennie Diez, a good friend of mine, who is a director and vfx artist. He did the wonderful photo- and video documentation. The trip was a great experience and the perfect compensation for the daily work in front of the computer. We tested the weight of our equipment bags before but did not think about the fact that we where hiking uphill most of the time. We were reaching our physical limit every day as we are not very trained hikers. During the trip we gained more and more endurance which made everything more enjoyable.

Before we started the trip we checked the air routes above austria to find the places with less air traffic to avoid as much aircraft noise as possible. mid-September seemed to be a good time for recording in the alps as most of the cows were not on the paddock any more and the tourist season was over. During our trip we obviously searched for places far away from the bigger roads. These were mainly conservation areas. In these areas we were looking for places like high valleys which seemed to be the best isolation from any civilization noise.

I’m curious about the quietest places you found?

As we went recording in september the nature was very quiet in general. The birds do not tweet like in the spring time. Everything sounded very distant and wide. Occasionally we heard a bird fly by or a distant crow. The only time where you could hear more birds was during dawn. But it was not easy to find these quiet spots as we very often had to interrupt our recordings because of air traffic or the sound of a motor bike. In the mountains these sounds can travel pretty far and it takes a long time until they finally disappear. So the main challenge of this trip was to find the right spaces and time frames to record just the sound of nature and nothing else. We are are very happy that we have found these really quiet places and daytimes where you could listen only to the sound of nature.

How were the gear setups?

The rig consisted of four omnidirectional Sennheiser MKH 8020 microphones, four single mic stands and a Sound Devices 788T multichannel recorder. We wanted to capture this impressive landscape as open and wide sounding as possible. Through four individual mic stands we were able to realize a big distance between the individual microphones. This distance and the omnidirectional pattern of the MKH 8020 were the key to this very pleasingly diffuse, wide and open sound field of these ambience recordings. We also chose the MKH 8020 for their very low self noise as we had to be able to capture very quiet and subtle sounds.

Any favorite recordings/places from the package?

One night we were recording on top of a hill, it was very quiet. Suddenly we heard footsteps of several deer. They seemed to be curious and approached from both sides. On the headphones it sounded pretty scary. We got a bit afraid as we did not know if they could get dangerous. So we stood up and started to talk to show some presence.
Then suddenly there was this big evil sounding grunt of a roebuck and we ran away… but I guess we were not in danger at all.

Another great moment was when this single bird flew by as we were recording in this beautiful valley. It was very early in the morning.

Designing Sound: The scene was very subtle and had a lot of quiet sounds. It also had a lot of tension. Was this a difficult scene to approach?

Harry Cohen: Technically the hardest part on that was all the production dialog arrived with a lot of hum on it from the generator. Luckily Izotope RX2 had a De-Hum plug-in in it that allows you to dial in the European frequency. That’s how I had to start, was by processing everything with that. You don’t try to get it all out, or it takes too big of a chunk out of the dialog.

After that, we wanted to come up with some background winds and tones that further helped mask that as much as possible…then do a lot of really detailed foley. We get into what we call hyper-reality, especially on a lot of the Tarrantino films. So, as the scene goes on, we start to back off on the backgrounds and the tones and stuff, and bring the focus in on the dialog We had to suck the air out of the scene a little bit, so that it gives you a little more closeness to the characters.

Mainly it was what Cristoph Waltz [ed. Hans Landa character] did with his performance, his eyes and stuff, as he turns from this bumbling almost Clouseau character into the menacing Nazi Jew hunter he reveals himself to be. It was riveting.

DS: It was a great scene. It really set the tone for the movie and grabbed you right from the beginning. It was awesome.

HC: You know, Quentin and his editor Sally Menke, who we’ve unfortunately lost, they’re so focused in on the tiny details…the sound of his pen, taking things out of his briefcase, and the smoothing of the paper. We agonized over every little sound in that scene.

DS: Well, it shows. You really had that emphasis on all of those movements and foley sounds. What do you feel that decision affords with respect to the dialog and visuals, when you’re getting that microscopic with the sounds? How does that affect the rest of the scene?

HC: My partner in crime…at least that’s how I refer to him…Wylie Stateman, worked on that. He’s a real master of that. He knows when to put the microphone further away from the source than you’d normally think, to get proper distance for marriage with production, and he knows when to go for that real close up detail. The embers of the tobacco lighting, or the cap on the ink bottle, the leather creak of the Nazi uniform, the creak of the wood of the chair, the slosh of the milk in the bottle…all so microscopically detailed out. The flow of the scene determines when we change the balance of those things. It was a very slow coiling of the change in the atmosphere throughout that scene.

Originally, there weren’t cows visually in the scene. But we put cows in the background [ed. referring to the audio], because they said it was a milk farm. Quentin liked the cows so much that he had them put visual effects cows in, after the fact. Then the very last cow that you hear as we’re leaving the scene is Quentin. He was saying, “No, I want one that goes like this.” He did it, and it was, “Well, we’ll just use that.”

DS: [laughs] I really like that. Going along with what you were talking about, as the scene progresses and making room for stuff…you mentioned these drones and other effects going on in the background. I noticed a point where there was a strong drone-like sound when Cristoph leans in and asks Denis Menochet [ed. Perrier LaPadite character] to have the girls step outside; there was something there that becomes much more present.

HC: Yeah, and while that may have worked artistically, it was probably brought up there because we were digging for the dialog track. The hum that was in there probably needed a little bit more masking. If that worked in a dramatic way, I’m glad. But it was probably for that reason.

DS: So a happy accident then, huh?

HC: Yeah. I can’t really remember, in detail, all of the decisions made in the mix. That movie was mixed in a relatively short time. As we’ve gone on with working with Quentin, we try to get it to a point where we start to…I don’t want to say we know what he wants…we start to learn what works for him and what doesn’t. We get things very much in shape; and again, Sally was a big part of it. We’d always run the stuff with Sally before Quentin would come for his playback. She was so tuned in to what we was going to like.

I don’t think we mixed final for even two weeks. I can’t remember, but it was something like that. Then they took it to Cannes and came back. Then we did, like, 4 more days of fixes…and that was it. That works for him, because he likes to keep editing and tweaking…and there was some discussion about some additional music being added. So, if he doesn’t have to spend a lot of time leading us through what he wants the soundtrack to do bit by bit, then he’s all the happier.

I think on Grindhouse, we only had him for five or six days on final “final,” but, again, Sally was there. Eventually, after enough films, we get to a point where he would trust us more; and we would send him more stuff, early on, for them to have in the Avid. When we first started working with him, we would get an Avid track that was fully built out, and it was real indicative of what he wanted. Then we would have to work to achieve the same thing, emotionally, that he did in his Avid track…with what we thought were better sounds. or he would just want to use his Avid track

Finally, we’d reach a point where…like the anime sequence in Kill Bill…when I got the Avid track back, there was nothing in there but what I had sent him. So it was, “OK! We’re starting to get the hang of it.”

It reminds me of the shoot-out scene in Inglourious Basterds. When I got the Avid track…production track…the gun shots that were in the production track were very unusable. It was just digital clips and smacks, and kind of ugly. But all of the really dirty, rough, sounds of the movement that were also in the track…I had a suspicion that they were going to be used to hearing that. I took that track, I cut out the gunshots, and I created a track of just the cleaned up production movement. Not totally clean, but cleaned up of anything that was obnoxious. Then I had Effects Mixer, Tony Lamberti, we were all done with the scene…and I said, “I got one more track. I just want you to prepare this and stick it in there. And just leave it until I tell you I think we might need it.”

Then sure enough when we were reviewing the scene, Quentin and Sally were saying, “It’s really good, but there’s something missing.” I said to Tony, “You remember that track? Turn it on now.” And that was it for them. They had gotten used to hearing all of that stuff. That was the final spice for them. I learned that, because I remembered back on the production of Kill Bill…the sound of all of the Crazy 88′s running around the club. I went through all of the wild tracks and collected all of that material. I created a supportive track for the foley, that ran around the way we did with the foley, but it all was production track. We used that to sweeten it, and, again, that was the final spice. Directors like him are very tuned in to their production, and they’re used to hearing what was there in the Avid.

DS: That’s not very surprising, considering all the things I’ve heard about him in the past. When this first scene opens, Menochet [LaPadite] is chopping wood, and it’s very rhythmic. Then there’s this pause that’s seemingly leaving space for the Nazi car to appear in the distance. And then we have the music that comes in, and the pitch of the car seems to blend in with that first chord of the music. How carefully was that moment planned out?

HC: The really cool thing about Quentin, from our point of view, 95% of the music he chooses from his own library. And he cuts that into the Avid. I suspect he also cuts some of the sequences to the music, and that’s why it works so well. So, when we get sequences and reels turned over to us with music in it, almost all the time, that’s the music that’s going to be there.

So, we can work against the actual music, and do things like tune elements in the effects track so that they’re constant with the music…or dissonant if that’s the point that we need. That’s something we do with elements like train whistles, and with that car engine as well.

DS: That music is kind of a “Spaghetti Western” adaptation of Beethoven’s Fur Elise, which is wonderfully absurd and crazy in a way.

HC: Yeah, I’m not sure where it came from, but that was the music that was in the temp track…and that was the music that was in the final. That’s such an advantage to us. Most of the time, the music that shows up is a temp score put together by the music editor. It just indicates what the director thinks the function of the music is…you know, the mood or what the density of it is. Sometimes when the composer takes their pass at it, they choose to go in an entirely different direction. Sometimes they’re correct in doing so. In any event, that means that sometimes I show up on the mix stage with a design sequence that’s very heavy and fat sounding, because I’ve been working against the temp music…but the composer has decided to do something really ethereal and wispy. So, my stuff is completely wrong, and I have to re-engineer it at the last minute. That’s not very likely to happen on a Tarantino film.

DS: Did the tone of that music affect the way you approached that opening scene?

HC: It did in both conscious and unconscious ways. When you watch the movie, and you’re getting the vibe of it, we’re affected by the music emotionally just like the audience would be. I affects the choices of everything you do. It even affects choices in what you’re going to go out and record…especially on this film, knowing that that’s going to be the final music.

I’m sure there are other directors that do that, but I can’t think of any other directors we’ve worked with…other than Quentin.

DS: I work primarily on television and documentary programming at work. It’s rare that we get stuff the way you are with Quentin Tarrantino there, but I do have the advantage that the music composer is in the room right next door to me.

HC: That’s great. So, you can have a dialogue with him.

DS: Yeah.

HC: We try to do that as much as possible on the films, but it’s successful to varying degrees. The composers are going through the same kind of thing that we are. They’re doing something, then going over it with the director, and making changes. The more I work on these films, the more I see that everything is prone to remain fluid later and later into the process. Visual effects are showing up at the last minute, final tweaks in the music are happening at the last minute, last minute ADR…the picture is often still being edited until the last day. So, the days when we used to be able to do something and consider it locked are long gone.

The more we can position ourselves to respond to the final shape of the film, of all the elements, the better off we are. And we make a point of knowing that’s what we’re going to go into there. It’s especially true on a lot of these big budget movies. With a solid release date and late breaking visual effects, lots of pressure from the studio… So, really what it means, is that no matter how long we work on a film, or how much lead time we have, there’s always a big crunch at the end. That’s just the nature of it. I bet it’s the same for you.

DS: Yeah. I just got a project this past week that’s already behind schedule.

HC: Yeah, as soon as you get it, you’re behind schedule. On Green Lantern, which I helped out with with some creature stuff, that fell into our laps at the last minute. It had nothing to do with the quality of what had already been done. Someone had been working on it for months, and they had been doing really good work. The project fell to Soundelux, and they made the decision to redo all of the sounds. My first day on the movie, was the first day of effects pre-dubs. So I was WAY way behind…crash and burn schedule. So, I see more and more of that.

DS: I don’t think that’s going to go away anytime soon. It’s probably just going to become the norm.

HC: The more they advance technically, the more that’s going to be like that. When they finally get to the point where we’re delivering the final directly to the movie theatres…with digital, where they don’t have to strike prints…we’ll be working until the day before it’s in theatres. You can see from the film-maker’s perspective…why not?! If you can make the film better, then why not go ahead and make it better?

DS: Let’s turn back to this opening scene that we were talking about. As far as the ambiences, we’ve talked about using the background sounds to mask noise from the production track, but these cows and roosters that pop in every once in a while… Were they primarily to mask noisy elements, or were they more? I feel like they accent certain moments in the scene.

HC: Yeah. Not so much story as rhythm. You might have a bird track, but go and put in certain spotted birds or something. In the opening scene, the backgrounds were done by Ann Scibelli. The individual spotted stuff is something that we went through and shifted the position of once Quentin was there. “Cut that one out. Move this one here. That one’s good.” You know, you put a cow moo in the wrong place and you make something funny where you don’t want it funny. And you put it in the right place, and it just accents the pause in between the dialog.

DS: The cow when Christoph pulls out that giant pipe was just perfect.

HC: [laughs] Yeah. It becomes about rhythm…the rhythm of the scene and rhythm of the dialog.

DS: Would you talk a little bit about your approach fro the spaces above the floor and below the floor?

HC: We wanted it to be a definite shift, and a subterranean feel, when we went below the floor. It’s kind of a cave tone, and we played with some more subjective things, almost haunted…it’s very light in there. It’s not a huge thing, it’s just enough that we’re closer to the ground and below the house. It’s very cool, because up until that point we didn’t know that anyone was under the floor.

DS: The rhythm of this scene just carries itself through so perfectly.

HC: That has everything to do with the edit and the dialog. I first watched the rough cut of that scene in Berlin, working on the sound of the movie within the movie, Nation’s Pride. We went to Berlin and actually posted that as its own movie, so that it had the soundtrack ready…so Quentin could project it in the theatre while he was shooting the rest of the movie. While we were there, Sally ran the opening scene of the movie for us. It was already 90 percent of the way there with its rhythm and spookiness. It was such a great performance from the actors, that it was very powerful even before we did anything to it. It’s really great when you watch a sequence, and just by watching it, it let’s you know pretty much what to do with it. That’s how that sequence felt. There wasn’t a lot of searching going on. It just kind of speaks to you, and let’s you know what is needed to do.

DS: And as it carries through into the climax, we’ve got the guns, and the music, and Mélanie Laurent [ed. Shosanna Dreyfus] escaping…

HC: And the guns. [laughs] Let me talk for a second about the guns.

DS: Go for it.

HC: I swear, we do new guns for every film. I start off thinking, “Are we going to have to do new guns? We’ve got so many guns.” You know, at Soundelux, we’ve been out to record guns, like, 45 times in the last years at least. And yet, when we get to the next film, and it’s got its own vibe and its own feel, I find that I wind up making new guns all over again.

In Inglourious it’s a “Spaghetti Western,” and I know Quentin was inspired by the vibe of The Dirt Dozen in some way. I went back and listened to that movie, and the effects didn’t sound great, but I thought that I wanted to get a real analog sound for the guns. The gun sounds have come a long way since the early stuff. You’d just over-record on Nagra and get that big powerful analog smeared sound. Now we have a lot more detail and a lot more transient, and low-end and size, the gun sounds have become a lot more realistic…in some ways. In this movie, I wanted to give a nod to that analog character that was in some of those movies.

The same with the soundtrack on Nation’s Pride. If you watch the movie within the movie, you hear late 30′s/early 40′s audio on the gunshots…but most of that is either production audio that I processed, or other gunshots that I processed to sound vintage, then added surface noise to take it the rest of the way there. Technically, that was probably the hardest part of the movie; creating a “vintage” chain that worked for us. In mixing, I think I had five versions of that chain going; one for dialog, one for backgrounds, effects and such. And we did that all in Berlin. So that was pretty audacious.

We weren’t sure. We had developed two approaches to “vintagizing” the sound for Nation’s Pride. One was that real extensive chain that I developed, and I think I even detailed everything that was in that chain in an earlier interview about the movie in the Editor’s Guild magazine. We also took this old Magnavox metal speaker horn, that Wylie had come across, and we brought that to Berlin with us. We finished the mix on the movie, and we looked at the engineers and said, “Now we want to run it through this.” They just thought we were out of our minds. [laughs]

But they did it. They had to remove panels, because they weren’t set up to do anything outside at all. We had to do some extra wiring and stuff, and we did it. We got the whole track coming through this ancient Maganavox horn, which I still have in my office. It sounded really cool, but ultimately we decided that it sounded maybe just ten years older. You know, maybe early 30′s rather than late 30′s/early 40′s. So it was kind of specific.

DS: With these guns that you just talked about, the moment of most tension is when Christoph is aiming at Mélanie…the gun doesn’t go off. But, we still we a gunshot like sound. Was that an effect, or was that part of the score?

HC: I can’t remember exactly. I think he does something with his mouth, and he chooses not to shoot her. He chooses not to, but he could have.

DS: Right, but there’s this kind of heavy sound that implies gunshot that cuts off the music…

HC: It was either music, or it was something that we did dramatically…just as a little stinger. Honestly, I can’t remember which at this point. Sorry.

DS: It’s been over two years since the film was released now. I’m impressed with how detailed your memory’s been so far.

HC: It’s been seven or eight films for me. It stays fresh when you’re working on it, but a few weeks after your done and it starts going away. So, I can’t remember all of the details of the mix. Do you get to go and sit in on the mixes of the documentaries you work on?

DS: Oh, I do the mixing as well. I have to do everything where I work, soup to nuts.

HC: That’s great. I was bringing it up, because some editors don’t necessarily get to be on the stage and hear how they’re sound is utilized…and see what works and what doesn’t. That’s a really important part for the education of designers and editors. We try to make the mix an additive process, instead of subtractive one.

We figured out a while ago that if we present too much stuff to the mixer, every eventuality, then we put the mixer in the position to have to figure out what they need to remove to bring clarity and focus to the scene. So, we’d rather work on the stuff that we know we want the focus to be on, and add stuff as we need it during the mix. That makes for a much more intelligent process.

I’ve seen a lot of mixes in the past where, especially if it’s a short schedule on a temp mix, the editors will cover everything and try to sort it out at the mix. But there’s never really enough time to sort it out in the mix. So, often times you end up with a mix that’s full of great sounds, but it’s so thick that you don’t really hear anything clearly. That’s a real shame.

DS: So you sat in on the mix during this film?

HC: Yeah, I’m there the whole way there; to help make all those billions of decisions…and to be the effects mixer’s wingman. Not to be obtrusive, or obnoxious, but to let the mixer know, “This track is supposed to do this,” “What you’re looking for is over here,” or “I gave you this, but you might not need it.” More than that , to at least influence the shape of the effects mix so that, as a starting point what I hear, represents my original intentions. Sometimes an editor will cut sounds, and when they hear the mix back, it will sound like the mixer has reinvented their sounds. I think it’s not a conscious reinventing of the material. But when the mixer hears those tracks, he’s got to listen to them one by one and make decisions right?

You life a fader and go, “Well, I like that.” You lift another, “I’m not so sure about that.” You make decisions as you go along, because you’ve got a lot of decisions to make. So, if I’m there, I can at least say, “Leave it up. It works with the other ones, and we can come back to it later.” That way, it presents itself from the starting point I wanted it to, then we go from there.

DS: So, what about the end of the scene. The ambiences come up heavily again, we get cicadas and insects that come in for the first time. Did that come out the way you had envisioned it?

HC: Some of that shaping definitely has to do with the mixers themselves. We didn’t necessarily sculpt the rise and fall of all those elements as much as they are, other than to prepare them and make them available there. I think on that film, if I’m not mistaken, I think Mike Minkler pre-dubbed the backgrounds. I can’t remember if he mixed them as well. Because he pre-dubbed them, he was very familiar and very tuned into them. If a mixer pre-dubs material, he almost assumes a little bit of ownership over them. Mike Minkler has a big voice in the shaping of the whole track. He’ll give me notes, and I’ve learned to take them very seriously, and try to figure out what he’s after with those notes.

It almost always makes the scene better. He’s thinking in terms of momentum, like in the last sequence…the burning of the theatre. He would come to me and say, “I think we need a long rising vocal thing that sweeps up through the whole thing to a crescendo.” That was something I hadn’t thought about, but when I prepared it to put into the track it worked really well as an element of the background crowd that’s screaming. There’s this thing which is rising in pitch, like a big choral swell, and that helps build the tension and the drama throughout that scene.

So, he was probably the one who said, “Let’s push the backgrounds at the end of that scene.” He was very familiar with them, and probably had a concept of what they might do for that portion of the scene.

DS: And is there any particular moment in this opening scene that you’re really proud of? I know you talked about the Nation’s Pride work as being the most grueling challenge, but is there anything in this scene that makes you say, “I’m really glad this came out this way.”

HC: All of it really. I, of course, love all the guns that are in the movie. [laughs]

Just the fact that we were able to effectively support the way that scene unfolded as a whole, and the way it just goes from benign to really threatening so subtly. It’s hard to pick a point where you realize that it’s gone from bumbling fool to real menace. Everything was just trying to be in support of that whole vision.

DS: The scene just works perfectly together. The dialog, the visuals, the editing, the sound…Of course, a lot of that credit goes to Quentin Tarrantino, but it also goes to the people he had working on it…pulling in the people that he did. I think it just came out beautifully.

HC: I agree. And, you know, it was funny. When we were working on it, they knew it was really different and unusual. But they weren’t sure it was going to find its audience. He gets to follow his muse.

A lot of films, they do screenings and reviews, and they change things according to the audience reactions. Quentin pretty much gets to make the movies that he wants to make, and his audience is there for him or its not. But it’s what he intended, so it’s not film-making by committee.

DS: Thanks for taking the time to chat with me.

HC: My pleasure. Thanks for taking the time to talk about the film.

Twisted Tools has released Transform, a new sound design library created by Jean-Edouard Miclot, french sound designer from Vancouver, Canada..

TRANSFORM is an extensive collection of field recordings, sound effects and designed sounds developed by sound designer Jean-Edouard Miclot (a.k.a. JEDSOUND). Bundled with sample mappings for many popular formats, TRANSFORM’s painstakingly recorded and processed sounds will find their home in the arsenals of sound designers, editors and music producers alike.

TRANSFORM features over 1.6 Gigabytes of 24-bit/96khz audio, all meticulously embedded with Soundminer enriched metadata and processed using a plethora of sound design tools, such as Symbolic Sound’s Kyma. Whether you’re a sound designer needing a massive Hollywood impact, an editor looking for a radio stinger or a musician wanting to add some ice crunch to a snare…or bass wobbles made from a processed moose, TRANSFORM has something for you.

TRANSFORM comes with sampler presets for the EXS24MKII, Kontakt, Battery, Maschine and Reaktor as well as the all new MP16c sampler for *Reaktor. To top it off, we’ve included MP16c templates for Maschine, Kore and TouchOSC for the iPad, as well as bonus material by Richard Devine.

Transform is available now at $69.

Below is an interview I had with Jean-Edouard talking about his work on the library. There’s also a SFX Lab dedicated to explore sound transformation and featuring sounds from the package.

What were your main inspirations for doing this library?

Josh Hinden from Twisted Tools contacted me in early 2011 to ask me to make a sound library for musicians and sound editors looking for new sonic textures. At this time, Amon Tobin was just releasing his new album ISAM and Josh advised me to listen to it and maybe draw some inspiration from it. So one evening, I came back from work, turned off the lights and dived into Amon’s universe. I had that feeling of being an acrobat walking on a wire swinging from left to right. I loved that feeling of instability and metamorphosis between organic and melodic sounds.

Twisted Tools gave me enough time to develop the ideas. At first, I went about things a bit too rigorously, but I quickly noticed it was a hindrance to creativity. I knew that I had to experiment and let things go wrong. You do something and then you innocently think ’What if I do the exact opposite?’ That’s something that I do when I cook too. Well… very often as you imagine, it doesn’t work. Once in a while though, you capture that essence that makes your emotions react in a certain way. In my opinion, that’s how I get the most characterful sounds, or the most interesting flavors. I’m still young and full of misconceptions so I always feel I have to experiment more than others. Science, history, economy, politics, art, nature, gastronomy etc. can inspire you to be better at what you do, whatever you do. The world is an immense source of inspiration and only being focused on one domain like films or games is in my opinion, an error that can only lead you to copy what others have done in the past.

What were your main/favorite tools for recording and designing these sounds?

The tools that we have on the market today can be very expensive so you want to make sure you make the right choice for yourself and that this investment is profitable in the long run. I own the classic Sound Devices 722, a Neumann RSM191 AS and a Sony PCM D50. I also have a few custom guitar pickups, a H2 XLR hydrophone and a couple of cheap contact mics that I’m looking to change. Plug-ins can also be very expensive and I thought a few years ago that if I could analyze how some plug-ins worked and if I had a modular system that allowed me to combine processing chains, I could probably create my own… I bought a Paca hardware unit from Symbolic Sound and I started to customize my own processing chains in Kyma. I mostly learned not to use Kyma to do the same things that other plugins do. Instead, I save all the sounds that return glitches and weird behaviors that stimulate my emotional response. If that makes me giggle, makes me scared or gives me a “wow” feeling etc. then I know I have to record it. Otherwise, I work with the industry standard Pro Tools, Soundminer, Waves, Sound Toys and Altiverb. What I like to play the most with are probably the toys that I collected over the years. I have tons of different springs that I sometimes stretch out on stairs, some neonodyum magnets that keep my colleagues entertained, some DC motors that I control on a breadboard with variable resistors, many whistles that I sometimes stick on arrows or at the end of a weighted string, some bullroarers, a double windwand, a professional whip, some bungee cords, some goopy liquids etc. I also recently ordered a custom Tesla Coil that I’ll be able to remotely control. If you don’t know that website yet, check out instructables.com, there are tons of interesting things to build for almost no cost if you’re willing to get your hands dirty.

Could you tell us about the elements you recorded/combined for some of the categories of the library?

The sound library is subdivided in 10 categories:

• Bass: This category contains any kind of low pitch or tonal materials. It could be an elastic stretched out, a low hum picked up from a transformer, a sub speaker rattling a file cabinet, a moose moan, a mouth gurgling water, some dry ice pitched down etc. anything that has a main frequency component perceived under 200Hz and that could be used as layers to support other mid-range sounds. Most of those sounds were processed in order to reinforce the tonal quality or add more sub and harmonics.
• Crunch: This refers to props or natural elements that produced loud transients like wood, metal, ice or plastic and glass stress. These natural transients were amplified to give a bigger and sharper edge and not necessarily sound like what they originally come from.
• Eerie: Eerie means here any long sounds recorded in a large acoustic space or processed in order to recreate an abstract tonal atmosphere. It could be props clinked in a garage, a door squeaking in the washroom, a bowed spring,  a wine glass rubbed underwater, a large metal sheet mangled like thunder or any spectral processing done with Kyma, Metasynth and the Michael Norris plugin suite that give musical and emotional qualities.
• Fx: This category represents sounds that don’t belong to the reality of our world. Although, they could come from DC motors, drills, a Kazoo amplified with a guitar amp, animal vocalizations, kitchen appliances, engines, small motors, fax machines, printers etc. They were all processed to help to express feelings of threat, anger, happiness, silliness, rage, surprise etc. without necessarily showing what the source of the original sound actually was.
• Impact: As the word says, anything that smashes, slams and hits hard. Most of them come from objects that we see everyday like doors, latches, cabinets, drawers, washing machines, fridges, jackhammers etc. Because the nature of something that hits really hard sounds usually noisy due to the addition of harmonics, those impacts were actually performed pretty softly and have been later aggressively compressed to reinforce their dramatic energy.
• Mecha: These are made of various metal objects that get locked, unlocked, pushed, cranked and hit in order to give a sense of mechanics and build-up.
• Micros: These are very short samples (close to one wave cycles) extracted from natural sonic waveforms and supposed to be played back with the Kyma microsound sampler provided within the library. They’re made of springs, neonodium magnet buzzes, wood stumps, door creaks, mouth noises etc.
• Organic: These sounds are supposed to come from the world we know but not necessarily from places we expect them to come from. Some tire skids could be just a plastic headset rubbed on a wooden table, a bird call is made by blowing on the edge of a sheet of paper, a muddy footstep is made by a plunger pushing 3L of hair gel, a subway squeal is a shovel pressing a block of dry ice, a cobra snake hiss is a garden hose spraying water, a frog croak is a finger rubbing a shot glass, the glass of a water tank cracking is just made by bending a plastic CD case etc.
• Whoosh: These sounds are composite and made of several different tonal layers going through a treatment in Kyma and finally performed with a doppler effect to express velocity and movements.
• Composites: Those are just examples of combined sounds from the other categories.

I wonder if you had a reference for designing the sounds. Visual images, text, etc… or just wild and pure imagination?

I didn’t directly use specific materials as a main source of inspiration for this project. It represented challenges as it was open to any kind of experiments and you might go too far sometimes. I tried to restrain myself to the tonal and emotional quality of what musicians and sound editors could be looking for. Some other things might have subconsciously influenced me though. I like going to the theater, exhibitions, music concerts, classical performances, improv theatre, good restaurants… I went a few years ago to a black and white photography exhibition in Paris of an individual who traveled all around the different landscapes and cultures of Africa (I wish I could remember his name). I remember all the feelings and emotions I had looking at those pictures… They were so expressive I could hear sounds coming out of them. It took me a while to realize that what I saw changed my vision of listening to the world. Now I usually only record sounds that express a feeling or an emotion to me. If a sound doesn’t result in an emotional response, then it is just noise because it has no meaning and so in my opinion, it doesn’t need to be recorded. Some people say that you have to record as much as possible and that is true, but it doesn’t mean you have to record everything. Building your own libraries can take a big part of your life and you have to make sure you make good use of it. This also works when you’re experimenting with chain processing for conceptual sounds.

What are your favorite sounds from the library?

Tough one… Hmmm, I would say the UnderwaterGlass in the Eerie category. I scrubbed the top of a wine glass to make it ring and recorded it underwater in a bathtub with the hydrophone. The glass was horizontal so that I could control the amount of water going in it. As the glass was going up and down in the water, it was changing the pitch of the tone resulting in some kind of whale calls. I processed it through the Altiverb and that’s how I arrived at the final product. I like that sound because it’s organic enough to make you believe it’s real but you don’t know necessary what it is. It’s also very expressive because it was performed like a musical instrument and has a wide range of frequencies that makes him a good candidate for an ambient music track.

You’re currently working at Ubisoft, right? Could you tell us a bit about your work there and what you’re currently doing? Also, are you interested in releasing another library any time soon?

That’s right! I got hired by Dorian Pareis at Ubisoft to work on a XBox Kinect/PS3 Move based title called MotionSport Adrenaline that just got released a few weeks ago. I came at the end of the project to help to create and implement new assets (FrontEnd, HUD, Spfx and NIS sequences) and I had to quickly learn their proprietary tools. The user interface for example is very colourful and I was basically asked to make it happy without being cheesy. I took the wine glass, layered some rings pitched in major chords and it made those transitions very smooth and melodic that fit very well with the visuals. In Wingsuit, you have a strong wind that pushes you violently as you’re diving down in the Himalayas. It had to be scary enough to tell the player that it’s a dangerous obstacle, so I used some “clichéed” animal vocalizations dopplered with some kind of Kyma shimmers panning very quickly and when you hear it, you know you’re in trouble! In Rock Climbing, multiple waterfalls come out of the wall like giant vomit and it will make you fall if you’re there when it happens. I morphed some walrus vocalizations with some flushing toilet sounds that gives that sense of danger. Another example in Kayak, you have some whirlpools that suck you in the water if you don’t  avoid them. One night, I came late home from work (I mean later than usual :-) and my landlord was gardening. She dropped the hose on the ground and it made a hissing that sounded like a threatening sea snake. I came later back with the mic when it was quiet and that basically became the sound of the whirlpools. We really tried to give some sense to the audio, it informs you about the environment, what happens around you and if you pay attention to it, it will help you to beat your competitors.

We have a small audio team consisting of Dorian as the audio lead, Peter Strachan as our audio programmer, myself as audio designer and we hire contractors depending on the needs of the project. Dorian spent a lot of time with sound designer Bill Westwell recording in the field for the game. They recorded mountain bikes placing different mics on the chassis and their bodies and they had a professional rider performing what they needed, at different speeds and on the different surfaces, take offs, landings etc. They went on the North Shore to capture rivers, brooks and waterfalls and brought tons of pristine materials that Dorian used in Kayak. They also did a lot of wild recordings in a swimming pool and bought a canoe that ended up almost destroyed. All the field and studio recordings became extremely useful and really helped to enhance the gameplay and the player’s feedback.

I don’t think I’ll have time to release another library soon but I’d love to do it again. The guys from Twisted Tools really have talent to create avant-garde Reaktor ensembles and they really put some time and effort in searching what professional music producers, performers and sound designers really need.  You will always be amazed with what they can come up with! Go check them out now: http://twistedtools.com/