Ambience and Interactivity

[We are currently having a few problems with page formatting and embedding SoundCloud links, please bear with us until we fix it!]

‘Ambience’ is a word with a broad definition. It is perceived differently and can mean different things to different people. We are constantly surrounded by a wide range of sounds – some natural, some man-made. Ambient sounds don’t necessarily have to exist in the real world. We’ve all felt the power of music and know how subtle changes in a mix of sounds can add up to affect us at a subconscious level.

Continuing on that train of thought, I dug into the world of ambience in interactive mobile applications with:

Yann Seznec: Musician, sound designer, artist and founder of Lucky Frame – designers of award winning iPhone apps – Pugs Luv Beats, Bad Hotel
Peter Chilvers: Musician and software designer, best known for the series of iPhone apps created with Brian Eno – Bloom, Trope, Air, Scape
Robert Thomas: Interactive composer and CCO at RjDj – The Inception App, Dark Knight Risez Z+, Dimensions
Stephan Schütze: Composer and sound designer, Director of Sound Librarian and creator of the iPhone app Carmina Avium

DS: What does ambience mean to you?

YS: From a sound perspective, to me ambience is really a situational thing – it is sound that is occurring whilst something else is happening. So my current ambience is the hum of the computer, the typing of someone else in the room, a plane passing overhead outside. Those same exact sounds would stop being ambience in a different situation, so it’s quite relative.

PC: I think of ambience as the set of near-subliminal cues that quietly define an environment. They might be continuous sounds, like the rush of a river or distant traffic noise, or occasional point sounds, like the drip of a tap or a car horn. They invariably carry information the conscious brain is rarely aware of in the reverberation of the sound; they tell the size of the room the listener is in, the materials on the walls, the distance and location of objects and so on. I sometimes contemplate wearing a blindfold for a day to increase my awareness of these sounds. Then I remember that I work all day with a computer screen, and think better of it.

Ambience can be interesting when it’s used to evoke an imaginary world. The same ideas of distance and presence still apply, but the sound sources are unreal while still “natural” sounding. Obviously Brian [Eno] is the master of this kind of domain! With Scape, [the app that we recently released for the iPad,] we’ve provided a library of these types of sources, which the user can recombine in any way they want; it’s as much a way of building worlds as it is building music.

RT: To me its something to do with a place (which may happen to be real or virtual) which forms your state of mind or shapes your perception.

SS: Ambience is significant for me for a variety of reasons and I will refer to an App I recently launched to cite some examples, but I want to start with a brief story.

Here in the southern hemisphere it is springtime. My wife and I rent a small unit on a block with four residences and each have a very small garden space; nothing amazing but quite pleasant. Our neighbourhood in general is quite good for trees and plants. A few weeks ago we had the perfect combination of season, weather and environment. As I sat in our garden with my eyes closed I could hear about a dozen species of birds across the neighbourhood and smell the glorious scent of dozens of different flowering plants. I could have been anywhere in the world, on top of an amazing mountain range, in a royal garden or a magic forest. It was the non visual equivalent of a stunning landscape. Ambience is not limited to non visual components of our environment, but because we are so visual based as a species we tend to gather direct information from our eyes and emotional influences from our other senses.

As audio designers this provides us with an incredibly powerful method of communication. We can suggest locations, time or seasons, emotional states, potential risks and other information by what we present as an ambience. Carmina Avium is an App I spent the last 12 months developing. At its core it allows users to select from bird species and create their own ambience. It uses generative audio to create non-looping, non repetitive personalized ambiences in real time. In this case I have created a product to allow a user to simulate a natural environmental ambience wherever they may be located. Most people associate the sounds of natural ambiences with relaxation and focus, but I could have easily created the same App to create scary or spooky ambiences for Halloween or dramatic ambiences to which to jog to.

DS: Ambience = Mood. Your Thoughts?

YS: I suppose it depends what you mean by the equals sign! Certainly ambience can create or manipulate mood, but it is not always going to be a direct correlation. Chirping birds could sound beautiful or menacing, depending on the situation. So I think they are deeply related, but it can be misleading to think of them interchangeably.

PC: The relationship between ambience and mood is a rather varied one. Where the ambience is taken from the natural world, the listener’s mood is likely to be governed by their own associations with the sounds. Where that ambience is depicting a more imaginary world, there’s much more room for the character of the sound itself to exert an influence, whether it’s soothing or abrasive. Much of ambient music is calming, but it needn’t be;  when Brian and I were working together on the Spore soundtrack, our first project together, we found ourselves creating plenty of extremely alien, often rather troubled sonic worlds.

RT: In a way, although I think you could be in a sad mood, but in a place which is has a hectic ambience.. or a light ambience.. The ambience augments and shapes your mood, allows you to see the world and yourself from an altered perspective.

SS: Defining emotional states and emotional triggers is always tricky as seldom will two people react to an emotional situation in the same way. We do use the “ambience” of a situation to measure mood, both the existing mood as we enter a space and perhaps the expected mood we should adopt in the environment. I would suggest sound will often play the dominant role in defining this. We whisper in a church because often it is a very quiet environment, it feels out of place to be loud when all else is quiet, whereas a beach with crashing waves has an inherent energy about it. The active nature of big waves energizes us, while in contrast the quiet lapping of the tide at your feet is more likely to calm you on a quiet beach. We all understand that music can be used to influence emotional states, but I suspect that the ambience that accompanies music goes a long way to establishing the foundations of how we will end up feeling.

DS: With interactive apps, too much control for the user makes it difficult to set up a mood and control the experience. Its a fine balance. Any tips from your experience?

YS: Yes, controlling the user’s experience is really important. It will obviously be different for different types of apps, but I generally like to take control away from the user. In terms of ambience and mood, that is usually something you’ll want to design from the top down, based on artistic, technical, and creative design decisions. It can be something that the user will change based on their decisions, but that change can be designed as well. An example from my own work might be the background audio during gameplay of Bad Hotel…there are a few different ambient loops going on at various pitches and speeds, which creates an endless but non-repetitive bed of audio on top of which the musical generation happens. But that bed of audio will subtly change based on the size and structure of the hotel that is built (opening up a filter, adding some delay, etc). So the user is changing it, but it is essentially controlled and designed. This lets it be different for each level, and serves to frame the “foreground” audio.

PC: The greatest difficulty is in preventing too many things happening at once, and all of our apps, which are exposed to the whims of their users, have some sort of dampening mechanism. If a user plays a flurry of notes rapidly on Bloom, for example, newer notes will get quieter and quieter, and older notes will be less likely to play.

Users tend to enforce this themselves over time. Almost everyone starts the same way, peppering the screen with as many notes as they can. Then as they get more used to listening to what they’re doing, they play less and less. Bloom is at its best when it’s populated with a sparse collection of notes and left to run with minimal additions.

RT: I think a lot of our successful apps have been quite passive as experiences, even though the listener is partially involved in the creation of the music. Although interactive media has changed a lot of things for artists, there is still a role for the artist to play in shaping and staging the experience theatrically, even if the experience can be moved through in many ways. Perhaps this is even more artistic than creating a single linear narrative / structure. Somebody once said to me at an event that interactive music is like taking the Mona Lisa and giving the viewer crayons to scribble all over it. I think that displays the trepidation with which some people regard this field. Its clear that sometimes that can be true. However I would argue that truly great interactive music is like transforming the listener into/making the listener feel like Da Vinci. Letting them explore and melt into the artwork itself. This doesn’t necessarily even have to take place explicitly.

SS: This was particularly relevant when creating Carmina Avium. The potential level of control we could have provided users was quite significant, and yet this would possibly have overwhelmed many users. In the end we also provided some premade ambiences as we understood that many people may greatly enjoy an experience but do not necessarily want to go through the process of having to create it. When establishing the variable controls, it was necessary to define the range for each species of bird individually. We provided a user setting to define three levels of how often a bird would speak, but where a blackbird may sound very appropriate calling every second a peacock is going to sound positively painful using the same interval.

The trick was to provide a consistent simple level of control for the users: select the Birds you want to include, set the volume level for each bird and how frequently you want them to speak. Underneath however each bird species was tuned with its own dynamic range and specifically tuned rates or speaking suitable to the particular species. Ultimately it did not even matter if the end results sounded exactly as they would in nature (although that was always achievable) the purpose of the App for most listeners was to create a calming audio environment to listen to while relaxing or to help them focus on work or study. Reality is not always the goal.

DS: When designing ambience for interactive apps – whether it is procedural, generative or purely file based – there is always a problem of repetition. How do you approach this creatively?

YS: I generally try and split up any sort of ambience into a set of component sounds, and approach each one separately. This lets me really concentrate on what each type of sound is bringing to the ambience, and also allows me to design the repetition (or lack thereof). If I am creating ambience from a library of sounds, this will let me adjust how often some of those sounds are played, and I can make a system that will trigger the sounds at different times and loops.

PC: Computer programmers have a tendency to be extremely regimented in their thinking, which makes sense when you realise that the computer is, at all times, doing its level best to find a way to misinterpret the instructions it’s been given. I always enjoy trying to work against this part of my brain, and build in plenty of opportunities for controlled chaos, allowing parameters to vary randomly where possible. Very small variations can be enough to stop music becoming repetitive. In the case of Scape, where all the elements “listen” to each other, and follow rules in response to the actions of the other elements, a subtle change in one element can be enough to trigger an unexpected and complex chain of events.

RT: I think we use a number of approaches.
1. Reactivity. Making the material change and evolve quite a lot based on input from the user or their environment is quite a good technique. This works well if the reactivity is mapped to something which is changing quite a bit. However it can work against you if that sensor doesn’t change much. Sometimes then its good to build automatic change into that algorithm – however then you can confuse the user who might think they caused that change. Its a tricky balance.
2. Evolving algorithms. We try to build in ways of seeding the algorithms differently each time, but also letting them evolve in a slightly different path each time. Again this comes back to defining the boundaries of the ambient space carefully so it still has a definite identity. Its also a difficult balance to make something rich and varied for experienced listeners, but simultaneously a simple thing for new users to understand. Over the years we have learnt (the hard way) that mainstream music listeners need to have one simple reactive / generative idea going on at one time if you want them to understand it. Its fine to have lots of other more subtle things going on, but don’t expect them to understand it (and therefore explicitly value it).
3. Wide variance of source material. Whether it be types of synthesis or samples – the range of initial material really helps define the quality and range of the artistic expression. I personally think a hybrid approach, using many different techniques results in the best quality results at the moment. Having a large amount of material we could play as compressed files within Pure Data helps with this. However we continue to use lots of synthesis and procedural sound generation from scratch and things like reactive granular processing of samples.

SS: I have given many conference presentations, lectures and workshops on the issue of repetition and I feel it is still something we have not entirely overcome in my field of game audio production and can also be an issue with Apps. Just to clarify, absolute repetition (i.e. perfect duplication) of sound is almost completely impossible in nature. Even man made machines that operate cyclically have subtle variation in the sounds produced. This is why repetition in produced media stands out and should be considered undesirable in the extreme.

I use generative audio design processes almost exclusively these days if I am creating anything that is not a linear format. Perhaps we still have a way to go in educating people about how some simple techniques can drastically remove repetition (I’ll just have to keep doing conference presentations). Generative audio using small sound building blocks, staggered layered loops and randomization of various properties within audio design can entirely remove repetition from a project if planned carefully. I continually try to develop new techniques to tackle this issue and delve further down to smaller component levels of generative audio creation to hear what is possible. The first step is to explore the technical capabilities of whatever resources you are using. Once this is established you then have some guidelines in which to work creatively.

DS: Share three ambient sounds you enjoy listening to/experiencing (it could just be text descriptions of something you have experienced)

YS: I love listening to field recordings of animals and nature environments, and for some reason I tend to slip into an ambient listening mode when I hear them. One of my favourites recently is this recording of a Bearded Seal from 1979: http://macaulaylibrary.org/audio/112595
Similarly I love recordings of icy lakes. I’d love to make some myself one day: http://musicfromtheice.blogspot.co.uk/2012/02/sounds-from-yosemites-frozen-lakes.html.
I’m also a big fan of the Shipping Forecast on BBC Radio. It’s not meant to be ambient at all, but it puts me in a trance.

PC: I had a protracted stay in Brighton, on the UK’s South Coast recently, and loved to walk along the seafront. There’s a section where a large number of sailing yachts have been moored, and in a high wind, all the rigging starts to rattle with a pleasant clanging sound. All of the ropes are moving at their own speed, and the resulting sound is a cross between wind chimes and a gamelan. All of the ropes are moving at their own speed, and the resulting sound is a cross between wind chimes and a gamelan. Brian once mentioned that he’d used a similar recording, slowed down, on Ambient 4:On Land. I’m not certain which track, but the background on The Lost Day sounds similar.

My local train station is on a freight route from a seaport, and huge trains of containers that seem to go on forever often pass through slowly, with a wheezing, clanking industrial rhythm. I keep meaning to sample it.

Last but not least, my favourite ambient experience at present has to be Brian’s new album Lux, which I’m listening to as I type. Getting to know Brian as both a friend and collaborator over the last six years has only increased my admiration for his work, and I think Lux is his finest to date. It echoes the best of his early work, and introduces new structural ideas as well as new developments in sound. I was lucky enough to be present in the vast marble gallery at Venaria in Turin as he was assembling the piece, and listening to it takes me straight back there.

RT: I get pretty fascinated with ecosystems of sound. Observing how elements within them react to each other. Listening to the structure of a forests sound ecosystem and the way aspects of it have evolved to occupy different parts of the frequency spectrum and within different timings is fascinating. Especially when you consider the function of the sounds. Birdsong is especially fascinating here. Cities are also amazing like this and large public areas like train stations. Another one is shortwave radio static / messages in various formats.

Three specific ones are :

Symphony of Drone on SoundCloud
An amazing construction site drone symphony created by workers on a nearby site to the RjDj office.

Bizarre squeaky escalators on SoundCloud
This was a totally bizarre set of sounds being generated by an escalator at Stansted airport.

Liverpool Street on SoundCloud
This is actually a field recording of Liverpool St station which I was fascinated with so I made a project (as part of the disquiet junto series) which generated music from it in realtime using Pd frequency and event analysis

SS: The Sound of wind through pine fronds in the snow is incredible. The snow creates almost an entirely silent environment and the wind through the pine fronds is like the world telling you a secret in a whisper (sadly I do not have a good recording of this).

I spent three years living in Japan, by the end of that time I was able to fairly accurately identify where Japanese films were shot if they were set during the summer months. Japan has an incredible range of summer insects that are quite region specific, so the sounds of a particular species of cricket or cicada is like a song sung by the locals of a region. I love the natural soundscapes in Japan and always find them relaxing.

Tokyo summer insects cicadas on SoundCloud
Takao summer insects loop on SoundCloud
(Both recorded during my time in Japan)
There is something incredibly calming about the sound of waves from beneath the water. I need to spend more time exploring the potential for sounds in our oceans.

Underwater waves on SoundCloud
(Recorded with a hydrophone, but this example is created using generative audio from 8 sound-files)