[This is a first of a series of interviews/articles on procedural/generative sound]
‘Pugs Luv Beats‘ is a hilarious music composition game for iOS devices developed by Edinburgh based studio Lucky Frame. It’s about guiding pugs (in costumes) around a galaxy of worlds, whilst creating an endless variety of music. It sounds fantastic and runs on a generative sound/music engine developed in Pure Data.
Lucky Frame is Yann Seznec (artist, musician and sound designer), Jonathan Brodsky (artist, designer, musician, coder) and Sean McIlroy (illustrator and print maker). Jon and Yann were kind enough to make some time right after the release of the game to talk about the sounds and technology behind Pugs Luv Beats.
DS: How did Pugs Luv Beats come together?
Yann: Jon and I made an app called Mujik a couple of years ago. A lot of people downloaded it and there were a lot of good reviews. It was basically a different approach to music on a mobile interface. After playing around with that for a while we started thinking about how much further we could take the idea and Jon started getting into the idea of making games. So, we started thinking about how we could really bridge that gap between music and games. What if you could use a game interface to create music rather than to play music that is already there? That was the starting point. Jon made a demo which we dubbed ‘Space Hero’. The idea was that you were controlling a little ship that was shooting enemies. As the enemies came on screen they made a sound and as you destroyed them they made a sound, with the twist being you could edit how the enemies came after you so it was like a piano roll hybrid drum sequencer. It was more of a proof-of-concept than anything else. We took that to Channel 4 and to make a very long story short they ended up eventually telling us that they liked the idea and that they wanted to invest in it. Interestingly they told us, ‘We want to invest in the concept but don’t make that game’ [laughs]. So we started making various different prototypes for what became Pugs Luv Beats.
[youtube]http://www.youtube.com/watch?v=SkU8RLf53G8&feature=context&context=C39ca8a1ADOEgsToPDskKwCNOrWBVnsX5G2JRHjkbH[/youtube]
DS: What is Pug Luv Beats about?
Yann: We were just prototyping with lines and dots on the screen and I think we went through two full prototypes. We ended up with a little bug and you had to tell the bug where to go, and once it did it would create a sound and then you would have a second bug and you can tell that bug where to go and it would instantly start creating these interesting rhythmic loops. That was the core of the game. Once we had a fun-to-play instrument, we tried to figure out how to make a game out of it – which was quite a challenge. Eventually the bugs became pugs and we designed this whole crazy world around them and decided the sounds that they made would be dependent on what terrain they were on.
We then made a bunch of different terrains and terrain sound banks and each of them react slightly differently. We then had the idea of the pugs being slower on certain kind of terrains, and you can speed them up with costumes – if you want. If you don’t, then it means that they are slow. If they have a costume its going to be 25-50% faster which is really interesting because something that has a slow melody and rhythm will speed up.
The way we have it set up with ratios is quite cool because there are eight terrains and each of them has a slightly different ratio of ‘slower-ness’, which means you can get a ratio of 66% between two of the same distance and if it’s 4/3 or 5/4 or 7/4 you can get all these interesting rhythmic ratios going. The other level that we’ve added on top of it is that where the pugs are hitting are given a value. So as the pugs are running around generating all this music, they are raising your in-game currency and allowing you to explore other places and find new terrains and new costumes and new planets. I’ve just described the full game right there – Pugs Luv Beats.
[youtube]http://www.youtube.com/watch?v=V0i18_–8Yc&context=C39ca8a1ADOEgsToPDskKwCNOrWBVnsX5G2JRHjkbH[/youtube]
DS: So it’s a generative music game which was designed in Pure Data? Why Pure Data?
Yann: That was a decision we made at the end of the prototype phase, it was almost too late.
Jon: LibPD (embeddable PD core) came out some time in 2010 and PD has been around for god knows how long. The embeddable core is so great for us.
Yann: It’s really amazing because before that we didn’t really have a proper way of doing sound at all.
Jon: I was writing the entire DSP engine and right before we got in to LibPD I had a chain-able DSP system that was defined through code. But, it wasn’t twenty years of audio research [laughs] that we could just drop in to our program.
Yann: I know very little code. Whereas, my training, if I can call it that, is in graphical programming with software like Max. I felt our time wasn’t being used effectively. As soon as we were able to integrate LibPD in to the whole system, it made our sound development strategy thousands and thousands of times better. Pure Data, being a brilliant and occasionally frustrating thing, was overall really good. The integration through LibPd and in to Objective C is surprisingly painless once its all setup.
Jon: A lot of people have done a lot of good work. The toolkit that we are using for running the game is called openFrameworks, which like Max or PD came out of the art and code movement. And a guy made a wrapper that takes the nice parts of LibPD and none of the nitty gritty things and it was probably less than a day’s worth of work to drop PD in to the game engine I had been building for a few months. LibPD itself was made by Peter Brinkmann with help from Peter Kirn. We have a lot to thank them both for.
DS: How is PD integrated with the game engine?
Jon: I basically send messages, just like how you would send messages in PD or Max/MSP between your different sub patches. I send those same messages from my game engine about what’s happening in the world in to LibPD. I construct lists that say, “There’s a bug on this tile, he’s picking up a beet and the BPM is whatever and the tile is this type”. There’s really not much integration, I can just tell PD what’s happening in the game at any point in time and then PD/sound designer deals with it.
Yann: Yeah, it literally was me making Pure Data patches as if I was making Pure Data patches for anything. The only difference being I have a little receive box in there that Jon is making the sends for and it outputs a giant list.
Jon: There’s one list for sound effects, there’s one for what happens in the world and there is one when the world is setup – I use a random number generator to generate the world.
Yann: It’s really great. It’s an amazing system. I get these three lists. The last list Jon described is the first thing that happens.
DS: So the world creation is completely random?
Jon: The nice thing about random numbers on computers is that they are totally predictable. The way random number generators work is that you give them a seed value and then every sequential random number is based on that seed value, so I don’t store the worlds, I just store seed values. When you go back to the world, I see the random number generator and regenerate the whole thing. That is also how we communicate about what world is on screen – just through those seed values.
Yann: We never even see that number.
DS: So it is a sort of identifier for the world?
Yann: That is exactly what it is. I use this number to do things, like pick a sound library for the beats.
DS: How many libraries does the game use?
Yann: At the moment there are only five. We may add some later on.
Jon: But that’s partially true. Only half of the sound engine is the beat sounds.
Yann: So beats are what happens when a Pug hits a beet [laughs]
Jon: The spelling of beats varies between vegetable beets and musical beats throughout the code. Sometimes the Pugs are still called rats for historical reasons [laughs]
Yann: Going through the code is hilarious! There are 5 beat libraries, which are each comprised of: a kick equivalent, a snare/clap equivalent and sound for when the beet is harvested. Otherwise there are libraries for each terrain as well. When it chooses a random number, it uses that random number to select which beat library it will use. It also sets a couple of other things – like there’s a little synthesiser you can play along with the planet. It will choose an attack and a decay on that synthesiser as well, that way each planet will have a lightly different sounding synthesiser. Another important thing that it does is it picks a scale or a mode (depending on how you want to think of it) and it applies that scale or mode to all the sound libraries and the synthesiser as well (the synthesiser is a semi-tuned synth). That way whenever you got to a planet it will be one of eight scales/modes. The way I’ve done that is for each library I have made a thirteen file sound-set for a chromatic scale, but that random number will select which look-up table it’s using and will select only eight of those thirteen. It does that across all the sound libraries. Another part of that list is the BPM. Each planet sounds quite different, even on a global level – in terms of being slower/faster, minor, mixolydian, major, etc.
DS: Is the BPM random?
Yann: The BPM for each planet is random, but will be the same when you return to that planet each time. Jon generates it from the random number, but sends it to me separately so its easier. I use that BPM not to trigger the sounds, because I’m getting the triggers from the code, but to calculate the delay times so that they are synced. I’ve put a soft delay on virtually everything to give a ‘rounder’ feeling. This whole process happens in a tenth of a second. Once you are on a planet, you have all these pugs running around. If you set a node and the pug starts running out, every time the pug jumps on to a new square I will get a list of numbers from Jon’s code. The list will be an identifier for the pug, what terrain he is on, whether he is wearing a hat, whether he is wearing a skin, whether the node he is jumping on is a node you placed or if he is just jumping on it to get to the node (I use that to make two different sounds).
Jon: It easier to think of the world as a heightmap rather than separate pieces of terrain. So there is actually an even more specific number. You can say, “this is sand, but also sand 0.5”, you can get a number between 0 and 1.
Yann: In practice that number is giving us a separate value for each terrain tile, which gives us a kind of location. If we have a specific number for each tile, every time a pug goes to that tile it plays the same sound.
Jon: It makes it like an instrument you can discover. It’s like a piano where you don’t know what note each key is going to play, but if you go back to that key it will play the same note.
DS: Each terrain has a different sound bank then?
Yann: Yes. I tried to match a vague conceptual thing to each sound bank. For example, the mountain is a kind of far off sounding synthesiser. The desert is an Iranian dulcimer called the Santoor, which is quite cool. For water I have an African harp. The links between the actual sounds and the terrain are somewhat tenuous but I was prioritising on making everything sound good. It can get quite chaotic when you randomly get a planet that has got five terrains – but that’s part of the fun.
DS: Were there many technical limitations in creating sound for such a game?
Yann: It limited me in the sense that I wasn’t expecting it. I was very used to patching on computers, so I had loads of buffers and then at one point Jon asked me, ‘How many buffers are you running?’. And, I was like, “There’s eight instances of that and there’s four buffers in each, so there’s thirty-two”, and he was like, “YOU HAVE THIRTY-TWO BUFFERS?!”, and that was just in one of the sections [laughs]. In PD doing polyphony is much more round about than in Max, so I did have to learn that kind of stuff. Artistically I didn’t feel very limited by it. What was really fun about setting up this patch was trying to make it as flexible as possible so that I could develop it further down the line. One of the updates for example might have pickups that change how the audio is played, so you can have the sound in reverse for or you could have the sound engine going all wonky at some point or the other.
DS: Considering you can do anything you want, do you set up artistic limitations for yourself?
Yann: It’s really hard. I think for us the biggest advantage in forcing us to be limited is we have a real goal in mind – a released commercial piece of mobile gaming software. That’s a real interesting framework to try to place around everything. You could use the same data from this game and each formant of a sound can be applied to each pug and you can do crazy harmonic re-synthesis stuff. It would be awesome! Very cool! Who would want to play that? [laughs]
Jon: I think to a certain extent you are limited by time and complexity. There are a lot of defaults happening. The art style is very much like what’s in Sean’s sketchbooks. The African harp and Iranian dulcimer, that’s the most default Yann sample-set [laughs].
DS: What about the choice of sounds, considering it’s for iOS devices?
Yann: We made two choices very early on from a sound design perspective. One being we didn’t want to rely on people wearing headphones. Whilst I of course wish that everybody played all music games with headphones, I think it is unreasonable.
Jon: Or more preferably with studio monitors [laughs]
Yann: Yeah! “Please use medium field speakers with proper sound insulation” [laughs]. We decided very early on to make it sound awesome through headphones and awesome through the built in iphone speakers. It is a challenge. What it means is that you have to have mid-range in all your sounds and you need to make that mid-range not sound annoying when you listen on headphones. Stuff like the santoor cuts right through. And when you are dealing with generative sound like this you have to be careful about volumes. We haven’t put a compressor on anything, I haven’t really felt the need for one yet. The big sounds like bass drums were quite tricky, but it’s about putting a peak at the mid/high range to make stuff cut through.
DS: Considering most of the sounds occupy the mid-range, did you also try to create contrast between them?
Yann: The beats, the vegetable beets that create beats are the main percussive element. Otherwise I wanted to have a bit of a range. There’s a kind of a distortion guitar thing, there’s a sine wave synthesiser, there’s the santoor, there’s a xylophone/toy piano.. they need to be cohesive but also different enough so that when you land on new terrain it is really clear that a new sound is happening. It was quite challenging to make it sound like you were in the same game still. Also, everything runs at 22k but it does sound good and it’s funny how you much can get away with at 22k. It’s still in stereo.
DS: You mentioned about updates to the game. Anything around the corner?
Yann: Yes indeed! We are currently working on our next release, which is going to be a free app that is just the synthesizer part of Pugs Luv Beats, developed slightly to allow you to dress up the pugs in outfits and hats to create different synth voices. People really love playing with the singing pugs so we figured it would be a fun spinoff. That will be released in a couple of weeks.
Shortly afterwards we’re also going to release another update to Pugs Luv Beats to tweak a few things like the in-game beat currency. We’ve gotten some really great feedback on some small changes to make, so we’re going to implement a bunch of those.
After that we’re going to make a different music game entirely, but that’s all still hush hush!
DS: Pugs Luv Beats just got nominated for an IGF award for ‘Excellence in Sound’. Excited?
Yann: We are super excited! It’s absolutely amazing news for us. The other nominees are of such a high quality, it’s incredible to be associated with them. It’s particularly great to be recognised for the audio, since it is a music game and all.
http://luckyframe.co.uk
Twitter: @lucky_frame
The Amazing Rolo
More information on Pugs Luv Beats and LibPD integration on the CDM blog
[…] The sound of Pugs Luv Beats! […]