• Home
  • About
  • Site Policies
  • Contact

Designing Sound

Art and technique of sound design

  • All Posts
  • Featured
  • News
  • Interviews
  • Reviews
  • Tutorials
  • Resources
    • VR Audio Resources
    • Independent SFX Libraries
    • Events Calendar
  • Series Archives
    • Featured Topics
    • Featured Sound Designers
    • Audio Implementation Greats
    • Exclusive Interviews
    • Behind the Art
    • Webinar/Discussion Group Recordings
    • Sunday Sound Thought
    • The Sound Design Challenge

What’s The Deal With Procedural Game Audio?

October 31, 2014 by Varun Nair

Guest contribution by Martin Roth

data
Image Credit

We’ve all heard of the promises of procedural game audio. A veritable Valhalla where sounds are created out of thin air, driven by the game engine, eliminating the need for huge sample libraries and tedious recording. Sounds great! So why aren’t we hearing more of it in games today? We’ve all experienced Rockstar’s work in GTA 5; those bicycles sure do sound great! Some indy games such as Fract or Pugs luv Beats have dabbled. But it seems that if procedural audio were all that it promised, it would be much more common. What’s the deal?

The hard truth is that while the idea is great in theory, no one knows what they’re doing in practice. The field is lacking in design principles, tools, and technical performance. This is especially true considering the end-to-end workflow. On one end, high-level tools are needed to give designers the flexibility to explore sound and its interactions. On the other, low-level tools are needed to make those creations available where they’re needed, be that on the desktop, mobile, console, embedded systems, web, or anywhere else. The end-to-end workflow is key to the adoption of procedural audio.

For the purposes of this article the terms procedural, generative, and interactive as they relate to sound and composition will be used interchangeably. Their distinction is important, but we’ll leave that for another article.

Scarce Design Resources

The field suffers from a lack of resources to learn how to make procedural audio, including standards for judging its merits. Undoubtedly the best learning resource is Andy Farnell’s book Designing Sound. The presentation focuses on design from first principles, but may leave those without a technical background struggling to understand the reasoning (but don’t let that stop you from reading it!). The book is written for clarity, not for absolute performance or maximum sound quality. Resources are otherwise scattered, usually compensated for by personal interest or continued education specifically on the topic.

Tools, Well Almost

Undoubtedly there many excellent tools available to design sounds, especially musical ones. A near fifty year history of electronic music has created a wealth of knowledge, best-practices, and interfaces for exploring sound. But here the end-to-end argument is critical. Unless the designer can run the sounds on the target platform, the tools are not helpful except as a part of the creative process.

In order to satisfy this requirement, the available tools are generally limited to any number of audio programming languages (or even general purpose programming languages). There include Pure Data, Max/MSP, SuperCollider, Csound, Chuck, C/C++, the list goes on. Many of these have robust and knowledgable communities supporting them. All of these tools allow the user to “do stuff” with sound, but how well they meet the needs of sound designers is debatable. Many would say that the learning curve is far too steep. The target audience for these tools has typically been those more interested in experimental work.

This leaves us in the difficult situation where the ideal solution is fragmented between tools that satisfy the high-level design requirements and those that satisfy the low-level technical requirements.

Low-Level Really Is Low

But let’s suppose that you’re able to design a sound that you’re excited about, with the interactions that you want, and the tools available. Now you need to embed that asset in some environment where it will run. Pd’s BSD license makes it very flexible. Max/MSP’s gen~ code export is a great step in the right direction, while Supercollider is open source under the GPL making it practically incompatible with almost all interesting platforms. Even if you manage to embed your work, chances are that performance will be poor.

As sound designers we dream of bigger technical budgets, but in the meantime we may be better served by making our assets as efficient as possible. We want to use a really high level polished tool like NI’s Massive to design sounds, the interactive possibilities offered by Max/MSP, but need the efficiency of an optimised native implementation. No toolchain comes close.

The Only Way Is Up

So where does that leave us? A dearth of design resources, incomplete tools, and poor performance. That’s why procedural audio hasn’t taken off. How can we improve the situation? Where do we see ourselves as sound designers, not only in games, but for any application, in the next few years?

Design Principles For Procedural Audio

Starting from the beginning, clear design principles are needed. Whereas sound design is typically only judged on its aesthetic merits, procedural sound must also be judged on its technical merits. An incredible sound isn’t useful if it uses up all of the CPU budget. A lightweight process that sounds bad isn’t helpful either.

From the aesthetic perspective, a lot can be learned from the world of music synthesis. There are many resources available teaching how to make certain kinds of sounds with various synthesizers, including large communities of musicians. These are typically musical sounds like bass drums, the Hoover, or vocal pads, and not the kinds of sounds that games would be interested in, like gunfire, footsteps, or environmental effects. But the aesthetic principles are the same and skills in one practice translate easily to the other.

Of course musicians often have dedicated hardware for their sounds in the form of synthesizers. Game sound designers do not. Unfortunately the knowledge necessary to generate sounds in a lightweight manner is typically reserved for those with a lot of technical knowledge about computer architecture and digital signal processing. Those techniques need to be either taught, or to simplified and made accessible and usable to a wider audience (preferably both).

For these two merits to come together, community resources such as books, forums, and showcases are needed. Open source sound is the order of the day. Forums are needed for practitioners to not only ask questions, but also to showcase their work. The questions and presented sounds can be collected into a ongoing list of best practices. Designers such as Anton Woldhek and Graham Gatheral have already taken it upon themselves to post their own work, but such examples are rare and hard to find. It’s also worth mentioning that many impressive techniques exist in academia, but they rarely find their way into practice because there isn’t a professional community to receive them or to translate them.

New Challenges New Tools

New tools must be written to specifically address the needs of creating interactive audio. To be clear, it isn’t just about putting synthesizers in video games. It’s a shift in thinking about audio and audio production from linear to non-linear. All musicians and sound designers are familiar with traditional linear-time DAWs. The tools that we need must create the bridge from the linear DAWs, not in the direction of performance such as Ableton Live, but in the direction of design without a timeline.

It is important for these new DAWs to make traditional sound design skills portable. This means being able to work at a very high level, with a great deal of visual feedback (e.g. spectrograms, per-connection preview, frequency responses for filters, etc.). All underlying complexity must be abstracted, one detail which most existing audio programming language IDEs ignore.

88 Miles Per Hour

And finally the DAWs cannot constrain the designer to the desktop environment. The designs must be exportable to any interesting platform. This includes embedded systems, desktop, mobile, or even the web, in the form of libraries, plugins, standalone executables, and applications. The key requirement is performance; flops per sample.

Reminiscing About The Future

Technology always seems to travel in circles, or spirals at least. The opportunities and challenges presented by procedural audio today seem to offer parallels to the development of modules and trackers for computer game music in the late-1980s. At the time it simply wasn’t possible to deliver hundred of megabytes of compressed audio, so music and effects had to be created locally. And today, all of the recorded audio in the world can’t make a world sound alive. Could it be that “in the future” we won’t be downloading raw music data off of the internet at all? Perhaps we will go back to the future when modules reigned supreme; the data is the program. Sound won’t just be replayed from bulk data, it will be executed as a program.

Martin Roth is co-founder of London-based Enzien Audio. Enzien are the creators of the Heavy audio compiler, a tool focused on solving the performance problem of procedural and interactive audio. He has previously served as CTO of RjDj, software engineer at Google, and researcher at Deutsche Telekom Laboratories. He earned his PhD in electrical engineering from Cornell University. Find him online at @enzienaudio and @supersg559.

Filed Under: featured, guest post Tagged With: game audio, martin roth, procedural audio, programming, sound design, synthesis 2014

Comments

  1. Jack Menhorn says

    November 1, 2014 at 4:22 pm

    I think we missed an opportunity here by not having a picture of Jerry Seinfeld as the top image.

    • Dave DeLizza says

      November 5, 2014 at 12:54 am

      is it….is it too late?

  2. Martin Roth says

    November 1, 2014 at 8:39 pm

    I added some comments about what one might expect from a procedural audio forum.

  3. Lucas Mengual says

    November 7, 2014 at 10:56 am

    Totally Agree!! Having issues exporting. Really looking forward for Nov. 27th

  4. Peter Todd says

    March 19, 2015 at 5:47 pm

    Rockstar developed some pretty interesting tools for audio in GTAV
    http://www.gdcvault.com/play/1020587/The-Sound-of-Grand-Theft

Trackbacks

  1. S.M.I.G. – Week 3 – Procedural Sound Design – ciaran devlin-rushe says:
    October 17, 2016 at 12:48 pm

    […] Nair, V. 2014. What’s The Deal With Procedural Game Audio? [Online]. [Accessed 9 October 2016].  Available from: https://designingsound.org/2014/10/whats-the-deal-with-procedural-game-audio/ […]

  2. ciaran devlin-rushe says:
    February 14, 2017 at 10:26 am

    […] V. (2014). What’s The Deal With Procedural Game Audio? [Online].  Available from: https://designingsound.org/2014/10/whats-the-deal-with-procedural-game-audio/  [Accessed 9 October […]

  3. Procedural Synthesis in Video Games – Rory Hay says:
    October 26, 2017 at 11:36 am

    […] Nair, V. (2014). What’s The Deal With Procedural Game Audio?.Available: https://designingsound.org/2014/10/whats-the-deal-with-procedural-game-audio/ […]

Social Media

  • Twitter
  • YouTube

Posts By Month

#sounddesign on Twitter

#sounddesign

tunepocketTunePocket Music@tunepocket·
26 May 2020

Download royalty free gunshot sound effects, incl. one shot gun sounds, gun cocking sounds, gunshot with silencer, machine guns, and more #gunshot #sfx #sounddesign #videoediting #gamedev #royaltyfree https://t.co/bTTDQkveKF

Reply on Twitter 1265206307927162880Retweet on Twitter 1265206307927162880Like on Twitter 1265206307927162880Twitter 1265206307927162880
Reel2MReel2Media@Reel2M·
26 May 2020

At Reel2Media we are harnessing the power of smart speaker technology, to deliver cutting edge skills that solve problems and add value. To find out more, click here https://t.co/VAAw7vTW1V #sounddesign #smartspeaker #alexaskill #amazonalexa

Reply on Twitter 1265206269255847936Retweet on Twitter 1265206269255847936Like on Twitter 1265206269255847936Twitter 1265206269255847936
SoniKSoundLibSoniK Sound Library@SoniKSoundLib·
26 May 2020

Listen to #soundcloud preview of our next release! #np https://t.co/S3bxxrnqrh #audiopost #soundlibrary #soundediting #sounddesign #ambisonics #spatial #surround #ambience #roomtones

Reply on Twitter 1265204382779559936Retweet on Twitter 12652043827795599361Like on Twitter 12652043827795599361Twitter 1265204382779559936
asoundeffectA Sound Effect@asoundeffect·
26 May 2020

Want to know what the independent sound effects community has been up to? Hear the very latest SFX libraries here: https://t.co/OUBM2SqMoP #sounddesign #soundeffects #soundlibraries #indieSFX #gameaudio #filmsound #filmmaking #gamedev #indiedev #indiefilm

Reply on Twitter 1265199255838511105Retweet on Twitter 1265199255838511105Like on Twitter 1265199255838511105Twitter 1265199255838511105
OblikLinesOblik Lines@OblikLines·
26 May 2020

Working on new tracks with many instances of Misty Valley ! ⁠
Misty Valley is free for all NI Reaktor users⁠
You can get it here: https://t.co/bb53Gtkw5e

#reaktor6 #reaktor #nativeinstruments #synths #vst #sounddesign #virtualinstrument #musicproducer #obliklines #mistyvalley

Reply on Twitter 1265198647307907073Retweet on Twitter 1265198647307907073Like on Twitter 1265198647307907073Twitter 1265198647307907073
Load More...

Copyright Info

All content on Designing Sound is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Copyright © 2023 · Magazine Pro Theme on Genesis Framework · WordPress · Log in