Audio Implementation Greats #1: Audio Toolsets [Part 1]
After steeping in the current console generation, several examples of best practices in audio implementation have been exposed through articles, exposes, and video examples scattered across the internet. In an attempt to overview some of the forward thinking front runners in the burgeoning field of Technical Sound Design, I’ve been pulling together examples of inspirational audio in games as a way to highlight innovative techniques and the process behind them. As a way to kick off the series we’ll begin by focusing a bit on the legacy of available toolsets that have been used to incorporate or implement audio content. Part 1 will tackle audio middleware and in 2 we’ll take a deeper look into proprietary tools. Stick around as we peel back the curtain on the veiled art of audio implementation.
Audio Toolsets Part 1 - Audio Middleware
If there is one thing that currently separates us from our spotted history in game audio, it is the publicly available middleware toolsets that allow us to sound design “in the box” interactivity in a close approximation of how it will sound once it finds it’s way into the game. With the powerful combination of several game and audio engines, this trend towards enabling the Sound Designer to be more involved and in control of game related sound decisions is in stark contrast to the old model of handing sounds off to a programmer for implementation. In an attempt to expose the history of game audio toolsets I’ve rounded up a selection of interfaces I was able to find at large either on the web or embedded in various presentations over the years and link to said documents where applicable. I’m always on the lookout for public screen shots of proprietary toolsets for education purposes in order to illustrate a greater understanding of what has become a critical component to interactive audio.
Microsoft Direct Music Producer
Based on the Microsoft Direct Music/ Direct Sound for PC’s introduced in 1999 version of Direct X, Direct Music Producer enabled functionality within the toolset to create and audition interactive functionality using the features of the low level audio libraries. This included the ability to loop files, specify random or sequential playback behavior, create sample banks (DLS), and specify parameters for interactive playback using MIDI. More info on the specific functionality can be found in a terrific write up by Dave Javelosa over at the IASIG website: DirectMusic Producer for the Masses. While the interface and pipeline of DMP is relatively esoteric and punishing in relation to what is currently available, the tool continues to hold court natively in the Hero Engine from Simutronics where segment (.sgt) files are necessary to enable seamless looping for ambient backgrounds and emitters.
In order to access features of it’s sound cards and the OpenAL audio library, Creative Labs released several tools to assist audio developers in bringing realtime 3d audio to games.
EAGLE: Environmental Audio Graphical Editor
The Eagle toolset, announced in 2000, provided access to environmental modeling in order to simulate Environmental Reverb, Obstruction, and Source Model parameters for use in conjunction with EAX (Environmental Audio Extensions). The toolset provided Sound Designers with the ability to “easily create a variety of different audio data sets, or models. These models include: unique environments which simulate reflection and reverberation properties of a room; source property sets which provide initialization parameters for sounds such as distance attenuation and sound cones; and obstacle behavior models that simulate the effects of sound moving through, and around, doors, boxes, windows or other virtual objects. In addition, EAGLE aids the sound designer in creating these models by providing interactive 3D graphical representations as well as real-time auditioning of multiple data sets.”
The EAX2 toolset allows for the auditioning of environmental Reverb parameters to be used in the programming of presets for EAX used in OpenAL and various middleware solutions such as the FMOD and Miles Sound Systems.
Creative Labs ISACT
One of the tools that most resembles modern middleware toolsets is Creative Labs ISACT or Interactive Spatial Audio Composition Technology. When it was released in 2005, ISACT provided one of the first visually focused toolsets which allowed an audio designer to make decisions about, and audition, the way their designed sounds should be used in conjunction with a 3d audio engine beyond simple sound file playback. In a lot of way it set the standard as an all-in-one environment to author abstract sound with the immediate feedback provided by the tool. ISACT was not freely available, and required some kind of developer affiliation in order to acquire a copy of the tool.
Microsoft XACT & XAudio2
Microsoft has taken great strides to provide audio tools to specifically leverage the features of it’s audio libraries, beginning with Direct Music Producer, and extending to the XBOX and XBOX 360 consoles in the form of XACT. Some of the standout additions to the engine and tool include the ability to author sound propagation curves, global variables including Doppler and speed of sound, realtime parameter control, and Reverb presets. While many of it’s features and functionality have been forward thinking and directly in the service of advanced audio professionals, there’s been a recent backlash in the XNA community due to the difficulty involved with simply “playing a sound file” as opposed to creating an XACT project to perform this simple function. XACT continues to be bundled with the Xbox 360 development SDK and provides a solid foundation for implementing a high quality abstracted audio solution.
MSDN XAXT Overview
The Microsoft Cross-platform Audio Creation Tool
XACT Audio Field Guide
Hands-on Workshop – XACT Audio
GameAudioTools – Scott Selfon
USC Game Audio Course Slides
Firelight Technologies introduced FMOD in 2002 as a crossplatform audio runtime library for playing back sound for video games. Since it’s inception FMOD has branched into a low-level audio engine, an abstracted Event system, and Designer tool that allows access to several features of the runtime audio engine without programmer involvement. Within the FMOD Designer toolset a Sound Designer can define the basic 3D/2D parameters for a sound or event, in addition to the ability to effectively mock up complex parametric relationships between different sounds using intuitive crossfading and the ability to draw in DSP curves to effect the audio. FMOD Designer was among the first fully available toolsets that could be downloaded and used regardless of development affiliation for educational purposes. Additionally, due to their flexible licensing structure, FMOD became a solid and widely adopted audio middleware choice for all levels of game development and continues to be a major player in todays game development.
Introduced in 2006, the Wwise (Wave Works Interactive Sound Engine) and toolset provides access to features of their engine from within the comprehensive content management UI. In addition to an abstracted Event system which has become a sort of standard across many audio solutions, they further enable the ability to make choices regarding additional functionality such as; volume changes, logic, switch/ state changes, attenuation profiles, and in-game connection and profiling utilities. The ability to mock up every aspect of the engines ability brings the Wwise toolset further into a full prototype simulation outside of the game engine. Their commitment to consistently providing new features and improvements, including a manageable upgrade strategy at the low level, adds to their adoption at several publishers and larger developers over the past 4 years.
This article is meant more as a brief history of Audio Middleware, and not a comprehensive investigation of features and functionality. If you’re interested in a deeper look into the differences between toolsets past or present there is a link to a great series of articles from Mix Magazine that covers in depth all of the available audio middleware solutions from 2007.
In addition to this I would recommend each manufacturers website for further information on the current state of development and feature sets. Until next time. Please drop us a line with any feedback or insight!