Sulpha: The New PS4 Mastering Suite

Sony for a very long time has spearheaded the effort to standardise loudness in games. The recent PS4 SDK update (2.500) includes a mastering suite — Sulpha analysis tools — to help developers master their titles for a variety of playback systems, from full range surround sound systems to TVs, to mobile devices.

It features a 4-band equaliser, a 3-band dynamics processor, gain and limiter controls and loudness management and analysis tools. The interesting thing about the toolset is that it utilises resources from the operating system and is therefore compatible with game audio middleware, third party engines and all PS4 titles.

I briefly interviewed Garry Taylor, Audio Director at Creative Services Group, Sony Worldwide Studios and Marina Villanueva-Barreiro who is a senior engineer at SCEE Research and Development. I found it interesting that about 50% of users listen to PlayStation titles through their TV speakers. I was expecting the percentage to be much higher.

DS: Sony in many ways has been spearheading the loudness standard for games. Did the development of these tools seem like a natural progression from the development of the standard?

Garry: Very much so.  Having a loudness standard is all well and good, but we needed to make it as easy as possible for developers to hit the PS4 loudness target without having to spend big money on new equipment.  Loudness metering has been part of the PS4 operating system for a couple of revisions now, and this is the next logical step, allowing developers to manipulate overall EQ , dynamic range and loudness easily and quickly.  Smaller teams working on PlayStation titles may not have the resources or technical knowledge required to conform to a standard, so having one easy-to-use audio mastering tool that works on every single title made a lot of sense.

DS: How do the tools differ, both in performance and workflow, from those available in audio middleware?

Marina: The tool has several benefits over other middleware solutions.  In terms of workflow, it allows mastering of the final audio mix to happen in the low level audio system, after all the main game ports and the background music mixing stage, which the game or middleware don’t have access to. This allows the audio design and integration teams to work in parallel and seamlessly make updates to the mastering configuration. The Sulpha tool (PS4’s audio analysis and mastering tool) allows real-time auditioning on the PS4 target and the exporting of configuration presets that the integration team can just replace in the game without any impact to resources. This can be used independently of the game’s audio engine, providing a common solution for all PS4 developers, regardless of the audio engine selected for a certain project.

Garry: It will also run on a ‘package’ build, so you don’t need to be running all the game-side tools in order to work.  You can effectively take a near-complete game as a package and modify EQ, dynamic range and overall gain to suit.  This would also allow developers, if they wished, to take their game to a 3^rd party for mastering, without a mastering engineer having to learn their in-house tools.

DS: What sort of developer feedback influenced the design of the tools?

Marina: It’s very important for us to understand the problems game audio developers experience so that we can design tools that provide the means to achieve the best audio quality on PS4. The audio mastering suite has been the result of multiple discussions with audio teams and technical experts in the games industry, about potential ways in which an audio mastering process could help developers achieve a professional result on all types of playback systems, in a similar way to how it’s done in other industries. The solution is the result of a great collaboration across multiple teams at Sony including SCEI, SCEE Research and Development, the Advanced Technology Group, the Sony Worldwide Studios (WWS) Audio Standards Working Group, and the game audio teams across WWS.

Garry: We consulted and requested feedback from all WWS developers, as part of the development of the tool.  The feedback we received indicated that it had to be easy to use and instantly familiar to any audio professional, whether they’re a game developer or not.  It had to have as little impact on titles as possible in terms of performance and workflow, and lastly, as the various developers working on PS4 use a wide range of audio systems, it had to work with all titles regardless of the underlying technology.

DS: In your experience, which is a preferable approach to mastering the mix — master as you mix or treat it as a separate process before the release of the game (similar to the music industry)?

Marina: I believe audio mastering in games should be seen as a separate process, similarly to how it’s done in other industries. This gives audio designers the artistic freedom and flexibility to create the most amazing audio mixes without considering the restrictions imposed by the hardware or listening environment.

Garry: We wanted to introduce the concept of mastering as a separate process to the mix; the final process in the chain.  Often, the teams creating content, who may have been working on a title for years, sometimes find themselves too close to a project to make an objective assessment of their work.  That, at least, is my experience of game audio development over the 20 or so years I’ve been making games.  In my opinion, mastering should be about bringing fresh ears to a project.

DS: With PSN increasingly supporting indie teams who might not have a full time sound designer, do the tools allow for enough flexibility?

Marina: Yes, the tool has been released with a comprehensive set of presets to facilitate audio mastering activities for those indie teams with limited resources or expertise. The system presets provide an simple entry point for unexperienced teams and we will be providing additional documentation to help them master their games more efficiently and achieve a better audio quality.

Garry: For a team without dedicated audio staff, all the developer would need to do is recall a preset, measure the loudness, and make the appropriate gain change to ensure the loudness of the game hits the PS4 recommended target of -24 LKFS over a period no shorter than 30 minutes.  Obviously in-house audio teams, or the audio person responsible for a title might want to create their own presets that could be recalled as and when required.  These presets can be recalled at any time during a game.

DS: By utilising “operating system resources”, does this mean the tools run on the system kernel and shares resources at that level? 

Marina: The solution consists of two main parts, a system library performing the runtime mastering and a tool to allow audio design teams to create configurations for it. The Sulpha tool is part of the SDK and runs on a PC, connecting to the PS4 devkit in real time to configure and audition the presets. Those presets can be exported and are used by the game to configure the new audio mastering library, which is part of the low level system kernel and hence independent from the game resources.

DS: Would the tools have any impact on the performance of games with large, processor-hungry audio systems?

Marina: The solution has no impact on games’ performance at runtime, and is independent of their audio engine. Once the library is initialised and configured, the processing is done entirely on the system allocated resources.  The library provides a set of professional mastering DSPs that can be used without having to compromise on CPU resources.

DS: Were the tools developed with any existing user listening metrics in mind? 

Garry: The research we’ve done over the years indicates that around 50% of users listen to PlayStation titles through their TV speakers.  However, most newer TVs can’t really be called ‘an optimal listening experience’ in terms of frequency response, due to the small size of speakers in modern TVs.  We felt that mixing for TV speakers unfairly penalises those who have chosen to invest in a decent sound system.  Therefore, we needed to allow developers to mix for the top end, in terms of playback systems, but also, through mastering, provide options for players with less than ideal systems, or situations where players needed limited dynamic range. We believe the PS4 mastering suite solves a problem all audio developers have experienced at some point.  Being asked to modify a mix based on feedback from a team member listening on a low-end system can be hugely frustrating for those of us concerned with audio fidelity, although we fully understand that there will be many people listening to our titles on that kind of system.  The suite allows us to cater for those smaller systems, whilst still retaining the dynamic range and fidelity we would expect as audio professionals.