“As has come before; many of these posts will be philosophical in nature. Some will be in contradiction to previous postings. These are not intended as truths or assertions, they’re merely thoughts…ideas. Think of this as stream of consciousness over a wide span…Please bare with us as we traverse the abstract canals of audio musings.” -Designing Sound
With the advent of new technologies and new platforms that are being rapidly developed, our options for entertainment seem to be increasing exponentially. In my lifetime, I have seen the birth of smartphones as mobile gaming platforms, the explosion of consoles as high-end performance platforms, the resurgence of VR, AR and MR, and the commercialization of AI, including voice command tools and physical robots. And to be honest, I’ve probably still missed a few. But how do these new platforms affect us as audio professionals? How do we approach our creative design process moving forward into the Digital Age?
I work for a company called Anki, a robotics and technology company dedicated to making artificial intelligence accessible for the everyday consumer. Over the past three years, we’ve released three products: Cozmo, OverDrive, and OverDrive: Fast & Furious edition. A casual glance at Anki’s website will reveal the usual signs of a video game: explosions, cool cars, funny characters, you name it. But there’s something more to these toys…most notably that they aren’t your traditional “video games” at all. They are in fact: robots.
What does it mean to create sound for robots?
In order to answer that question, I believe we need to first start in a place we know very well: linear entertainment. How does music and sound impact film, television, and video games? In a study completed in 2015, researcher Nanette Nielsen at the University of Oslo, Norway(1), determined that music and sound, “…enabled the players to become more absorbed in the action,” and ultimately “…played a part in how they [audiences] understood their own role in the drama.” For most of us, these are not shocking results. We understand almost implicitly that our job as audio creatives is to help build immersion, drive emotion, and ultimately, tell stories.
Okay, so why start here?
As new technology emerges, the concept of “how things are done” needs to emerge with it. How do you create interesting and immersive audio for something which never existed before? And who will pioneer these new ideas?
Looking around my audio team, each one of us has a similar history. We worked at companies like Sony or Microsoft, honing our skills as sound designers, implementers, and workflow builders. Or we worked at music studios, recording live string quartets or rock banks. But as our generation moved into the Digital Age, we started seeing new platforms emerge; a place where nothing was defined and anything was possible. And that was intriguing to us. So, we decided to make a change – a leap of faith even – into something new.
As we move into the future, each of us will have a choice. The Digital Age is here, and everything will be affected. The question is, how will we respond? For those of us specializing in film, how will our workflow or creative choices change by new and more immersive platforms such as DTS:X and Auro3D? Do we take the plunge into VR? How could the mobile gaming industry be revolutionized by AR? And what would it feel like if every home had their own personal assistance robot?
I believe that our experience with and love for linear media combined with the endless potential that comes with this new technology means that we will use what we know, combine it with what we believe the technology can give us, and dream up something that’s even more exciting, more immersive, and more satisfying than ever before.
(1) University of Oslo. “Film music captivates us.” ScienceDaily. ScienceDaily, 7 May 2015. <www.sciencedaily.com/releases/2015/05/150507082439.htm>.
Very nice “sound thought” Ashley. I think the world of sound design, especially for the media you reference, is becoming more like interaction design, just with the intentions of using sound, and perhaps even music, instead of purely visual media.
Ultimately I think this means the sound designer will inevitably become the coder/programmer/scripter of the design interaction, at least partially. I don’t think this will mean the skills developed from linear media will be irrelevant but will evolve and adapt to fit the needs of the interaction design.
I know many sound designers who have resisted learning how to code or script because of the time investment and also the difficultly it presents, but as designers we at least need a common technical literacy with the media we are designing for, which for many this means code. Moreover, we have a tremendous creative opportunity as designers to impact the overall experience of the end user.