Picture by mtmmonline
Technical and artistic achievements in entertainment are becoming greater and greater every day but not everyone is able to take full advantage of it. According to the WHO, in 2014, 285 million people are visually-impaired worldwide and, as for other disabilities, it is extremely important, from a human standpoint, to take advantages of the ever-growing technology to help our beloved entertainment media to reach those people (and vice versa).
The purpose of this article is to give an overview of how audio can help the visually-impaired people enjoying movies and video games. It is obviously far from being exhaustive so if you ever encounter a specific sonic idea or technic used in a movie, a video game or something else, please feel free to mention it in the comment section.
Picture by robertlischka
A few years ago, movie theaters began converting from film projectors to digital, each with a server that processes and distributes the movie data, including audio description tracks. Each month, more and more movie theaters install one of these systems, making the audio description available to people who need it. With digital, everything (movie, multiple sound tracks, and captioning) are all delivered in something called the Digital Cinema Package (DCP), which any company can access through common standards, so the competition for equipment is increasing and the cost of installation is decreasing. One DCP can server multiple screens in a movie house. While the DCP is currently a physical product (a hard-drive), eventually the data will be delivered to movie theaters via satellite.
(http://www.acb.org/adp/movies.html)
To help me with this article, I had the pleasure of interviewing Joel Snyder, author of The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description and President of Audio Description Associates.
Audio Description is the process used to help visually-impaired people to enjoy movies, theater or other types of live events (opera, dance, …) thanks to a narrator describing, between two batches of dialog, the current action and layout of the stage or setting.
Joel started working in this field in 1981 when he participated at “the world’s first ongoing audio description service in Washington, DC”, a project directed by The Metropolitan Washington Ear, a radio reading service in the DC area, along with Washington’s Arena Stage. Since then he never stopped witnessing and participating to the various stages of the movie industry, an ever-changing industry that is now bigger than ever. When asked about the current state of movies in relation to Audio Description, he mentioned that “As of August 2016, it seems that a majority of film releases now have an AD track that accompanies a film for its theater screenings. In addition, development is proceeding apace on several mechanisms that would allow folks to hear the description track via their own smartphones, via advance downloads and use of an app that syncs the description with the film or simply taps into the transmission in a theater.”.
Sadly, these great technological advances in movie theaters are shadowed by their DVD counterparts who are, (too) often, sold without the Audio Description tracks played by the aforementioned theaters. As Joel puts it, “[In the US,] only a relatively small proportion of those AD tracks, available in a movie theater, find their way to the subsequent DVD release! And if it does, it’s often not included in all versions of the DVD. The U.K., a country with a population that’s about 1/6 the U.S., has *many* more DVDs with description.”. Furthermore, from a localization point of view, an film with an English AD track on DVD doesn’t necessary means that this track will be translated and available in a localized languages, which reduces even more the reachable audience.
As mentioned above, Audio Description is now yet another field using smartphones as its next stage of evolution. Applications such as Earcatch or Actiview (available soon) are helping both movie theaters (by reducing the cost of buying/renting and maintaining a set of Audio Description devices) and spectators (by the convenience of having your own device and a free app, along with the ability to set all that up however you like it) and are without any hesitation the present and future of Audio Description as far as broadcasting those audio tracks goes.
It is then far from a surprise to see big companies such as Disney or Sennheiser, investing into this technology by allowing, for instance, the whole Pixar catalog to be compatible with Audio Description thanks to Disney’s proprietary app, Disney Movies Anywhere. As for MobileConnect, the solution from Sennheiser, it allows for any device to receive lip-synced audio matching the current experience of a viewer, be it in a movie theater, in front of an opera or in a museum.
With, as mentioned, the movie industry being now the gargantuan entity it is, it is unfortunate that Audio Description occupies such a hidden place for the general audience. Luckily, as Joel mentioned, things are moving in a good way: “The American Council of the Blind has done a good bit in this regard–but while we’re no longer in our infancy as a field, we surely have have not emerged from “pre-adolescence”! On a related note, the ACB is actively exploring how a certification program for describers/description producers might be established in an effort to ensure high quality AD production.”.
Picture by superanton
As game technology has advanced, allowing games to provide almost movie-like graphics, accessibility for blind and visually impaired players has rapidly decreased. In the 1980s, the early days of the game industry, text-based games, such as text adventures and Multi-User Dungeon games (MUDs), were very popular. Although these games were originally designed to be played on a PC by sighted users, with the support of text-to-speech software, text-based games became accessible for blind and visually impaired players. However, accessibility for gamers with visual impairments changed when graphical operating systems, such as Windows were created.
(Mangiron & Zhang, forthcoming 2016)
The video games industry, on the other hand, seems quite reluctant to implement an Audio Description-like technology in its ever-growing output of games. A situation that can be explained by the very simple principle behind every video games: interactivity. Indeed, if creating an Audio Description track for a story with a linear narration is quite straightforward (but nonetheless a hard work, of course!), creating one when you have no idea, when recording, if the player will have their gun withdrawn or not, which outfit they will wear, which equipment they will possess or which choices they made before, can easily become tricky!
Nevertheless, Audio Description could (should!) definitely be used in cutscenes (as they are only CGI movies in the end, despite some usually minor changes such as the outfit of the main character chosen by the player) or menus, using any of the various Text-To-Speech solution available.
Another issue with an Audio Description solution integrated during gameplay would be its superposition with sound effect that are designed to give players either a clue of their environment (enemies position, weather type, area, …) or a feedback of their own actions (a gun shooting, an enemy screaming, footsteps sounds varying depending on the surface, a bonus/malus sound, … ).
Indeed, having a voice covering those sound effects would be dramatic and would definitely serves its opposite purpose, as the player would immediately get confused and lost in its surrounding, maybe knowing what’s going on but not knowing what to do or where to go.
Some genres of video games could be better receptacles to AD than others, though. Indeed, if having a narrator during a game of any fast-FPS or heavy-action-oriented game could be more annoying than useful, some slower games such as adventure games or (some) puzzle games could definitely benefits from that. Indeed, having a voice mentioning the item on which your cursor hovers, for instance, or announcing the color and shape of the next piece of the puzzle would greatly help the progression of visually-impaired players and, therefore, make any game of this type more accessible.
That said, I believe there is a part where AD could be used outside of menus and cutscenes in a video games: in association with the HUD. Indeed, even if some of the HUD elements are already conveyed to the player thanks to sound (the most obvious example being the player health, often represented by the character being out of breath, a redundant or continuous beep or an abstract but stress-inducing sound effect), there are way too often some other elements that aren’t and could be, such as the player switching his current weapon or the location of elements on a map, for instance.
As mentioned before, one of the purposes of having sound design during gameplay is to help the immersion of the player in the game world by giving him clues about the world and feedback for their actions. When well-done, a good soundscape do more than immersing the player in another world; it completely sends them in this world, leaving their current location and trouble behind them to enjoy the experience of being everything from the last hope for humanity to a breeze of fresh air. But when done (almost-)perfectly, a good soundscape even allows you to play and feel the game without even looking at it, making visual deficiencies another trouble the player can leave behind them while jumping into the game world.
One of the most famous example for this is a game of Mortal Kombat: Deception played by series creator Ed Boon and a young player named Brice Mellen, who beat Boon as his own game. As if it was not impressive enough, a lot can be added to this achievement by a tricky catch: Mellen is blind. This exploit was only made possible by him after weeks and weeks spent memorizing the attacks but also the whole soundscape of Mortal Kombat in order to get the necessary feedback for his attacks and his opponents.
Another example is Terry Garrett, who has been interviewed by Oddworld Inhabitants about his way of playing various games from the Oddworld series. Indeed, as Terry is blind, he manages to play and finish the games by only using the sound design produced by various elements of the game. As he mentions in his interview, “(…) the sound is vital for other parts. Chant orbs make a humming sound so I can tell they are there, the blinking bombs are better done by sound anyway so I am a master at those. The creatures make noises, the spirit lock shakes and makes noise, and (most excellently) the motion detection beams even make noise when they touch you, which in a lot of games they don’t worry about (…).”.
Brice and Terry are obviously not the only visually-impaired people who enjoy, or would like to enjoy, video games. And Mortal Kombat and Oddworld are not the only popular series or games that has been praised for their helpful use of sound. But sadly, while the quality of soundscapes in video games never cease to increase from game to game, too few games are using the technologies mentioned above (Text-To-Speech recognition for menus, Audio Description, … ), implicitly forcing volunteers to invest their own time into the adaptation and accessibility of video games, such as the creation of a YouTube channel specialized in Let’s Play for visually-impaired people.
Some games made specifically for people with visual deficiencies obviously exists such as audio games (remember Listen to me, Clara from another article?), creation by volunteers of accessible video games based on actual, non-adaptable counterparts such as Shades of Doom, AudioQuake or Blind Hero or games with a specific focus on sound such as The Explorer and the Mystery of the Diamond Scarab available for the Wii.¹
So what can actually be done in our beloved AAA or indie games? And how can it be done? Well, a cheeky answer would be, I believe, “a lot of things in a lot of ways”. As mentioned above, I truly believe that Audio Description should be implemented in modern video games, at least in cutscenes and menus. The major issues with this idea are time, since this process can be quite time-consuming as games grow to be more and more sophisticated, and space, as this big batch of Audio files will obviously take some precious space on a disc/hard drive.
For the first issue, time, a lot of resources are available online to help whoever wants to create an Audio Described project, such as Description Key, for instance. For the second one, a solution (which, I am aware of, has its own flaws) could be to offer the Audio Description as part of a free DLC for the game, as were the English voiceovers of Fable III for the localized versions of the game, for example.
Furthermore, as Carme Mangiron and Xiaochun Zhang mention in a publication coming later this year ², “since game technology keeps advancing, research about how to overcome this issue – whether increasing the storage capacity or including screen reading software and text-to-speech technology by default to portable consoles, as with mobile phones, should be promoted in order to make mainstream games accessible for the blind and visually impaired”.
This text-to-speech technology has already been implemented in a few games, as the 2012 release from Lab Zero Skills, Skullgirls. As Mike Zaimont, lead design director, stated here, “I’m pretty surprised other PC developers haven’t done this. Most text and informational things are already updated on screen so you don’t have to write special code to generate new text for most situations. It takes very little time, and if more people can potentially enjoy your game, there’s really no reason not to do it.”. More information on this technology can be found on the same page on the same website, Game Accessibility Guidelines, which is also an amazing pool of resources for game developers who are curious about adding accessibility at various levels to their games.
Picture by Unsplash
Today many theatrical companies engage the services of trained audio describers to enhance the theatre experience for people who are blind or have low vision. The describer occasionally speaks “between the lines” to describe entrances, exits, actions, and key props to people who cannot see them. Often, before the show and during intermission, a describer “sets up” for the performance by detailing the stage layout, props, characters, and costumes. There is never any charge to the theatre patron for the use of audio description, although the theatre may pay a fee for the service.
(http://www.acb.org/adp/theatres.html)
Video games and movies are not the only types of entertainment that can benefit from a sonic help to be more accessible to visually-impaired people. Television programs offers Audio Description as well, be it through the broadcast provider or the channel itself. Even live performances such as opera, dance or theater, as well as museum tours, can nowadays (and since quite a few years, actually) be audio described for a better accessibility. On this subject and on the topic of incoming improvements, Joel Snyder stated that “While AD, as a service, began in the theater, far more focus–not surprisingly–has been on its use for media. Still, developments in the performing arts include:
– recorded description timed to light cues
– provision of description at *every performance* via scripting and consideration of the describer as a “cast member” or incorporation of description within the play itself (most performing arts description is still a volunteer activity and not enough work is done to make the public aware of the service/encourage potential consumers to use the AD when it’s offered). In museums, there [is] a great deal of growth in the training of docents in audio description techniques (for use as they lead tours) and the development of either separate, recorded AD tours or the incorporation of AD elements/techniques in existing audio tours. The National Park Service is very actively incorporating AD at its various visitor centers.”
We saw it, for every main type of entertainment out there exists an audio solution capable of enhancing the experience so it can be enjoyed by a bigger share of the population, regardless of their disabilities. As the technology will evolve, the creators behind our games, movies, TV programs and other theater plays will take advantage of it to develop new and original ways of expressing their visions and entertain us. Let’s just hope that a share of these technological improvements will be used to help broadening the scope of their audience to those with disabilities, which would not only result of a chance for a bigger financial success but also and mainly of a necessary improvements on a human level.
References:
¹ This list has been curated by Carme Mangiron & Xiaochun Zhang in their forthcoming 2016 publication along with many more games and interesting ideas about Audio Description in video games. Mangiron, Carme; Zhang, Xiaochun (forthcoming 2016). Game Accessibility for the Blind: Current Overview and the Potential Application of Audio Description as the Way Forward. In Anna Matamala & Pilar Orero (eds). Researching Audio Description: New Approaches. London: Palgrave Macmillan UK. p. 75-95.
² Mangiron, Carme; Zhang, Xiaochun (forthcoming 2016). Game Accessibility for the Blind: Current Overview and the Potential Application of Audio Description as the Way Forward. In Anna Matamala & Pilar Orero (eds). Researching Audio Description: New Approaches. London: Palgrave Macmillan UK. p. 75-95.
Useful links:
The American Council of the Blind’s Audio Description Project
Researching Audio Description: New Approaches
Thanks a million times to the following persons for their amazing help and support during the writing of this article:
Joel Snyder, author of The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description and President of Audio Description Associates.
Carme Mangiron, Pilar Orero & Anna Matamala, researchers and lecturers at the Universitat Autònoma de Barcelona.
Fantastic overview, and hopefully it will raise the issue to more people. Let’s hope this information reaches a larger audience.
I think people with disabilities will find the following resource useful: https://www.textfromtospeech.com
It is tool that converts speech to text and vice versa (useful for people with problems with eyes, hands and/or back).
As far as I know it is based on Google speech recognition, gives
accurate results and free.