Dynamics In Education – Interview With Michael Sweet, Professor of Game Audio at Berklee College of Music

 

As the Artistic Director of Video Game Scoring at Berklee College of Music, Michael Sweet leads the development of the game scoring curriculum.  Michael is an accomplished video game composer and has been the audio director of more than 100 award winning video games.  His work can be heard on the X-Box 360 logo and on award winning games from Cartoon Network, Sesame Workshop, PlayFirst, iWin, Gamelab, Shockwave, RealArcade, Pogo, Microsoft, Lego, AOL, and MTV, among others. He has won the Best Audio Award at the Independent Games Festival, the BDA Promax Gold Award for Best Sound Design, and has been nominated for four Game Audio Network Guild (GANG) awards. In 2014, Michael authored the book “Writing Interactive Music for Video Games” which is now available from Pearson Publishing.

Michael was a professor of mine during my studies at Berklee College of Music. Given this months’ theme of “education”, I thought it would be enlightening to hear Michael share his perspective as a professor of game audio with the Designing Sound community. So, without further ado…

DS: How long has formal game audio education been around?

Michael Sweet:  Video game education is a relatively new field of formal study in education compared with similar entertainment mediums like film.  Early film studies programs date back to 1919 in Moscow, and 1932 at USC in the US, video game studies really began to flourish starting in the early 2000’s.

In terms of audio education for video games, there are about two dozen programs around the world which have been growing substantially since about 2006.  You can certainly make a case that earlier music and sound programs were also indirectly preparing students for the game industry with music technology programs that went back to the 1980’s with the introduction of the computer for music production.

There is a wide breadth of knowledge that influences curriculum in game audio programs in addition to the game components.  Audio education for games borrows widely from techniques used in film and other media.  For instance, video game audio is influenced by many of the core fundamentals of audio storytelling from film, and opera before that.  In addition, there are many parallels between the techniques that video composers use today with non-digital experimental composers use of interactive form in the 20th century.

Game mechanics and interactive storytelling as they apply to music, sound design, and audio implementation are still of most importance and are where these programs really focus.

At Berklee, the initial push into video game audio began when students came together to create the Video Game Music Club.  This club convinced the administration to hire someone to teach video game scoring and develop new curriculum.  My hiring at Berklee in 2008 was an outgrowth of that effort.

 

DS: What inspired you to get into education, specifically game audio education and what excites you about teaching the topic?

MS:  I didn’t start out thinking that I would enter the world of education, or even that I would compose music for games for a living.  In 1992 when I first began composing for games (TimeWarp with the Flintstones and the Jetsons), it was totally by accident.  Games were not on anyone’s radar as a career path back then.  Since then I’ve worked on over 100 video games, many of them kids and casual games. Today, I meet many students that are looking to pursue video game composition as a career when they arrive at college.

The beginnings of my career in education began when I was asked to teach sound design at Parson: The New School of Design in New York.  I was able to teach one class in the evenings after work starting in 2003.  My day job was still being a composer/sound designer at the business I co-founded in New York, Audiobrain.  Then, in Fall 2008, Berklee wanted to hire a composer to lead the development of their game scoring curriculum.  I decided to make the leap from the professional world to education.

In those early years at Parsons, I discovered that I really love teaching – I enjoy inspiring students to become composers and sound designers for video games.  Equally rewarding is seeing students succeed once they graduate and enter the professional world.  In parallel, since I find myself caring deeply about many of my students, it can be it can also be disheartening when students fail out in the industry.  When this happens, I always question whether there was something more that I could have done.  This helps me continue to strive to develop and revise our curriculum.

The two things I enjoy most about teaching are seeing the student’s evolution from novice writers into professionals, and creating collaborative relationships between students.  It’s remarkable how much students can grow over just a couple of semesters and it can be really fulfilling to watch.

 

DS: How important has your prior and ongoing experience as a working professional been to your time as an educator?

MS:  I’m not an academic, so my professional experience is essential in my teaching.  Game audio faculty come from various backgrounds, and much of their style of teaching has to do with the kind of college or program that they teach at.  Berklee’s undergraduate focus tends to be preparing students for the real world, in some ways similar to a trade school.  Masters programs on the other hand, explore broader perspectives, theoretical concepts and advancing music as an art form.  My background is better suited for the former since I’ve owned several business that focus on creating music and sound design for games.

I try to balance the creative with the practical business practices that students need to know to get work.  So I do my best to relate my past experiences in the real world, as well as my mistakes so that my students can better succeed.

Since I’m working in the industry on only one or two games a year, I augment my industry knowledge by frequently speaking to alumni working in the industry and then bring that knowledge back to Berklee.  I try to track alumni that have graduated from our video game courses to see how they’re doing, and ask the question, “What is one thing that you wished you had learned at Berklee that would have helped you, and your career?”

DS: Often part of an educator’s role is to inspire and generating interest in their subject. As you work at a specialist music school, do you find students already have that passion and drive towards your classes?

MS:  Most of the students that I engage with have spent much of their youth playing video games, so many of them have a passion for it when they enter into the program.  Their passion though, is generally only from a purely aesthetic point of view, so it’s generally my job to get them inspired about the creative design, technical processes and team aspects of working in game audio.

One of my favorite parts of the curriculum is to allow students to work on real games created by other students.  The student composers and sound designers feel empowered with independence, and are able to express and contribute to a collaborative vision.  At Berklee, we’ve worked hard to create collaborative relationships with other universities with game design programs so that our students have the opportunity to work one-on-one with game designers at the student level.

 

DS: What are some of the biggest misconceptions or surprises about interactive audio that your students have?

MS:  Many students really don’t have an understanding of the business aspects required to be successful in working in game audio.  Many believe that if they can just write better than everyone else that they can find success.  The business aspects of selling oneself is equally important in being successful. If you’re unable to convince someone to write you a check to create game audio, then it’s likely that you won’t find success as a composer or a sound designer.  In our curriculum we focus on many business aspects including how to get work, how to price yourself, how to bid a job, create sales materials, how to write a resume, and the creations of demo materials. My experience as a business owner helps me relate experiences that I’ve had as a professional to students.

The actual game mechanics and interactive structures can also be difficult for students to understand even though many of the students have been playing games for many years.  Many students don’t understand interactive music structures, and how the music adapts in real time based on player actions.  Composers have to get used to the idea that their music might never be played back the same way twice because players have control of the action.

Lastly, there is a difference between writing for oneself as opposed to writing in service to a game.  If a composer is not able to satisfy a developer’s vision for the sound or music, then they’ll have a difficult time working for that developer in the future.

DS: Your courses are currently taught within the Film Scoring department’s curriculum. What role does your student’s experience and familiarity with linear media play in your classes? Do you use it as a teaching tool, a jumping off point, or is it a hindrance?

MS:  At Berklee, we currently have six classes at the undergraduate level which teach game audio, and two more at the graduate level.  These classes support the two game audio minors that are offered – Audio Design for Video Games, and Video Game Scoring.  I primarily teach the game related courses in the scoring minor which are part of the Film Scoring Department.

The Film Scoring department at Berklee has been extremely supportive of the game curriculum and advocating for the creation of new courses for students.  Because many of the students in the Game Scoring Minor are majoring in Film Scoring, it allows me to leverage other parts of the curriculum, and focus on areas which they’re unable to teach.  I really rely on those classes to teach orchestration, dramatic storytelling with music, so I can concentrate on interactive music structures, artist development and the business of composing for games.

Students that come from a film scoring background may struggle with adapting their linear writing techniques to interactive frameworks.  Composers working in video games aren’t able to score the action as precisely as they can in film and linear mediums.  In linear media, the composer can plot out how they resolve a piece of music at a particular place in time.  In games, the action is determined by the player so the composer doesn’t know when the music will need to change.  The music that they write needs to be adaptable in real time so certain techniques that are used in film scoring do not apply in video games.

In addition, many students are taught to write strong melodies in film scoring whereas in certain game situations, having strong melodies can quickly become repetitive.  The length of a modern AAA game is frequently 15 hours or more of gameplay.  Typically a game of this length still only has 2-3 hours of music.  So the composer must write with adaptive techniques in mind so that the score isn’t as repetitive.

 

DS: Other than video games, what mediums do you encourage students to look towards to fuel creativity?

MS:  Writers and artists can get burnt out if they don’t find a balance between their work lives and the rest of their lives.  Much of their lives outside of their work can inspire music and sound that they create at work.  There is so much great art around oneself if one is willing to go out and look for it, whether it be museums, live music venues, movies, 20th century experimental composers, exhibitions, radio – inspiration is everywhere.

There are many other mediums that also utilize sound and music interactivity.  These skills can be applied to a whole host of additional related industries from amusement parks to toys, apps, museum installations, physical sound art, and many more.  Many of these areas are very interesting to look at in terms of fueling ones own creativity.

 

DS:  In your book you say “limitations can force a composer to look for new ways to be creative”. What are some of your favorite examples in game audio that exemplify this thinking and do you impose limitations on your students’ work to encourage creative thinking?

MS:  Having limitless numbers of options can actually be quite a hindrance when working.  It can be much easier to innovate in one small area as opposed to trying to innovate in all areas at once.

Koji Kondo’s limited instrument palette in Super Mario Bros is an excellent example of how when the focus is shifted to just melody, that he was able to come up with one of the most iconic themes in video game history.

Limitations in curriculum and compositions isn’t new, and in fact one of my favorite examples is Ligeti’s “Musica Ricercata”.  In these compositions, György Ligeti imposed limitations on himself for each one of the eleven pieces.  The first movement only uses two notes, the second uses three notes, etc.  These pieces are extremely beautiful, and also very unusual and unique.

Focusing an assignment on a particular set of limitations can help the student really hone their skill in a particular area.  For instance if I were to design an assignment to help students utilize more dynamics in their score, I would limit the other parts of assignment like harmony, instrumentation, and rhythm.  This way the assignment becomes solely about creating the best dynamic piece

There are many types of assignments like this to help students improve specific aspects of their writing.  In our first applied composition for games class, students are required to write a standard midi file, and a mod file.  These restrictions help the students focus their creative ideas and innovative on only the aspects which are more flexible like melodic, rhythmic and harmonic development.

 

DS: In your book you say “the idea behind a composition can be just as important as, or more important than, the actual aesthetic qualities of the composition”. Do you find students focus too much on the “sound of the thing” as opposed to the underlying ideas behind their work?

MS:  Composers and sound designers can certainly take the easy road when composing.  It’s easy to write in one key, one meter, or one tempo etc.  It’s much harder to look at the story of a game or film, and tell that story through the decisions that you’re making when composing.  The best scores are not necessarily the ones that are aesthetically pleasing, but the ones that do a good job supporting the story.  The best scores are the ones where the composer makes choices based on that story – instrument, key, meter and tempo changes all reflect the story.   These scores have more depth, and in my opinion are inseparable from the stories they tell – one cannot exist without the other.

 

DS: Yes, I’ve often heard you describe truly great scores as those that ingrain themselves so deeply with the larger work, that neither the music, nor the film or game, can work without the other. What are some of your favorite scores that illustrate this quality?

MS:  Games and films sometimes have such indelible audio that if you hear just a few notes or the sound, and you remember the experience.  I could fill several pages of examples, but here are some of my favorites that absolutely wouldn’t be the same without their soundtrack and audio:

Game Scores: “Transistor”, “Fez”, “Super Mario Bros.”, “Bioshock”, “Mass Effect”, “LA Noire”, and “Dead Space”

Sound Design in Games: “Limbo”, “Battlefield 3”, “Half-Life”, “Fallout 3”, and “Dead Space”

Film Scores: “Alien”, “Psycho”, “North by Northwest”, “Cinema Paradiso”, “Chinatown”, “Lord of the Rings”, and “Catch Me if You Can”

Film Sound Design: “The Conversation”, “Eraserhead”, “Apocalypse Now”, “Requiem for a Dream”, “Saving Private Ryan” and “Wall-E”

 

DS: How has the tool-side of interactive audio education developed? Is it still a challenge to find suitable teaching tools or has the increasing popularity of middleware solutions such as Wwise and FMod allowed you to focus more on the composition process?

MS:  Tools change rapidly.  Many of the sequencers, tools and middleware that I used in the 1990’s don’t exist anymore.  I believe that the tools shouldn’t matter, and that the fundamental concepts and creative design behind the work is more important.  Students should be able to compose great music in any sequencer.  If a student knows the fundamental language behind the creative tools, it doesn’t matter what the front end looks like.  For instance, you can use Fmod or Wwise to create an adaptive score that uses vertical remixing.  Which audio middleware you use isn’t important, but how vertical remixing works is very important.

 

DS: Despite game development courses now being commonplace, do you still find there’s an element of stigma (whether in the world of music or within formal education) around the topic of game audio courses?

MS:  At Berklee, our vision statement is to be the world’s leading contemporary music institution,  therefore they have put a lot of support into helping create our game curriculum.  But, at the same time, people that didn’t grow up playing games don’t understand how sophisticated they’ve become.  Older generations of music teachers are surprised that there is more orchestral recording for games than all of the orchestral work for television, or that 20th century aleatoric techniques are used in modern games. This stigma is slowly going away, but at some more classical based programs they may not consider it ‘real music’ or a serious artform.

Over the past two years, Berklee has been working on developing a full Game Audio Major for the college.  The proposed major focuses on the following primary disciplines in game audio:

• Music Scoring
• Sound Design
• Implementation and Programming
• Game Design and Development
• Business and Entrepreneurship

We hope to get final approval at the administrative and board levels to launch the new major in the Fall of 2015, or Fall 2016.

 

DS: What challenges do you come up against when adapting or developing new course material?

MS:  The biggest challenge at Berklee was that we don’t make games.  If our students are going to work on actual games we need to go out and find collaborative relationships with other universities which have game development programs.

In addition, building actual game examples that teach a fundamental technique to students can also be a challenge because they end up being all the same.  For instance, the audio middleware Wwise includes two basic games as examples when you download the software.  Many composers and sound designers re-skin these two games, so you tend to see it over and over again.  Creating game frameworks that are more flexible are more difficult to build, but they allow composers and sound designers to take more creative liberties with their artistic development instead of being slotted into a genre or framework.

 

DS: What is the hardest element of game audio to teach?

MS:  I believe that the hardest parts are teaching students about the reality of a working composer or sound designer – and that there are no guarantees of success.  There are so many elements that determine success, but most important is a desire to work really, really work hard to reach your goals.  Success may also take quite a long time, and as a young person it may be hard to have that kind of patience.

One common thing that many of the guests artists that have come to Berklee to speak say is that they would being creating audio for games anyway, meaning that even if they weren’t making money from it, they have so much love for the creation of music or sound design for games, that they wouldn’t give it up.

 

DS: Given the present environment of rising student debt, a crowded workspace and alternatives to college education in the form of online courses, video tutorials and certification programs, do you feel a formal college education with a focus in game audio, whether at an undergraduate or graduate level, still has value?

MS:  This is one of the more difficult questions that all higher education institutions are examining with the high cost of education and the volatile economy over the past few years.  Although the games and apps industry has exploded and provided lots of work for composers and sound designers, there is no promise of a position once a student graduates.

A creative career for many is more desirable than a desk job.  Many people want to pursue their dream in music as a career.  Colleges accept students based on student’s dreams as opposed to the number of positions available.  This means that there are sometimes many, many, creatives all trying to get the same gigs.  This raises the bar in terms of being the best, and it can be disheartening to face rejection.

One of the questions I pose in my classes to students is “How many game composers can we name?”  Then I usually follow that up with, “How many people are we graduating every semester that want to be game composers?”  There is a realization in the students that the people at the very top level is a very small club.

Students that graduate wanting to be the next Nobuo Uematsu, or Marty O’Donnell are going to have a very difficult time initially because these are not starting positions.  Composers and sound designers frequently have to build a resume of smaller gigs, before being able to submit their work for a large project.  Building these connections is essential at the beginning of a young person’s career.

I encourage my students to cast a wide net when first starting out; work on any audio project where you can get paid.  If they wait for a Nobuo’s job, or Marty’s job, they may never get that opportunity.  Fortunately there are many gigs out there if graduates do not set their sights too high when first starting out.  The skills that are taught in game audio programs are skills that apply to many different industries beyond games that utilize relevant interactive music and sound design skills.  One connection always will lead to another, as people that you’ve done work for recommend you to their colleagues.

Online education has been growing at a rapid pace with independent game audio programs like Leonard Paul’s School of Video Game Audio, to online extensions of universities including Berklee’s own online program.  In many ways these programs can be a better alternative for the student that already has a degree because it can easily fit into the schedule of a person that’s working.  Online programs are generally more economical than going to a college.

Online education has it’s downsides as well, including having to wait for feedback instead of having a instant dialog, and it might be more difficult to make the kinds of connections that help composers and sound designers succeed.  Also, I believe that my students learn almost as much from the interaction from one another, as they do in the classroom – this may be harder to duplicate in an online environment.

 

DS: Where do you see game audio education in 10 years?

MS:  Video games are innovating at such an incredible pace.  I feel like we are just discovering what we can do with the medium, and how we can push the boundaries.  I’m a firm believer that composers and sound designers that have a broad understanding of interactivity and how it impacts the audio, will write better music and design better sounds for games.  Video games is not just another medium, it’s a new artform where we cannot simply apply what works in linear media.  This re-conceptualization is going to take some time to really develop to see it’s full potential.

Game audio educators need to keep looking at the future to try and define trends, and continue to develop a new language for game audio which didn’t even exist a few years ago.  Organizations like GANG and the IASIG can help us organize new techniques and ways that games utilize audio to push these boundaries.  Technology also advances our medium by creating a language of it’s own.  In order for audio educators to remain current, they need to constantly take note of this innovation, and relay these concepts to their students.

In audio there are things that we are good at now, but there is a long way to go.  One example of this is that even though we are using adaptive music in our games; we typically define a closed system.  If I play through one level of many games, I know exactly how the music is going to play throughout the rest of the game – almost all game audio systems end up being predictable and repetitive.

What if our systems could dynamically change, from level to level; not just the aesthetic qualities of the music but the way that it’s implemented changes throughout.  To use an example from film, John Williams created a two note theme for “Jaws”.  In the movie, the composer sets the viewer up by equating this two note theme to equal shark appearing.  But, in the middle of the movie, John Williams doesn’t play the theme, a shark appears and it scares the entire audience.  I’ve never seen this in a game, but I’m hoping that people think about dynamic systems in the way John Williams did in Jaws.

I believe game audio programs will continue to grow in the near future.  In today’s competitive environment, it has been trendy to add game courses to the curriculum as it helps universities look “cutting edge”.  At some point it will stop being trendy, and as a result the game audio education programs that survive will hopefully be the really innovative ones.

 

Thanks to Michael Sweet for taking the time to do this interview. His book “Writing Interactive Music for Video Games” is now available from Pearson Publishing.