Reevaluating the Practice of Testing Prospective Candidates

Eureka moments are often steeped in excitement and wonder. They are the great discoveries and innovations of the ages. Others are a slap on the forehead, a “why didn’t I think of this sooner?” moment. This particular Eureka moment I’m about to share is of the latter variety and came about three paragraphs into an email I was writing. The monologue from my brain to myself went something like this: “My hiring process is fairly unique and was created as a reaction to what I feel is a misguided norm within our industry. I should write it down and share it with people!” So there you have the impetus for this rant.

Hiring a new employee is always a gamble. You’re basing a very important decision for you, the candidate, often that candidate’s family, and your entire team on a few conversations, a demo reel, possibly a test, and a few hours spent one-on-one with this person. Is that really enough time to gauge the multitude of questions like: Is this person a good sound designer? Will this person fit in to our team’s culture? Can this person handle their tasks professionally in a deadline-driven environment? If they don’t already know our tools can they learn them in a relatively quick time? How do they behave under pressure? If they were an item on a Taco Bell menu, which one would they be?

With good interviewing skills (and lots of practice), you and your team can uncover answers to many of these questions. But one critical thing missing from a set of interview questions is related to the applied skillset(s) of sound design. We generally ask for a demo reel to help weed out unqualified candidates and assess skill level, but a demo does not show us the nuts and bolts of how they work, often how they design and integrate. A bad demo doesn’t explain what the candidate’s contribution to the various clips actually is. These are some of the reasons why we also often test our candidates. But rather than using the test as a rehash of their demo reel, we should be looking at our hiring tests as a way to pull more information out from the candidate to answer these other critical hiring questions.

The way it’s been done

Testing is not novel to audio teams; many disciplines do it, from artists and animators to programmers and designers. Interestingly, many audio tests I’ve seen are often just sound replacements for a cinematic. In our line of work this is usually a minimal part of what we do as sound designers. Naturally, many games have cinematics and sometimes there is even a sound designer (or team of them!) dedicated to cinematics, but designing for a linear piece of media lacks a demonstration of any of the technical or nuanced differences between linear and interactive design.

I’ve had other tests which included a portion of gameplay and asked the candidate to design the scene with both their own sounds and a set of sounds provided by the company. This is a bit more interesting and appropriate scenario in that it looks at how the player may approach designing both on their own and with a set of limitations. But it’s also worth considering on some level that asking someone to design a scene outside of their work environment can provide an unfair disadvantage to a talented sound designer who may not have access to sound libraries, microphones, recorders or plugins due to financial constraints. With that said, there’s plenty of free or cheap tools available and passion and talent can often make do with less-than-top-of-the-line gear or samples.

The final issue where many tests fall into unfair territory is in considering time expectations for the candidate. We work in a deadline driven environment, and it’s good to see how a potential employee can handle a heavy workload, but we also need to remember that a candidate may have a full time job (and may even be crunching!), and additionally they most likely have a life outside of work, be that friends, family, small children, pets, etc. Is it fair to demand free work of someone that may help you evaluate them, but also puts them at risk professionally or personally? The answer to me is yes– within reason, and that we should be putting greater emphasis on what we can evaluate from a candidate based on their past work (if it exists) and from in-person interviewing and exercises, and put constraints on our tests so that they do not become time sinks for potential candidates.

A new way forward

I have shared my test and interview plan with some colleagues and it has struck most of them as novel. It has been overwhelming regarded as positive by candidates and has so far done a good service in helping me find qualified and talented candidates, much more than I’ve been able to hire unfortunately.

Obviously the first order of business in the entire hiring process is writing or revising the job description. Try to outline the actual duties of the job, qualifications, and skills or experiences which are required vs. optional. A lot of these bullet points can be a bit loose, but try to err on the side of what you’re really looking for. A requirement to have shipped at least 3 titles is pointless if you find an amazing candidate who has only shipped two. So try to be descriptive so that candidates understand what they are applying for and you’re not over or under-asking of potential candidates’ qualifications. I also like to throw in a “fun” question to be included as part of their cover letter such as “Please describe your favorite sound toy,” or “What’s your favorite sound you’ve designed?” A question like this serves multiple purposes: First it shows they’re paying attention and you’re not just getting a carbon copy cover letter. The cover letter is the first glimpse into how a candidate communicates [in this case in written format] and perhaps you can start gauging their passion levels and cultural fit as well. Also it’s critical to make it clear that a demo reel should include annotations for designer’s contributions. I’ve had to send a few back to get clarification and others have been so poorly communicated I rejected the candidate outright.

If cover letter, resume experience and demo reel are all up to par, which, depending on the position, may eliminate the bulk of candidates, next up would be a phone interview, which is usually the basic stuff: explaining the position to them, talking about their experiences, their current or most recent position, trying to get a sense of how they communicate verbally, and how much their representation on paper (their resume) matches their experience. I know some managers who use this time to weed out candidates using excessively difficult or esoteric technical questions. But if you’ve already been through resumes, cover letters and demo reels, a simple conversation should usually reveal whether this person is real, passionate and whether or not your needs and their goals are a match. If they need to understand the difference between binaural and ambisonic and cannot describe it, perhaps they are not the best candidate, but if they can’t give a concise definition of dithering or when to use DC offset, these issues are generally less of a concern for a modern day sound designer.

If the phone screen goes well, like most hiring managers, I move on to the test and this is where I diverge a bit. For the reasons mentioned above, my test is a bit unique. It is not based on how well they can create sounds because I should ideally be able to get a sense of their sound design from their demo, and their process from the phone screen. Instead I’m looking for other critical aspects of their work habits and analytical skills which will ideally help demonstrate whether or not they will be a good employee AND a good designer.

Part one of my test actually has two parts. I give them a few movies of gameplay from a past game along with a ton of in-game assets. Part one is a written portion asking them to choose one video, and write about what they feel the most important aspects to cover with sound design are in the scene, but also explain details they feel may be overlooked by others as well as provide pie-in-the-sky design ideas. The second part of this exercise asks them to design the scene they wrote about using only the assets provided. They can manipulate those sounds in any way they like and are also instructed to only focus on what they deem important and then write about their choices and why they made them. I actually stole this idea from Andy Martin who set up the sound design test at Sucker Punch before I joined. I feel it does a good job in discovering how well a candidate can analyze audio, communicate their vision and discuss their design philosophy.

The next part of the test evaluates both critical interactive sound design skills and basic Wwise proficiency (Wwise is the audio middleware we use, so a basic understanding of it is required). Rather than designing to picture, I give them a video of a specific event– in this case the powerful ionic smoke power from InFamous Second Son and have them create all the events they would use in the real world to design for this sound, create a SoundCaster session in Wwise for these events, write a brief explanation of how and why they chose these sounds, when they would like them to trigger in the animation, and any additional mixing techniques they would use to enhance the scene. This is a good litmus test for seeing how they approach the ever critical facets of sound design within a middleware tool. So much of what we do in regards to the nuance of sound design happens more and more in the middleware and it’s great insight to see how someone thinks creatively within the tools they’ll be using everyday.

The last assignment is more of a technical assignment strictly in Wwise, which is to construct a modular clothing system for a character with three different outfits and create events for 5 specified animations (arm swing, jump, land hard, etc.). In this assignment, the user only creates the structures in Wwise, they do not do anything with sound assets, which is a great way to evaluate how they think within the middleware and how they build more complex systems. The interesting thing I found in this portion of the test is that candidates came up with a raft of unique ways to complete the assignment and some of the more interesting ones have come from candidates who were not as well versed in Wwise, which I view as a positive– someone able to take a new tool and find an efficient means to achieve a task generally extend these latent skills to other areas of their work as well.

I explicitly acknowledge that candidates are doing this test for free on their own time and as they most likely have a job and a life outside of work, I ask them to try and spend no more than 8 hours on the test, and I give them ample amount of time to complete it, usually two weeks, with the option to extend more if they need to. Again, I don’t want to put anyone at a disadvantage because of their personal work/life situation so I try to keep a level playing field by providing these limitations. There is no real way to monitor candidates to ensure they stick to this request, so it’s really a matter of trust that I try to emphasize when explaining the time constraint aspect.

After the test, I will usually schedule a second call with prospective candidates and follow up on their tests, ask them any other questions that may arise either via their test or through our conversation and then take all of this information to determine if we want to bring them in for an onsite interview.

My onsite interview process is meant to gauge a variety of things: how well they would get along with teammates both on the sound team and within the rest of the production group, to allow other members of the team to evaluate the candidate, learn about their experiences, ask them questions and also gauge their cultural fit into the larger company. I also encourage my co-workers to be creative in the questions they ask, or at least to have some fun with the interview. For example, Michelle Thomas, one of my sound designers, likes to ask candidates if they could be a dinosaur, what kind would they be. The answer to this question is never going to be a deciding factor in the hiring decision, but it’s a fun insight into a candidate’s personality and we’ve got some really great answers: one candidate said he’d be a velociraptor because they hunt in packs and he likes doing things as a team. Another said they would be a brontosaurus because they were vegetarian. Interviews can be nerve wracking and adding some fun into the day can help relieve tension and get to know the candidate better.

I usually kickoff the interview day with an introduction of myself and the project we’re working on as well as going into more detail about the role. We’ll often go over the test together for any final clarifications or questions. Then I’ll hand the candidate off to a designer who, after getting to know them a little, gives them a hypothetical design scenario like designing a collectible or walking through a mission and discussing sound needs with the game designer. Next one of my sound designers will discuss more sound-focused topics and gauge how they look at their past career, previous employers and co-workers. We have two people take the candidate out to lunch, usually a programmer and an animator since we work closely with both departments. These two are responsible for evaluating both the candidates past experience with these two disciplines as well as cultural fit within the studio, whether or not they’re impulsive, can give fair evaluations of people, etc. We’ll have a producer evaluate their task tracking and interdepartmental communication skills, and lastly I spend another couple hours with them asking more questions, posing hypotheticals and doing thought experiments and exercises.

My interview is meant to gauge several skills: how well they can handle an ever-shifting production environment, how they adapt to change, and even how passionate they are about various aspects of sound design. My favorite part of the interview, and the favorite part for most of the candidates, is when I lay out a bunch of props on the table and tell them to play around with these things and see what kinds of sounds they can make out of them. There is no right answer, it’s an exercise to see how their brain thinks on the fly, to glimpse at their creative process when performing foley, and help evaluate their passion levels. The props I use are random, but sometimes I’ll use similar ones across candidates and it is fun to see what props certain candidates gravitate to as well as the common sounds that many people find within these items.

 

Then comes the difficult part of choosing a candidate. Usually if they’ve made it through this whole process it is either clear they’re not going to work for one reason or another or we want to hire them and deciding between multiple talented candidates is both a difficult decision and a gamble. Even though we know a lot more about this person now than we did before the interview process, we still don’t have the full picture of how they work with a group of new people in a different production environment with different tools.

There is no perfect solution for wading through candidates to find the best, but this process we have developed has evolved into a pretty solid measurement of how well potential candidates will thrive or fail within our specific development environment. I hope some of these techniques prove useful to others because I feel the testing and interview processes in finding a new job should be looking at a candidates skills in a way that minimizes detrimental impact on their life while also promoting creativity and getting a glimpse at the passion that makes a good sound designer great.