Rain and Terrain: How Animal Crossing’s Audio Design Helps Even the Totally Blind

Note: This post is a reblog of a post I wrote for the Audiokinetic Blog, and was published on June 17, 2020.

Animal Crossing is playable by the totally blind! These words have shocked many people to whom they have been spoken, and they even shocked me the first time I heard them. I couldn’t fathom how the blind could play this kind of game. I had certainly never played anything like it before. Nevertheless, now that I had heard it was possible, I had to find out how, and of course I had to try it myself. It shouldn’t come as much of a surprise to you, dear reader, that the primary reason Animal Crossing New Horizons is playable is its audio design. What may surprise you, however, is just how good it actually is. Sit back, relax, and I shall explain it all to you.

For any game the blind can play, we need a lot of auditory information. We need to know where things are so we can find them, or in some cases just use them as landmarks. Animal Crossing does this very, very well. Not only are there different sounds for each type of terrain, making it possible for us to tell the difference between the beach, or grassy areas, or the plaza, but there are plenty of environmental queues. Get this. If there is any wind blowing in Animal Crossing New Horizons, you can hear it blowing through the trees, but not as just another ambient effect. No, you can hear the wind blowing through the trees at their exact positions on your island. This is the first game I can think of where we can actually use the wind itself to discover where individual trees are located. It’s phenomenal!

One of the greatest and most helpful things, though, is your ability to make your own audio landmarks by placing certain items in important areas. My house, for instance, is on the edge of the south beach, allowing me to use the beach as a baseline. I then placed a permanently lit campfire just to the left of my house, ensuring that if I walk along the south beach, I will hear that campfire and find my house every time. To make life even easier, I later placed the shop on the other side of that campfire, so all I have to do is step outside my house, walk around to the left so the campfire is on my right, and there’s the shop door. A third example is my placement of my island’s museum, which is right next to a waterfall on the north side of my river. Why? Because the waterfall is very loud and easy to detect, making it the perfect natural landmark.

Don’t worry, though, I’m not even close to being done. There’s plenty more audio goodness to be discovered in this game. I’ve already talked about how we know where trees are, but I haven’t covered the act of collecting from those trees, or anything else really. There isn’t much to say except that when an item drops to the ground, it makes a sound as it lands. These sounds are also perfectly positioned. So when I swing my axe at a tree, I can tell sometimes that the fruit from that tree landed on the same side of the tree as I’m on, whereas the wood landed on the opposite side. The audio positioning is, truly, that good, and keep in mind this game doesn’t even use surround sound.

Speaking of collection, though, what about those balloons? What about bugs, fish, and fossils? Well, balloons are the easiest. In fact, the deaf/hard of hearing community has lodged a complaint that balloons can be heard long before they can be seen, which is true. As a balloon floats through the air, the sound it makes is very, very distinct. All we need to do is move toward that sound so it becomes loud, then center it in our headphones, and fire that slingshot. I have taken down several balloons in a single shot because they’re so easily trackable. Sometimes it will take several attempts, but usually I’m just not close enough, or the balloon floated behind me. I always get it eventually.

Bugs are a bit trickier. There are bugs that make noise, such as the honeybee, long locust and loudest of all, the mole cricket, but many, many of the bugs make no noise at all. So yes, this is a bit more difficult a problem to solve. What we’ve come up with isn’t technically audio related, but I’ll mention it here for the curious. The strategy for a blind person is to section off an area of your island where you plant many flowers, and leave many tree stumps. Bugs will often come to these places, making it technically possible to catch a couple if you go to that area and start swinging.

Now let’s talk fish. On the surface, (that’s a water pun), fish wouldn’t be easy for a blind person to catch. The instructions for catching fish specifically say that you have to cast your line in front of the shadow of a fish. And no, fish do not make sound when they’re in the water. However, if you use fish bate, it actually summons a fish right to you, or very very near you. And as it happens, the clams you use to make fish bate do make sounds when they’re underneath the sand. The tiny little water spurts they give off are all the indication we need to find them and dig them up. Again, sometimes this takes a few tries, as we don’t always hit the correct spot exactly, but it is certainly doable.

There are a few interesting audio notes about fish, though. While they don’t make sound while swimming, a fish interested in your line does make a series of sounds. One for nudging your bobber, another much larger sound for when they latch on and get hooked, and finally the sound of you reeling them in. This last one is the most interesting, though, and here’s why. It is actually different depending on the size of the fish you’ve hooked. Little splashes for little fish, big splashes for, well, big fish. So, to fish, we throw out our fish bate, then cast our line a few times around us until we hear the interest sound. We then wait for the hooked sound, and one press of A pulls the fish right in. It’s unfortunate that we basically can’t fish without bate, but at least we can do it at all.
So let’s talk about fossils. Unfortunately, fossils are very, very difficult for the blind to dig up, as the appropriate dig spots make no sound whatsoever. Therefore, I don’t have much to say about them, at least when it comes to digging for them on your own. However, the next topic will cover a solution for even this problem.

Next up in this compilation of auditory awesomeness, interacting with friends online. I couldn’t ask for this to be any better without asking for things like chat narration via text to speech, which I don’t think is very likely. At any rate, interacting with friends online is as easy for your friends as it is for you. You can hear every single thing anyone on your island does. Not omnipotently, of course, but the distance from which you can hear things is very long. You might think this isn’t a big deal, but many many games, even online ones, wouldn’t play sound for things your friends do, even if they’re near you, unless they directly affect you or your enemies. Firing a weapon, for instance, but not playing a sound for equipping a new one. Well, in Animal Crossing New Horizons, you can hear… everything! If your online buddy pulls out a different tool, you can hear that. If they shake a tree and pick up the branch and/or the fruit from it, you can hear all of that. Everything they can do, you can hear them doing it. This is especially helpful if your friend is helping you with the previously-mentioned fossil collection. Since fossils don’t make noise, just have a friend hop on, find a fossil spot, stand near it, and clap. You’ll be able to hear it from far away, track it, and then dig up your fossil. Done and done.

I’ve also used this trick with friends that have agreed to give me things. They’ll stand near me and drop items. I can hear them drop so I go pick them up, but if for whatever reason I miss one, they’ll stand at the spot where they dropped it and perform any of the many emotes in the game, all of which also have unique sounds, by the way. It’s a system that allows for true online companionship, even if both players are blind, except of course for the fossil example in which case one must be sighted. But if I want to hop over to a blind friend’s island and drop him some fruit, that’s no problem at all.

Now we move into our final segment, which I will lovingly refer to as the crazy segment. Consider all the things I’ve already told you. Consider how awesome those things are, and now prepare yourselves to be amazed all over again. Did you know that, if I listen to someone walk in Animal Crossing: New Horizons, I can tell whether or not they’ve changed their shoes from the default? Yeah. All those terrain sounds exist, but they change depending on what kind of shoes you’re wearing. Nope, I’m not kidding at all. I don’t have these sounds memorized personally, so I couldn’t currently tell you which shoe was which, but I have definitely witnessed this happening for myself. It’s just stunning.

And speaking of stunning, I’m still not done. Let’s talk about rain. When it’s raining in Animal Crossing: New Horizons, everything changes. Yes, everything. The sound of wind running through trees becomes different as the rain also pelts them. The sound of things dropping to the ground changes as the ground is soggier, your footsteps change for the same reason, (and trust me stepping off a plane onto an island where it’s raining and walking across a wet dock is very noticeable), and that’s still not all! If you’re inside a tent or building, you can hear rain hitting the roof. But even better, you can hear this from the outside too, meaning that buildings and tents are technically easier to find in the rain! I feel like I need to say again that I am not kidding. This is truly amazing stuff.

So there you have it. This doesn’t answer all of a sighted person’s questions about Animal Crossing, (I didn’t go into detail on how we use optical character recognition for instance), but it does answer all the audio-centric questions. Next time you visit your New Horizons island, maybe close your eyes for a second and just listen. See what you can figure out. Animal Crossing has outdone the audio design of many, many triple A games out there, and I want everyone to hear it.

Why Being a Blind Gamer is Better

Over the course of my blog, I’ve talked about a lot of things. I’ve talked about the struggles, and the successes of being a blind gamer, I’ve talked about accessibility and how awesome it is sometimes, and where it could improve other times. Through all that, though, I haven’t revealed one of blind gaming’s biggest secrets. I haven’t explained why being a blind gamer is, in fact, better. I haven’t covered the hidden benefits. That’s what I’m going to do for you now. Prepare yourselves, because these are things many people don’t even consider until they witness it, or until we bring it up. Here goes!

Who needs a TV? No, seriously, who needs one? The answer, of course, is you sighted gamers. You’ve gotta have your polished graphics, and your 4K resolution. You’ve gotta have your HDR colors, and oooo those water effects! We blind gamers… We need none of those things. Why, almost every single day I stream those fancy console games I play, our TV is completely off. This is the one that really gets a lot of sighted people. My dad’s reaction was especially memorable when he walked in from work, heard game audio, and saw nothing. His brain didn’t know how to process it, so in a way it was almost like he got mad at me for not having the TV on. Then he actually thought about it, and it got funnier. Why bother turning that pesky TV on? All I need is the sound! This brings me to my next point.

Blind gamers can get cheaper hotel rooms! Of course, you don’t wanna go too cheap here. After all, you really don’t want a bedbug coming home in your bag. But hey, if one of the features of a hotel room you’re looking into is a 2000-inch TV, maybe you could scale back a bit. After all, you’re a blind gamer. Bring your console, bring your headset, plug into power, and you’re golden! Or another possibility if you don’t mind a little latancy, and if the wifi is good enough, and if your console is a PS4, just bring your laptop, and a controller, and use remote play! You can even afford to turn visual quality down a bit to ensure you can connect, because again, who needs graphics? 😊

Every console is practically mobile! Since you don’t need a TV, you can game wherever there’s a power outlet. Pro tip, this world contains many power outlets. If I could fit my PS4 into a carry-on bag, I could Playstation on a plane! Yeah, I know the Switch can do that, but we blind gamers, we awesome, fantastic, amazing blind gamers, are the only ones who can PS4 or even Xbox on a plane. This of course doesn’t take their size into account. You probably couldn’t actually do this, because both consoles are pretty large, and you couldn’t fit much else into a carry-on if you put one in, but in terms of mechanics once you got one onto a plane, you could totally do it. So clearly I’m driving the no TV thing into the ground, but to be fair, it’s pretty awesome. I actually have a friend who simply doesn’t have a TV, but owns and plays both a PS4 and Xbox One. But that’s not the only blind gamer benefit.

We can game long range! Both the PS4 and Xbox One’s controllers have the ability to route all game audio through the controller, and through a headset you connect to that controller. They also have surprisingly long wireless ranges, which most folks have no reason to take advantage of. You know, because they need to see the screen and all. Well, we are not so restricted. We can hook a headset up to our controller, launch a game, and take said controller out to, say, the porch swing. Ah, a nice relaxing gaming session far, far away from the console we’re gaming on. Feeling the sun on your face as you perform a gruesome fatality in mortal kombat, hearing the chirp of the birds as you take down a few more zombies in Resident Evil 6, these are the pleasures we blind gamers can enjoy. Now, I hear you again saying, “But, Nintendo Switch!” Sure, but both PS4 and Xbox One controllers, in my opinion, have superior battery life, and facing facts, the Switch is still a significant power level down from both of them. Still, this does lead to my next point.

Finally, finally, I’m actually going to talk about the Switch in a positive light, in order to demonstrate the fact that we blind gamers are potentially far more forgiving to ports of games. Mortal Kombat 11 is my example here, having just recently played the Switch version. To me, the port is essentially perfect. Yes, I notice the slowdown in transitional areas such as the boss fight, and I notice the bit of chop between gameplay and story cut scenes, but those are the only 2 things I knock off of it. Meanwhile, a review I heard on the Switch version suggested that it was so bad because of the scaled down graphics, especially in the portrayal of the crypt, that you definitely should not ever get the Switch version ever unless you don’t have any other console. Wow, that’s harsh. But guess what? I, and blind gamers everywhere, don’t care much about that, because they didn’t mess with the audio! I will say that I noticed very, very light compression, but we’re not talking MK9 on the vita here. The audio was still crystal clear, and as beautiful and savage as it is on every other console. Sounds like a solid port to me, and one I’ll be glad to take on the go. Ah, being a blind gamer rules!

Before I close, I want to be sure you understand that this post is all in fun. There are certainly benefits to being a blind gamer, and I think I’ve outlined them pretty well, but of course the sighted gamers out there have it pretty good too. Even though I can’t see them, I acknowledge there’s something to be said about game graphics being near photo realistic these days, not to mention the amount of games sighted gamers can play dwarfs those we can. Still, I had a lot of fun writing this, and I hope you enjoyed reading it. Continue to be awesome!

The Constant Rise of Our Standards: We are Funny Folk

This blog is not quite a gamebreak, but it is an accessibility break. I was recently struck by an interesting thought, and I wanted to blog about it. We are an interesting people. We do a lot of interesting things. In particular, and the subject of this blog, is the way we raise our standards, but also keep them in check in a way. I’ll explain what I mean.

Have you ever heard the phraise “movie quality graphics” applied to video games? I’ll bet you have, and in fact I’ll bet you’ve heard it several times over several years. You want to know how far back I heard that phraise? I heard it in a description of Mortal Kombat 1. Yes, the original Mortal Kombat, with its revolutionary use of digitized actors and such. Mortal Kombat 1, whose entire arcade imprint was only a hundred megabytes or so. Movie quality graphics.

That’s not actually me ragging on MK1, or the quality of its graphics. My point here is that we never, ever stop raising our standards. If we did, one would have to wonder exactly what movie quality graphics are these days. Follow the trail of graphically praised games, and you’ll see phraises like “movie quality graphics” or the word photorealistic used quite often. Yet, graphics keep improving. Games keep expanding, and the systems that run them keep getting more powerful.

This is what I meant, though, when I talked about how we keep our standards in check in a sense. We all know that progress is happening all the time, yet we are willing to hoist games on a pedestal that, quite honestly, many may not actually deserve. I genuinely think this is because we have a sort of maximum expectation. We believe in our hearts that a video game can only look and sound so good, so when it looks as good or sounds as good as we believe it possibly can, we hold it up to the highest height, only to bestow the exact same praise on the next game.

And speaking of sound, don’t worry, we blind folks aren’t immune to this either. I remember listening to the trailer for Mortal Kombat Deception and thinking, “Man! That sounds real!” The game didn’t sound as good as the trailer, but Mortal Kombat X, the most recent MK game, sounds far, far better than that trailer ever did. Yet still, at the time I was utterly convinced that this was it. My first reaction to hearing the fully voiced cutscenes and full motion videos of Final Fantasy X was that I was certain the game would be short, because there’s no way the PS2 can handle all that. I was glad to be wrong.

Again, I’m not trying to send a specific message with this blog. This sequence of thoughts that I’ve laid out on this page is simply something that intrigued me, and I hope it intrigues you too. We are funny folk, aren’t we? Thanks for reading, keep on gaming, and continue to be awesome!

Commentary: The Most Undocumented Accessibility Feature

Continuing the sports theme from my previous blog about MLB, today I want to talk about commentary. These days it is a standard part of any sports game, and has been since the N64 era, but today we’re going to talk about what it is in the context of blind gaming. That’s right, kids, it’s an accessibility feature. “What?” the sighted readers gasp. “It is? But… But how? It wasn’t made just for the blind!” True, but it serves that purpose for us. Let’s discuss.

One of the primary themes I use when discussing the challenge of blind accessibility with game developers is information. In most cases, blind accessibility involves figuring out what information we don’t have, and then figuring out how we’re going to get it. That is a simplistic explanation, but I think the principle holds up pretty well most times. While, to the sighted player, commetary is just a part of the presentation that makes their sports game more immersive, to us it is a source of information. Think of all the things commetary tells you these days. In Football and Baseball, it gives you a real understanding of how a play is going. Yes, that’s what play-by-play commentary is supposed to do, but the point is just that it does so, and it is a feature that can be turned off.

Play-by-play aside, commentary in newer sports titles takes it a step further, giving us access to information we simply didn’t have in the old days, like player and team stats. How cool is it to get an audio rundown of how a player is doing so far this year, or how they did last year, or find out what the team’s schedule is for the next week? Trust me, if you can’t see it, it’s pretty freaking cool.

Unfortunately, not all commentary is good commentary. The WWE2K series, for example, has managed to do something truly amazing. They have actually managed to make their commentary worse with every passing year. Do you know which game has better, more helpful commentary than, let’s say, WWE2K18? I’ll tell you. WWF Warzone for the Nintendo 64 and Playstation 1 has better commentary. It’s completely true.

The problem here is that 2K chose to focus the commentary on the side banter that sometimes happens during a wrestling match rather than the actual wrestling. What comments are made about the match are simplistic and unhelpful, like “Oh that was a great strike there.” If I’m not the one playing, or if I’m involved in a match with several wrestlers in the ring at once, I have no idea who threw this great strike, or who just did that incredible reversal… Sometimes I don’t even know who performed a finishing move, because they only say something like, “And there it is!” There it is indeed.

The banter itself is not necessarily the issue. I don’t actually dislike the banter, as it really is a part of wrestling’s presentation. The commentary is so focused on that, though, that there is no discussion of the actual moves, or of the wrestlers’ progress beyond the existence of rivalries, or their most recent win or loss.

The reason that Warzone, or its sequel Attitude have better commentary is because it is focused inward on the match currently taking place. They talk about the moves, they talk when someone is out on their feet, they have loads of responses to in-match events. These are games from 1997 or so, but I’d take their commentary any day.

The thing is, it’s appalling that the WWE2K series has such bad commentary. Last year, 2K went on and on about the auditory overhaul the game was getting. They also said the game would now be using the commentary engine from the NBA2K series. There is exactly 0 evidence that this was actually done. If you’re reading this, and are unfamiliar with either of these game franchises, do me a favor. Take a second and look for a gameplay video of NBA2K18. Listen to that commentary. Listen to how it flows in almost a natural way, and how sometimes small audio files are combined to form whole sentences. When listing some stats, for instance, they’ll have a basic sentence structure, filled in with the correct numbers for that player’s actual stats. All of this flows seamlessly as if it were one. You actually have to be listening for the breaks, or you won’t even notice them. Listen to that. Then find a gameplay video of WWE2K18, and listen to that mess. If you’ve done that, please comment and tell me if you believe WWE2K is using the same backend commentary system, because I sure don’t.

Ultimately, this article is not about wrestling games, though I think they served very well to demonstrate the point. This article is about commentary in general, and why it is so important to us blind gamers. Good commentary is so often overlooked by critics. Almost every sports game review I’ve watched just criticizes the commentary of any sports game, even the ones with amazing commentary, saying it’s repetitive. Of course it’s repetitive. The announcers who did the voiceover work to create the commentary for these games only spent a finite number of hours recording that commentary, and thus could only create a finite number of total responses. Game critics seem to think that commentary should be done live by the real announcers in realtime for every single game that is being played by a human being. It’s a ridiculous argument.

It doesn’t matter that we occasionally hear the same messages over and over again. In a game with good commentary, (MLB: The Show is another example of this), the important thing is that the messages are there in the first place. They give us much of the information we need, and thus, by the very existence of commentary, sports games become more accessible. As always, feel free to comment here or on Twitter or Facebook with any feedback you have, and thanks for reading. Continue to be awesome!

Game Accessibility is Happening

The feeling going into the first ever Game Accessibility Conference was a positive one, yet I can honestly say that I still wasn’t completely sure what to expect. How was this going to go? Would people really listen? Would they care? Those are harsh questions, but given the difficulty of making our wish for accessibility known in the past, they were legitimate ones. After all, I was once sent a form letter by THQ in response to some requests I made about their Smackdown wrestling games. The letter thanked me for my appreciation of their stunning graphics. Yeah, seriously.

This conference, though, was not that. It was so much more. For my general readership, keep in mind that this conference was about gaming with all types of disabilities. Blindness, deafness, those who require one-switch controls, even discussions about using VR while in a wheelchair. And the best part is, the conference was full of those who not only listened, not only cared, but kept an open mind, and looked to be inspired. I feel that everyone there wanted to know exactly how they could help make this work, and those who already knew were more than willing to impart that knowledge. I cannot describe how that made me feel.

The world is beginning to change. Accessibility is now understood to a far greater extent, and disabled communities all over the world are beginning to be recognized as gamers, just like everyone else. Of course, there are those who have advocated for disabled gamers for years, such as the Ablegamers foundation, but this conference represents a whole new level of recognition, acceptance, and willingness to find solutions, in my opinion.

I’m happy to report that my speech, which centered of course on video gaming from a blind gamer’s perspective, was extremely well-received, and that I was approached by many, many people afterward to talk about the possibility of blind accessibility for them. That, ladies and gentlemen, felt great. Even when I was at GDC in 2014, even though I was pretty well received there, and even though I got a lot of compliments, I also got quite a bit of negativity when I began approaching developers about accessibility. Few attempts were made to actually discuss solutions, and I was often just turned down, with the assumption that it was not possible. Not the case at all with this conference, not once.

I am writing this blog with a very specific purpose in mind. I do not want to repeat what I said in my presentation, as that will be available for all to watch. Instead, I am writing this as a followup to the conference, and as reassurance to all of my readers that all of this is real, things are really happening, and people do want to help make those things happen. It is not going to be instantaneous, but we are further along than we’ve ever been, and based on discussions I have now had, I know that we are going to keep moving forward. Games should really be for everybody, and I’ve never believed more strongly that they will be. And furthermore, I want to assure all those who read this that I will always do whatever I can to help this process along. This conference has only increased my passion for games, and I look forward to similar events in the coming years.