The First of Us: My Journey in The Last of Us 2 Blind Accessibility

“We’ve gotta talk to this guy,” said Emelia Schatz from the front row. I was on stage at the Game Accessibility Conference 2017, and though at the time I didn’t know Emelia, or EM for short, I soon would. I at least suspected who she represented, as the topic I had just been discussing was the Last of Us. I was talking about games I couldn’t play, but wished I could. I spoke of how I would give anything to play The Last of Us, and how it was a physical ache when I couldn’t play a game. That is when the infamous line was uttered, and as small a thing as it was, it’s a thing I will never forget.

They did indeed “talk to this guy,” and that conversation, which contained a bit where I told them how I would love to play a game for my sighted fiancé instead of her having to play it for me, started what would become a fantastic and beautiful working relationship. I didn’t know it then, but my pathway was set, and I had already started walking down it.

There’s a lot that I can’t say about my experience working with Naughty Dog. What I can say, though, is that it was unlike anything I could have expected. From the moment I stepped into the Naughty Dog studios for the first time, I felt welcomed. I felt like I belonged there. And that was before I knew for sure that I would really be working with them. This was just a conversation. This was my attempt to sell myself to this studio. I knew it was a huge opportunity, but it was on me to make it work. Could I do it?

Well yeah, I guess I did. I ended up working on the Last of Us 2 for 3 years, both in and out of the studio. My brain never stopped. I was always coming up with things. Things I simply had to get to the team as quickly as possible! I would often write emails that were both really long, but also really frantic, full of walls of text about how we should look into trying this or that thing. From the moment I started, I was all in. I was passionately pursuing this idea that we COULD make this work. I wouldn’t allow us to be stopped. This was going to be the one! I’m doing this!

But I was never alone. First, the passion and dedication of the Naughty Dog team themselves was unbelievable. They were just as all in as I was. I would often speak to them encouragingly, both in person and in emails, and I did this at first because I thought I had to. I thought it was on me to keep them on the accessibility train, encouraging them at every turn so they wouldn’t give up and say “Nope, all this is too much. Can’t do it. Done.” I was THAT terrified at first. But after a while, I started to see that they really, really did want to do this. I saw that I had already won. I saw that there was no turning back for them either. I kept encouraging them after this, but more just because I was saying what I felt, not because I was worried they’d just stop.

Secondly, this game would not be as accessible as it is right now if it hadn’t been for Misty, my wonderful fiancé. Working on a game like this from a blind accessibility perspective was difficult, and it was difficult for the same reason that playing it will be difficult for some blind people for a while. A game like this requires a tremendous mental shift. This is not your audio games, developed by only a few people. This is a huge, huge game, developed by a very large team. Because of this, this game contains massive levels, tremendous amounts of verticality, different ways to get into places, multiple options and approaches for each encounter, an insane number of mechanics and systems, and so on and so on.

I was not immune to needing this shift, and Misty was the one who helped me get there. Most audio games, for instance, require you to kill basically every enemy you encounter. You grow up on audio games and some playable mainstream games, you get used to that idea. But even having watched a zillion playthroughs of the first game, somehow it didn’t stick in my head that I didn’t need to kill everything, and that there were other options I could try for. That’s where Misty came in. Well, that, and keeping me focused on talking to the team instead of just playing the game the whole time. Her input was truly invaluable in helping shape this experience.

So, in case you didn’t know, we did it. 3 years of intense work, and the Last of Us 2 has achieved total blind accessibility. It is my utterly tremendous honor to be a part of that, and not just because it is THIS game. Not even because I longed to play the Last of Us at GAConf all those years ago. The Last of Us 2 is going to stand forever as the first huge, triple A game to embrace and fully support blind accessibility, and I will always, always be able to say that I helped make that happen. That I am in fact largely responsible for that happening. Of course, this game breaks down more barriers than blindness, providing accessibility for tons of disabilities, and for that I look to my other consultants. Steve Saylor, Paul Lane, James Wrath, and even folks I never met like Morgan Baker. All of us did this together. We became a team, and we made something that will be recognized for years to come. And let me tell you something, dear readers. I’m not stopping anytime soon. The Last of Us part 2 is not the last of anything. It is, in fact, the first.

The Accessible Future of Emulation

For the longest time, I have believed that the emulation of games is a bad, bad thing. If you emulate a game, you are doing wrong. You are stealing directly from the mouths of the hungry children of the folks who created it. But what if, just what if, a game could achieve near full blind accessibility through the use of an emulator? What if a game that presents the blind with some serious difficulties in its original form, a game to which no accessibility features are ever expected to be added, is emulated through a particular bit of software that is built with blind accessibility in mind, and thus adds some of those missing features to the game? That is the question we are discussing today, and boy oh boy is it an interesting one.

First, let’s talk about the software in question. After all, you, dear reader, might not know about it. The emulator we’re speaking about, for yes it is in fact real, is called Retroarch. This emulator has been modified to be self-voicing so the blind can interact with its menus, but the real big deal is its ability to read game text. And we’re not talking about just OCR here. I mean, we are talking about OCR, but OCR controlled by AI, as I understand it, which is smart enough to avoid giving you a dump of the entire screen, and instead just gives you what you need when you need it. This doesn’t magically solve every accessibility issue for every game, but it is staggeringly huge nonetheless, and there are certainly games out there that could be made entirely accessible using this kind of technology. When I listened to their video demonstration, which shows a player playing Dessidia Final Fantasy with these features active, I was blown away! I had played this game, even managed to complete it before, but that came with the understanding that I would be skipping a bunch of story text. I mean a bunch! That didn’t happen here, as all the text the player encountered was spoken clearly to them. I couldn’t help it. The part of me that just loves video games, and loves being a gamer, and loves playing whatever games I can, along with the part of me that appreciates effective new technology, just fell in love with this. I wanted the question we are here to discuss not to exist, and I wanted to not feel the guilt I was feeling, because I wanted to dive deeply into the sea of old games made new to the blind by accessibility. But the question is the reason we’re here, so let’s get to that.

I knew the best approach here was the simplest one. Wanna know how game devs feel about emulation? Ask the game devs. So I tweeted my question, asking if emulation would be OK if doing so made a game more accessible. I only got 1 response, but it’s one that may surprise you. It comes from former game developer Drew Thaler, and here is what he had to say. “Unofficially every individual game developer I know loves emulation. It’s great for history, great for speed runs and other enthusiasts, keeps franchises alive, etc. If it delivers accessibility too, awesome! Nobody’s making money off consoles that are old enough to be emulated.”

That wasn’t the response I expected, but boy was it a welcome one. It was, in fact, just what I needed. I have been so trained to view emulation as negative that, even in spite of the reaction my gamer and tech brains had, I was ready to simply never use this software. Now I see that it’s not only OK, but developers of these older games may actually even appreciate it. After all, many developers make games because, get this, they want people to play them. All that said, I do still believe that there need to be some considerations. The focus of our fight for game accessibility should still remain on working with developers to make future games accessible, and to promote that idea, I am still against emulating newer games. Developers need to know that we want to work with them, not just find a workaround. And if we do need to find a workaround for some games, it shouldn’t be this one. If a game is past its prime, though, if it is absolutely no longer being supported, and never had blind accessibility features to begin with, and isn’t making the developer money anymore, well then I no longer see harm in bringing the experience to the totally blind in an even better way with the accessibility features of Retroarch. So my gamer brain and my tech brain can now, officially, rejoice and say “Alright! Bring it on, Retroarch! I am ready to play!!”

To the Audio Teams: The Art of Sound

Dear video game audio designers, producers, mixers, editors, engineers, technicians, quality analysts, composers, and any audio-related field I forgot to mention,
This letter from a blind gamer goes out to all of you. I can’t speak for everyone, but I’m pretty sure I can speak for most blind gamers out there when I say thank you. Thank you for putting in the time, and making the effort. Thank you for adding those little extra touches that we notice all the time. The things that a sighted player doesn’t necessarily need, but that add to the audio experience. Thank you for your own desire to make games sound as alive and vibrant as possible. Thank you for immersing yourselves in the material so the experience you create matches the story being told. Thank you for being awesome.

We blind gamers bow before you, audio teams. Without you, we couldn’t do what we do. Without those little touches, in some cases, we would be further hindered in our efforts. I’ve said before that blind accessibility is all about information, and the things you guys do with audio give us that information, sometimes whether or not you’re aware of it. I commonly use the example of Kingdom Hearts, where equipping different keychains causes Saura’s footstep and attack sounds to change based on which one is equipped. It doesn’t have to be this way, but it is thanks to an audio team that cared to make it so. Because of that, we can tell you by taking a few swings what we have equipped, and from there branch out into how much damage it does, what bonuses it has, and so on. The importance of these little things cannot be underestimated.

But let’s go further. Things get crazier when we consider things that have ambient sound, but don’t necessarily need to. The Death’s Breath crafting component in Diablo 3, save points and treasure spheres in Final Fantasy 13, all of these things and more are examples of necessary objects that we can now locate within a certain distance thanks to audio. Without those ambient sounds, they would just be lost to us. That’s huge! It’s so simple and easy for you audio guys to do, but it’s so, so very helpful.

But really, we’re only scratching the surface here. Let’s go even deeper, and discuss the crazy things audio teams are doing these days, like accurate surround sound positioning and even 3D audio in some cases, realistic echoes based on the objects sound is bouncing off of, (often used to indicate distance between, say, you and a companion), and even dimming of sounds that are blocked by an object such as a wall. The tremendous amount of code that must take is mindblowing, and the fact that you guys work to perfect it, to make it sound right, is inspiring. Let me just tell you now, in case you had any doubt, that it’s worth it.

Now we can’t talk about audio without talking about music. Music is a huge, huge part of creating an atmosphere in a game. I once spoke directly with Austin Wintory, and told him that the music in Journey is what really told me the story. It was true. The music in that game is powerful. It connected with me on a deep, deep level. It made me feel every emotion right along with my fiancé, who was actually the one playing it. I felt the joy of flying for the first time, I felt the rising tension as we got ever closer to our goal… It was an astounding feat of scoring for which Mr. Wintory has received much praise, but if you ask me, not enough. As I’ve said, I wasn’t even the one playing Journey, but when it was all said and done, I felt as though I had. I cannot think of another game that has achieved that effect on me personally.

That is not to say that I don’t love other game music, because I do. There’s a really old game called Stonekeep, which was one of the first games I ever owned. During one part of that game, you wander from the relatively dangerous, though not-too-difficult sewers beneath Stonekeep into the lair of a monstrous evil creature that you, at some point, must defeat in order to proceed. The brilliance of this is that the creature does not immediately attack you. It is, in fact, the music, which changes drastically and ominously the moment you step into the lair, that alerts you to be prepared if you plan on moving forward. After all, the lair itself is not large, and should you actually proceed despite the warning the music provides, then the additional warnings provided by the piles of bones in the area, you will find the beast lerking just a couple corners away. It’s a wonderful, powerful moment in the game, and for me, mostly because of the music.

There are plenty of examples of moments like that in games, but I particularly like that one. The power of music is, I think, becoming more respected these days. I feel like people are taking greater care with music, and that is very much appreciated. Look at, for instance, what Crystal Dynamics did with the first Tomb Raider reboot, scoring every single encounter differently, and using a dynamic music system to make it all flow depending on what you did. That is… That is awesome! There is just no other word for it. Even Killer Instinct for the Xbox One does awesome music tricks, picking up the background music depending on the size of the combos you’re doing. What better way to make you feel good about pulling off an amazing feat of fighting game excellence than to deliver the rousing chorus of that level’s music? It fits perfectly, and it’s also awesome.

I feel as though I could sit here and write about game audio for hours. However, I think that what it boils down to is this. Audio guys, you are our lighting. You are our graphics. You are our art. You are our atmosphere. You are our information. Sometimes, you are our story. You are essential. You are needed. And most of all, you are awesome. Thanks for reading. As always, consider supporting this content if you can, however you can, and continue to be awesome!

Blind Accessibility: Past and Present

Anyone who is a gamer knows how much video games have changed over the years, and not just the games themselves, but the perception of them. This applies to the blind as well, as our perception of game accessibility, what is playable and what isn’t, has also changed. I’d just like to take a moment to reflect and discuss those differences with you now. Let’s see what you think.

Accessibility in video games has become, as the years have gone on, both more complicated, and yet more simple as well. More complicated because the complexity of games is much, much greater than it used to be, and the implementation of accessibility features would involve the writing and/or rewriting of a whole lot more code. At the same time, though, it has become more simplistic because the possibilities of what can be implemented have expanded. Game companies have drastically increased in size, and games are now developed by teams that can go from small groups to well over a hundred people.

And that’s not all. The fact is, accessibility in retro games simply wasn’t a thing that was considered. Not just for blindness, but for anyone. If there was a game you couldn’t play, that was the end of it. There were no patches or fixes you could hope for. The release of a game was the final product, and that was that.

It’s interesting to think back on how all of these things have changed. In the case of blindness, for instance, we knew to accept that there were things we could play, and things we couldn’t, and nothing could be done about the ones we couldn’t. We had dreams, we had frustration, but we also had acceptance. For this reason, some of what I’ve mentioned above didn’t actually matter to us, because that was the way things were.

After all, it’s difficult to argue the fact that the accessibility threshold was a bit lower, at least for the blind. For instance, most very old games didn’t even have a menu. Old consoles didn’t even have an interface of their own. You pop in the cartridge, you turn on your console, you begin playing. If there was a menu option, it was almost always 1 player or 2. That alone eliminates some of the accessibility problems we face today. No menu memorization, no concern over whether a console has text to speech or how good it is, no store to purchase DLC from… It was a simpler time.

Games themselves were more limited in scope back then as well. This is interesting because it meant that a game was usually completely playable by a blind person, or completely not playable. A fighting game, for instance, didn’t even have an in-game move list back then, and although fighters today almost always fall under the playable category, we still don’t have access to those features in most of them. Back in the old days, though, we had just as much access to a fighter as anyone, as a sighted person couldn’t look things up easily either.

There is also an interesting difference in what we had to learn in order to play a game versus what we have to learn now. In older games, there weren’t as many sound effects used. This was good in that it took us a shorter time to learn what each sound meant, but it was also bad. Less sound effects meant fewer indicators for events. This was made worse in games that used the same sound for multiple things. Older fighters, for instance, used the same voice audio for all male characters, and only used a different one if there was a female character. We had to use other things, such as the sounds of certain moves, to determine which fighter was actually present.

Now, there is so much sound in games. This, too, is a good and bad thing. More sound means more work learning each individual one and what it means. This usually takes much longer now as there are so many sounds in the game that it is possible not to notice a particularly helpful one right away. For example, it took me some time to realize that, in Kingdom Hearts, Saura’s footsteps change depending on what keyblade he’s wielding, but once I figured that out, it was immensely helpful.

The fact that the footstep sound difference I just mentioned even exists, though, is a great example of why more sound is a good thing. As long as we can figure things out, there is loads of information available via our ears, some of which the sighted community doesn’t even notice. This is why today’s audio designers love us. We tend to catch the little details that exist, even though there are so many of them.

Today’s sound goes beyond sounds themselves, though. These days, with technologies like surround sound and even 3D audio, we get even more information. We can tell where something or someone is based on the positioning of its sound in our headphones or speakers. Most older games played all their sound from a single speaker until around the mid 90’s.

I’m not really trying to make a point with this blog. This struck me as an interesting topic because of how different things were than they are now, and I thought it would be fun for you guys to think about it too as we move forward into a new age of accessibility. Yes, accessibility takes a lot more work these days than it might have if we had accessibility features back then, but it’s still happening. Developers are starting to communicate with the disabled community, and they’re starting to listen. It’s a wonderful, magical time, but the past, the way things were in the NES and SNES days, had a sort of magic all its own as well. I hope all of this made sense, and I hope as well that it got you thinking. As always, feel free to comment, and check out the support heading for ways in which you can, well, support this content. Thanks for reading, and continue to be awesome!

The Pain of Inaccessibility

This article isn’t for my blind readers, unless they are looking for something to relate to. This article is more for those game developers out there, or folks in the industry, and is meant to bring home a point that cannot be driven home enough. It’s a point I have personally made in many speeches, and even conversations with other developers. We are gamers, and we want to game. I am not writing this seeking sympathy, as it may appear I am. I am only writing it in an attempt to bring understanding to those who may not yet see why we keep fighting for accessibility. Please keep these things in mind as you read.

At midnight on May 25, 2018, I heard a cheerful little chime through my headset. It was a notification sound that told me Detroit, Become Human was ready to start. At this, I was filled with unmistakable delight. It was here! After 5 years, Detroit was finally here! Here at last was a game I have anticipated more and more with each passing year, following all the news and the hype because I had a great deal of respect for its developer thanks to their previous games. And now, it was ready to play! With one press of a button, I could… But that’s where my excitement ends.

Moments in your mind are speedy little things. So much can be contained within them, and yet they pass in no time at all. Just as I soared with happiness at the arrival of this game I had been long-awaiting, I then drowned in frustration immediately afterward. The thing is, I knew I couldn’t play this game. I knew already that it was inaccessible to the blind.

I knew this for several reasons. First, I had done my homework. I learned that, mechanically, it functioned like Quantech Dream’s other games, meaning you moved with the left stick, and used the right stick to perform most actions. There are also moments a blind person cannot anticipate where the method of performing an action unexpectedly changes. A moment in Heavy Rain has you brushing the teeth of one of the main characters, and this is done by shaking the controller back and forth instead of using a thumbstick. There is, of course, no audio indication that we must do this.

The second reason I knew the game was inaccessible is that I tried the demo that came out the previous week. Though I did manage to get through the scene that you play out, I got what I believe is the worst possible ending. The fact is, I didn’t know what choices I was making, and didn’t know where interactable objects were. It was an investigation in which I found absolutely 0 clues. That complete disconnection from a game can hardly be called playing it, in my humble opinion. But you see, none of this is the point. This is, if you can believe it, not a rant about Detroit’s inaccessibility.

The next question you should be asking is this. If I knew the game was totally inaccessible to the blind, if I knew I couldn’t possibly play it in a million years, why did I wait up for that little notification chime? Why did I experience that rush of delight? The answer is quite simple, and yet quite complex as well. It is because I am a gamer, and because I LOVE games. I love developers who have consistently demonstrated the ability to enthrall their players with an amazing story. I LOVE great sound design. I LOVE great voice acting. I LOVE a great musical score. I loved the premis of this game when I first heard of it. I loved the commitment by Quantech Dream to go back to their roots, and make a game where the choices once again really mattered. I loved how this game sprung up from something that was just a tech demo many, many years ago. In short, I already loved everything about this game. My love of games is independent of whether or not they are accessible. That is the point.

“OK then,” you say with a small nod. “Why not just watch a playthrough? Why spend your time blogging about your frustrations when thousands of people have probably uploaded millions of videos of this game to Youtube?” You are right. I could absolutely do that. In some cases, I have. I deeply respect Let’s Players, especially those who know their audience. The problem with a Let’s Play, though, is that it will always be someone else’s experience. A game like Detroit, in my opinion, should not be experienced that way first. A game that relies as heavily on player choice as Detroit should be experienced on an individual level if possible, and in an interactive sort of way if not. And that’s where my fiance comes in.

For those who do not know her, which is some, but not all of my readership, my fiance is a wonderful but very busy woman. She’s a concert photographer, which means that when she isn’t out taking incredible shots of rock bands, she’s here editing those photos and sending them on their way. I do not begrudge her for the time this takes, and I support her every endeavor. And yet, to be perfectly, brutally honest, there is a part of me that would like to tear her away from her work and somehow force her to play Detroit. If she did, I know that she would allow me to make many of the game’s choices, thus making the experience partially mine as well as hers. This, while not ideal, (of course I’d still love to be able to play the game myself), would be acceptable.

That, however, would be a selfish act, and I am not by nature a selfish person. And so, I am left aching. I ache for a game I love but cannot play myself. I hope my fiance finds some freetime during which we both can enjoy the game. I’ve said this before, but it is sometimes nearly a physical ache. Yet, it changes nothing. I still love games, and I always will. I will love this game even if, for some crazy reason, I never get to play through it. Games are in my blood.

I’m going to reiterate again that I am not seeking pitty. Many of us blind gamers can relate to this experience, and some do not have the option of a fallback sighted person to do a playthrough with their involvement. It would be harder still for those people. As I’ve said, this article is primarily for everyone else, and is meant to illustrate the ultimate truth of the fight for accessibility. We love games, and we want to play them too.