The Platinum Wireless Headset and 3D Audio: My Uncharted Adventure

When I first heard that the Playstation 4 Platinum Wireless Headset would support not only 7.1 surround sound, but true 3D audio, I knew I had to check it out. The potential for 3D audio in video games, especially for the blind, is staggering, and I wanted to see if playstation could make it a reality. What follows is a general review of the headset itself, along with my experience with its 3D audio feature in Uncharted 4.

I liked the feel of the headset as soon as I pulled it from its padded box. It’s not too heavy, with large ear cups to ensure the best sound possible, and even before you actually wear it, the feel of the padding lets you know it’ll be comfortable. This impression was proven accurate when I put it on for the first time. It is comfortable, and light enough while on your head that you don’t feel weighed down by a bulky piece of equipment. I was already looking forward to this.

The controls turned out to be surprisingly easy to locate, and to use. Though each control is the same shape, they are all well-separated from one another, and each is tactily different. The mute button for the microphone, for instance, is a little more inset than the rest of the buttons. The The power switch, which doubles as your switch for presets, has a couple of obvious bumps on it. And all of the controls, except for the switch that activates or deactivates virtual surround, are on the left side, making it even less confusing.

In case you manage to acquire one of these headsets, and you’re one of my primary audience, let’s go through what you’ll find starting from the bottom of the left earpiece. Nearest the bottom is a 3.5 MM jack, used with the supplied patch cable to plug the headset into any standard headphone jack. Next to that, the USB Mini port, used to charge it. Then, in order, heading to the top, we have the master volume buttons, the mute button, the power/presets switch, and the game audio/chat audio balance buttons. Down tilts the balance toward chat audio, up tilts it toward game audio. And as mentioned, the only thing you’ve got on the right is the switch for virtual surround. Up is on, down is off, though you’ll hardly need me to tell you that once you hear it.

And speaking of the sound, it’s fantastic. Sure, the first game I played was Mortal Kombat X, as anyone who follows my Youtube channel can probably guess, and that game doesn’t represent the height of surround sound, but still, it sounded great. Every hit sounded more impactful because of the bass that headset pumps out, and the surround sound did lend itself a little to the 2D fighting environment. When I tossed my opponent across the screen, I felt like I had done so. That game has great audio, and the Platinum headset made it sound even better.

And still on the topic of sound, I should mention for the visually impaired that the headset does possess some identifiable beeps and boops when you interact with it. Mute the mic and you get one beep, unmute it and you get a lower tone beep. More beeps when you press the volume and balance buttons, and some helpful power and connected tones when you flip the switch. The whole thing is very easy to use, just gotta make sure that dongle’s plugged in, and you’re off.

Now, the big moment you’ve all been waiting for. It’s time for my take on the 3D audio supported by the Platinum Headset. As of this writing, the only game that supports the headset’s 3D audio feature is Uncharted 4. Well, I haven’t yet gotten a chance to listen to someone play the other Uncharted games, (and unfortunately they aren’t accessible to the blind), but I still had to try it. So, with the help of my fantastic fiance to enable some of Uncharted 4’s famous accessibility features, (not actually intended for the blind but helpful nonetheless), I fired it up.

Let’s get the big statement out of the way first. The 3D audio works. You really can hear things above and below you, and all around you. Even in 7.1 surround, sounds tend to emanate from specific places. The 3D audio seems to really put sound all around you, which is awesome.

However, I have to say that, as awesome as it is, and as helpful as I believe it will be in games, it isn’t perfect. Initially, we had the aim setting set to toggle, which is an accessibility option that allows you to press a button once to activate aiming mode, and press it again to turn it off. While set to this mode, I attempted to locate my enemies after having presumably toggled the aiming mode on, and could not do so, even when audio told me they had to be right in front of me. I am not sure why this is, and it could even be that, as a blind gamer, there are fundamentals of shooting in a game that I don’t understand. Regardless, aiming worked a thousand times better when we switched it back to Hold, where you hold a button to aim instead. The aim assist snapped to each target for me, and I was able to take them out.

Stepping back in time a bit, though, I want to talk a bit more about the 3D audio, and how it did help me. In the beginning, you have to swim a little to locate your companion, who is working on your damaged boat. Because he calls to you at first, I was able to orient on his position. Then, by listening for the sounds of his work, I was able to find him. This in itself says a lot for the potential for 3D audio. In a truly immersive environment, and it seems with perhaps just a little in-game help, who knows what we’ll be able to do? I’ll tell ya this, even being blind, the sound of our enemies’ boat approaching from the far left, and stopping right in front of us as our companion frantically works behind us was appropriately ominous.

Flashing forward again, once we got aiming working, I was able to take out our enemies. Then, through some rather typical blind person trial and error gameplay, I was able to start the boat we leave on, (I believe it’s actually the opposition’s boat though I could be wrong), and drive away. The 3D audio didn’t really take part here, I just tried to turn regularly in hopes of avoiding objects. Left a little, then right a little, and so on. Then, something happens, (no spoilers here), that moves you to the next part of the game. Unfortunately, this is where my journey ended.

The next part of the game involves a significant amount of platforming, which even 3D audio could not help with in the slightest. After all, regular platforms, rooftops in this case I believe, don’t make noise. Now, don’t get me wrong. I’m incredibly stubborn. I tried. I tried a lot. But it was not to be.

Not wanting to be done testing the awesome 3D audio feature, I got a little help through that bit, and moved onto the next, which involved stealth. This didn’t work out much better, as it took a bunch of questions for me to figure out what I was supposed to do here. I finally figured out how to take cover against a wall that is next to you the whole time, (more on that later), and got through the first section of that. After that, though, I could progress no further without being seen. The audio just was not enough. That was the true end of my 3D audio expedition, though for the sake of playing the game blind, it really ended when I reached the platforming bit.

So let’s go back to that wall, and talk about the things the Platinum Headset’s 3D audio feature doesn’t do. As I said, it’s not perfect. Distance seems to be an issue, as although it can project certain sounds to what seems to be very far away, it seems as though there’s some kind of threshold there. If something is very far in front of you, the 3D audio may project it to sound like it’s very, very far in front of you. However, once it reaches a certain point nearer you, the sound puts it seemingly right in front. I suspect this was part of my aiming problem, though again that’s difficult to verify. My evidence for this is that the call from my companion in the beginning just seemed to be in the virtual surround field, but the agonized call of someone who saw me accidentally plunge to my death off a rooftop seemed remarkably far away.

The other thing it doesn’t do, though it makes sense as technology just isn’t quite there yet, is echo properly. There is a term that, to my knowledge, was coined by Stevie Wonder. That term is sound shadows. It is a name for the way you can hear how sound reflects off of objects around you if you listen, and it’s something a blind person uses all the time. Ever see a blind person round a corner without having touched the wall with hand or cane, and wondered how they did it? That’s how. We have trained ourselves to hear sound reflect off of the wall next to us. When that disappears, we know that wall has ended. We can use this same technique to gauge a person’s height, as obviously surrounding sound reflects off of people too. But in 3D game audio, this does not exist.

Don’t misunderstand, I don’t find this to be surprising. A system like the one that would be required to create this effect would be massive. Every single object, person, and wall would have to have their own pocket of sound-blocking that follows them around. Nevertheless, it needs to be mentioned, as true, binaural 3D recorded audio actually DOES simulate sound shadows. Listen to the 3D audio dramatization of Stephen King’s The Mist, and if you know what to listen for, you’ll know what I mean.

Because of these things, though, I was unable to tell both how far away I was from the wall I had to take cover against, let alone that there was a wall there to begin with. Had I been able to hear the sound shadows of objects around me, or the way my own jumps echoed off of those objects, I may have been able to intuit where my next platform was. I could still be wrong, and that might not always be enough as you could be jumping a great distance sometimes, but I think all this needs to be said. And hey, as difficult as I understand this would be, it’s never a bad idea to give audio designers something to strive for in the future.

In short, “it’ll sound like you’re really there” is very difficult to sell to a blind person, but again, please please don’t misunderstand, this was still a pretty great experience. I strongly believe that 3D audio could be one of the keys to blind accessibility of video games, and this was still enough to encourage that belief. In other words, it’s a great start, and I’m OK with that… For now.

Now, this is also a review of the Platinum Headset itself, and I think I’ve covered its many positives, so before I go, I must mention the one and only negative I have found. This negative technically isn’t about the physical headset. Rather, it’s about the Headset Companion app for PS4. This app, most unfortunately indeed, is completely inaccessible to the blind. Though the PS4 now possesses text to speech capability, and though that ability extends to 1 of their built-in apps as of this writing, (TV and Video if you’re curious), it does not work in the Headset Companion. Why does this matter? It matters because developers have programmed presets for their games, which can only be downloaded with this app, and those presets change the way those games sound to exactly what the developers wanted them to sound like. To a blind person who cares very, very much about game audio, this is a huge deal, and a huge disappointment that we cannot use it.

But, as most accessibility disappointments do, this only makes me want to strive harder for accessibility, so I will. I will focus on all the positives for now, (the headset is truly great, and hey, I got past the bit with the boat in Uncharted 4), and I will do what I can to make those negatives go away. I’m not going to give this thing a review score, as it’s also a bit of a blog post, but no matter what you came here for, I hope you got something out of it. Thanks very much for reading, game on, and continue to be awesome!

GDC: The Panels

So I’ve been thinking. Why do just 3 blogs about GDC when you can do 4? I said before i’d talk about my panel, the people I met, and the awards ceremony, but then I thought “Why not talk about what you learned from the other panels you saw there?” So that is what this particular blog post is about. I think you’ll find some interesting things in here, especially where audio is concerned, as it was honestly my focus when deciding which panels I’d attend.

So first, let’s talk a bit about the audio design panels I attended. I admit a lot of this talk was technical, and over my head even as someone who has done some minor audio production. Still, there was information to be gleaned. For example, you know that sound they always use in horror movies and games to build tention? That kind of screeching that sounds halfway like a musical note? I found out exactly what that is. That, my friends, is a sheet of metal when dry ice is applied to it. Dry ice causes metal to make that sound. I thought that was fascinating.

Furthermore, though I was already aware this existed to some extent, the things done with sound morphing are also impressive. Try taking the sound of a refrigerator, and turning it into a futuristic forcefield. It’s a longer process than you might think, and the sound goes through several iterations of pitch change and modification to finally get that sound. Or how about gunshots? Sure, guns can be recorded live to bring a more realistic sound to a game, but volumes have to be normalized so as not to destroy the ears of gamers everywhere. However, we still want our shots to sound impactful, so then we add a bit of a base boost, or we increase the reverb effect that falls off at the end. It’s amazing how one small sound can undergo so much to sound just right.

And then, once all that sound design work is done, you have to worry about the mix. This was a completely separate panel. I won’t say too much, partially because again, it got pretty technical, but think about it. For good sound design, everything has to be perfect. How loud should dialog be compared to sound? When should this sound fade in? Should there be a crossfade here wen the player does this? Should the audio duck in certain situations to give dialog, or another sound, more attention? How should ambient sound be handled? What about footsteps? All these things have to be considered, and the answers to these questions may depend on the game. They most certainly aren’t all the same. There is tons and tons to think about.

That brings me to another presentation I attended, this one specifically talking about the audio in The Last of Us, a game that has won so many awards, and won even more during the conference. Anyway, the things they did for that game were astounding. They built a system that changes the state of a sound if it is being blocked by a wall or a door, for instance. There are sounds in the game that you only ever hear through walls, and they’re fully realized sounds, yet this system muffles them in the perfect way to make it actually sound as though a wall is blocking it.

But that’s only the beginning. Get this. They were aware that a lot of the storytelling in the game would not occur through cutscenes, but rather through conversations between the characters during gameplay. However, it’s very possible for you to get really far ahead of your companions, maybe putting a couple walls between you and them. This would of course make it difficult for you to hear their dialog. They considered making all conversations mono and perfectly clear regardless of position, but decided that took too much away from the emersiveness of the game. So what they did instead is build an audio enhancer that detects when a character is a certain distance away, and enhances their dialog just enough so, when the game is actually being played, dialog can only be so muffled, or sound so far away. Regardless, you should still be able to hear it just fine. And trust me, they demonstrated what it would be like if the enaancer was off, and it’s just about impossible to make out words. Definitely a great decision on their part.

And here’s another tweek you may not know about in The Last of Us. The entire mix is dynamically modified if the game switches to a combat state. If you attack someone or they attack you, a couple things happen. First, all enemy audio is enaanced so you’ll be able to hear them regardless of distance while still factoring in positioning. Second, the object detection of the audio is dimmed down a bit so you can hear enemies no matter where they are so long as you remain in that combat state. As soon as you’re out of it, everything goes back to the way it was, readying you for the next jumpscare.

Now let’s talk a bit about music. I went to several music-related talks, one of which was both music and audio, but all were interesting. A presentation from Austin Wintory showcaded his process for developing music for an RTS game. We heard all the different versions of the game’s themes that he came up with, the things that ultimately didn’t work, and the final product. It was amazing to get a peak inside the head of a composer that good.

However, it gets crazier. There was a presentation on the new Killer Instinct, which was the one that was both on audio, and on music. Firstly, I got to hear the original Killer Instinct announcer voice completely unmodified, which was neat. But where this presettation really shined was when they talked about their dynamic music system. In fact I was ashamed of myself afterward for not noticing these things earlier. For example, though every stage has its own music track associated with it, the track is not played from beginning to end. It is actually broken into several 1 or 2 bar pieces that are played at random throughout the fight, except under specific circumstances.

Then there’s the intros. Each character has their own intro animation that you see and hear before the fight starts. Well, all these intros have different lengths. However, so long as you don’t skip them, the moment each intro plays is actually timed to fit the music, so the intro for the song itself can be the same length regardless. Just another one of those little things.

Moving on, though, things get crazier still. Each track has its own little effect noises that are mixed in when one or both players have a full instinct meter. This effect, whether it’s a voice that comes in singing a few notes in the background or an extra little bit of guitar, pans appropriately based on which side of the playing field the player whose instinct meter is full is on. If both players have a full meter, the effect just occurs on both channels.

Then, you have the ways in which the music system rewards players. For instance, you only hear the chorus of any of these songs if someone performs a combo of 15 hits or more, and even then you only hear 2 bars of it. To hear the entire chorus, you have to consistently hit combos that large and keep it going. Also, a bit of an easter egg. The music system even rewards players for figuring out there is a mssic system, and trying to break it. If both players do nothing for a set amount of time, no button presses at all, you’ll be treated to a remix of hhe original stage song for that character from KI1 worked right into the new music. As soon as you start moving again, the song will transition back to the new stuff after the current bar, so as not to break the rhythm.

Then you have the tons of little musicll stings that occur whenever there’s a combo breaker, and you have the musical ultra combos, which are done using a number system and a bunch of small notes. It’s all awesome, and makes for a really dynamic listening experience.

And yet, nothing blew my mind quite like the presentation on the music of the Tomb Raider reboot. Granted I hadn’t yet heard anyone play the game, but even if I had, I don’t know if I would’ve realized the incredible, insane complexity put into the music system. Get this, guys. There is no “main battle theme.” No particuaar music that plays when you go into combat. Every single encounter, every fight, every scene, is individually scored. Yeah, that’s already incredible. But just in case that’s not enough, the scores are dynamic. Things happen in the music depending on what you do. I’ll give you 2 examples.

The first, and perhaps the best example they gave is one where you’re wrenching an axe free of this door so you can go through it. You do this by quickly tapping a button, but as long as you’re tapping fast enough, it takes you a set amount of time to pull the axe free. The score is ready for this. A long, high-intensity note, created mostly with horns, plays while you’re pulling at the axe, and the length of this note is exactly as long as it should take you to pull it free if you’re successful. Because of this, the score continues without pause if you are, and the note blends perfectly with the next bit of music, seeming like nothing more than an obvious lead in. However, if you fail, the music is also ready for that. The long note that started playing when you grabbed the axe will fall off at the end, and more, similar notes will play instead, highlighting the urgency of the situation as you theoretically try to wrench the axe free again. All this happening in the midst of a chase scene.
Another smaller example is a stealth one. There’s a scene where you’re sneaking up on 3 guys. Music plays throughout this scene, but not much of it to allow you to get into the quiet, stealthy spirit of that moment. Well, there’s one particular piece to the music in that area that only plays if you shoot the guy in the middle first. If you don’t, you’ll never hear it, not anywhere else in the game or anything. Remember, every single scene is individually scored. Pretty impressive, huh?

There was more of course. I attended a musical demo derby featuring game composers given an unscored version of the same couple games, and asked to rescore it. That was also educational, as I got to hear composers with different takes on the same experience. I also attended a panel on the voice industry, which as far as I’m concerned just prepared me for the future. Heheh. Anyway, doing these things, attending these panels was truly a wonderful and enlightening experience.I learned and experienced so much, and because of them, there are games I won’t look at the same way ever again. I hope you’ve enjoyed reading of my experiences, and do feel free to comment below. More is coming!

Gang Awards go Live!

Ladies and gents, the GANG Awards, a ceremony I was honoured to be a part of during GDC, are now live for all to view as a Twitch archive. This ceremony stands out as one of the greatest moments of my life. Getting a standing ovation from about 500 people in the audio game industry was something I will never, ever forget. I urge you all to check this out. For your reference, my speech is just around the 48 minute mark, give or take. Enjoy, and please feel free to comment if you have anything to say.