(Dis)Assembling Experience
This talk was one of two keynotes at the second Game UX Summit, held in Toronto in October 2017.
Description
The confusion is always there, as are the turf wars: what falls under user experience design? What is game design? For that matter, what about narrative design, UI design? All these disciplines work towards fundamentally similar goals, using oft dissimilar tools but very similar principles. In this talk, we will make a journey into the tiniest moments of interaction, and see these commonalities—and differences—laid bare.
Slides
Video
Transcript
Disassembling Games
RAPH KOSTER: One of the things about starting out when I did is that I got to wear all of these hats at one point or another.
I’ve actually been credited in just about every single role on here. I’m by no means one of the real oldsters in the industry, right? If you go back far enough, you’ve got teams of one, and that meant they had to wear all the hats.
Even inside just game design, on Ultima Online I actually was in charge of all of these except monetization. Today there might be a team of five, ten, or fifty people for each of these, because these are roles. These are hats that we put on, that we have to wear.
Luckily, it’s the best possible training to eventually direct games, to have to wear all of these hats, because for better or for worse, if you’re directing a game you end up having to wear them all.
The interesting thing about that, of course, is that there was no such thing as UX even as a hat back then. It just wasn’t a term. Nobody thought about it.
So that means when I come to a conference like this, I’m like, “It’s a conference about what?”
Now, that doesn’t mean I’m unaware of what it is, right? In order to do all of those things, I still had to go and teach myself cognitive psychology, or read books on sociology or economics or whatever. Whereas today, what we might do is pile different people together and say, “Well, the UI and the UX person can be the same person,” or whatever. Whoever does quest design can also be the narrative designer and the game writer, and that kind of thing happens.
But the fact of the matter is that each of these is actually a full field and more. Each of these is actually many fields, often. Each one.
Even the diversity, the range of stuff we’ve talked about today in terms of what UX is, has already been incredibly broad. We’ve talked about elements of narrative design, community architecture, social design, governance. It’s been all over the map.
So it’s easy on teams for people to end up saying, “Well, then who owns what, and what does this mean?” And for that matter, “What is game design anyway?” Never mind the stories about “What is a game?” We won’t go into that one, but you can spend years having arguments about that one. Trust me, I’ve done it.
The thing is, the way I think of it, all of these are actually really, really, really similar at kind of a fundamental, pedantic level. This very atomic, dry way of thinking about things. It’s really straightforward.
Here’s a problem. The user has got to go, “Uh, a problem. What do I do? I push this.”
“Okay, I’m going to push that.”
“It said something! Oh, what does that mean?”
Okay, we’re done. That’s it.
I mean, that’s pretty much all of interactivity design. That’s all of interaction design. It’s certainly all of game design. And it’s actually also all of things like narrative design.
This is a standard loop that I tend to use. You’ve all messed with something like this. The player has a mental model, and they form an intent, either on their own or because of goals presented to them. Then they go, “Oh, there’s something on the screen, or on the table, or on the dash, or whatever it is, that looks like a tool for accomplishing that event.”
So they push it, or pull it, or twist it, or pick it up, or drop it, or break it, or whatever. Something happens inside the guts of the machine, and something changes, and they probably don’t quite know what. Then they get a response back, and from that they update their mental model.
That’s fine as far as it goes. That’s kind of basic HCI. It’s basic design. It’s very Don Norman. But especially with video games, it’s not detailed enough anymore.
That’s why the title of this is “Disassembling Games.” Let’s break it down a bit.
There is this black box, and I’ve formed some goal. Maybe in Forza I want to win the race. Maybe in Fortnite I just want to win, just kill everybody. Whatever it is, I’ve got that intent way down there.
Then there’s something on the screen that’s going to tell me, “Well, you can do this.” Oh, I can shoot. Oh, I can move.
There are a lot of steps in between making a simple movement and actually winning the race. Even that affordance then needs to accurately translate into an input, which it might not. Frankly, most games these days let you remap the inputs into some other command anyway.
If you’re really thinking carefully about the design, that means you need to actually think, “Well, what if my input has been moved someplace else?” We already had the example of this happening on Nintendo versus Xbox controllers, where that gets moved even to a platform difference. But it can also happen within one game, as any of you have had to sit down in a shooter and realize, “Oh wait, this is using default mapping number two? I don’t know how to drive. I don’t know how to move anymore.”
Let’s take chess, actual chess on a table. If we form the intent, “I want to go capture the pawn,” and the affordance is my knight, I’m going to move the knight. My input is actually my hand. I have physically used my hand to pick it up to execute the action of moving and capturing, and then that goes into the rules system of chess.
If I’m playing computer chess, the part that changes is my controller. The affordance of seeing a knight that I can manipulate stays pretty much the same, and really, the rules of chess are the same. I can play chess in my head, so I can do this without any of these boxes.
That means these things have their own standing. They have their own problem sets. In fact, you can almost think of these as each being their own game.
The first game I have to sit down and play is to think about whether I have a goal. If not, I’m going to play a game of finding one. If there is an affordance, I am going to play the game of recognizing it.
That might not sound like a very fun or interesting game, but Whac-A-Mole does pretty good with the idea of recognizing an affordance and making it into a game. That’s something that can be exploded out into exactly the same kind of interaction.
We can look at it at this granular level and say, “Oh, even recognizing a knight, recognizing a rook, recognizing an arrow coming down in Dance Dance Revolution, even that, I need to form a mental model, form an intent to find something interactive on the screen, decide how I’m going to interact with that thing, usually my eyes in the case of a video game, but it might be your hands in the case of a game of tug-of-war.”
Then it goes into the rules of that affordance, which are basically pretty simple. That’s not a very complicated rule system. It’s only everything to do with our visual perception, how we chunk visual objects, transform them into iconified versions of themselves, a little bit of pareidolia, those complicated, cool psychology terms.
Then we move that back out into our state change. We have identified the correct vaguely horse-shaped, decapitated horse—wait, does this have something to do with The Godfather?—object, into the feedback that I now associate that with a piece that I can manipulate in my mental model.
That’s just looking at the board for one piece.
Then we do it again, because there’s another game to play. “Oh, well, if that is something I can manipulate, what else can I do with it?” And there’s a whole new game to play.
Then we have to keep doing it again, because then, if I move it, is it going to do what I wanted it to do? I can move a knight. It moves in all these weird ways. It teleports. It’s non-Euclidean. Oh my God. And then, finally, I’ve moved it on top of a pawn. Now we go into the rules system of chess.
This is detail. I mean, it’s down here, down at the metal. It’s illuminating, in a way, I think, to think this atomically, to think in this incredibly granular way.
If we take a step back for a minute, the question is, “What is game design?”
Game design can be thought of as the whole. I think I even said in A Theory of Fun that the art of game design is the art of the whole. But it can also be thought of, in particular, as game system design.
Now, that’s really the piece that survives across media. It’s the part that we can move from representation to representation and from UX to UX. We can move tic-tac-toe, or Battleship, or chess, or cops and robbers, or capture the flag into a myriad of contexts, a myriad of user experiences, and the black box is the same. The rules remain the same.
I’ll often say that the heart of game design might be the whole, but the heart of game design is this immutable piece, the system that only games do, which lies at the center of it all.
So what goes in those? That’s a whole field, and frankly, it’s one we don’t even have a lot of language to talk about, although there has been more and more.
We know that huge swaths of types of games have to do with dealing with mathematical problems that fall into complexity ranges from PSPACE-complete up through the entire range of NP-hard problems. If that was complete gobbledygook to you, that’s fine. It just means they’re hard, and really good ones fry computers.
So that’s good. If you create a game that fries computers, there’s actually decent odds that it’ll be pretty fun. Our brain likes teasing problems like that.
A lot of them have to do with simple calculations over time. Our brain is actually not that good at a lot of kinds of calculations over time, so the games help us exercise those things.
It’s also about complex relationship webs. Black boxes are machines, in a way. They’re like the mobiles that we hang over cribs so that babies can poke them and they wiggle in unpredictable ways, and as they do that, they form an impression of a complex, connected system.
People are one such system, so we’re also talking about social relationships, relationships of trust, of hierarchy, of the schadenfreude and naches and all those other wonderful words that have to deal with all those social emotions.
Then there’s the fact that we are a machine, right? One that’s hard to steer. It takes us years to learn how to steer the machine that we are. So a whole other class of these difficult problems is physical. It’s motion, it’s movement, it’s oxygenation of your lungs if you’re a runner.
Emily and the others who raised their hands and said they enjoyed running: they’re performing calculations over time of volume, oxygen. That’s part of being a runner. They’re not thinking of it. Maybe some of them wear O2 sat meters or something, but most don’t.
Then there’s the one that we have brain bugs around. There are a lot of those, but for game systems, chance is probably the biggest.
Over at the other end, we end up having enough classes of problems that they’re actually designed to be as simple as possible. This is actually where a lot of UI design lives.
You want something with as little movement and as little ambiguity as possible. You want something with easy one-for-one mapping. You want completely intuitive mapping of input to action.
So it’s a part of the discipline of the whole, but it’s actually a very different exercise. In UI, you’re trying to make things as simple as possible in one sense, and in games you’re actually trying to make things that are as complex as possible in a different sense.
Even the process of thinking about a UI button is, after all, this same little game. You have to march through this process of, “What is my intent? What is my affordance? What is my input?” I’ve actually had enormous luck in tuning user interfaces by thinking of each button as a mini-game.
Like, okay, level one player. We have a newbie. Are we training them on this button? Is there a tutorial for this button? I’m working my way up. The nice thing about that is that it lets you build checklists across your entire product.
Are people actually learning? Because the whole point of a loop like this is learning.
A lot of the fun we have comes from learning. A lot of the things we do in games, and in UX, and in UI, come from learning. They’re about mastering and building not necessarily explicit understanding of things, but intuitive, back-of-the-head understanding of things, things that we can’t necessarily turn around and then explicate, explain, write down, or chart.
That means games nest.
I need to solve the problem of identifying a rook. I need to solve the game of picking up a rook. That’s a tough game for a three-year-old. I need to solve the problem of putting it down in the right place according to the rules. I need to solve the problem of establishing the proper territory on the board and controlling the center. I need to solve the problem of winning the chess game. I need to solve the problem of winning the chess tournament. I need to solve the problem of becoming a grandmaster.
Games always nest. We can go all the way from the simplest button all the way to the highest-end, long-term scale cultural game with something like being a chess grandmaster.
None of that is probably too surprising to you guys, because you’re all UX designers. But my background, believe it or not, is a Master of Fine Arts in poetry.
So I’m here to tell you that this is also how we write poetry. When we set out to tell a story, build a myth, share a culture, what we’re doing is assembling this arbitrary-sized set of signs and symbols, of metaphors, of narrated experiences, of stories, of characters, of objects that have complex resonance that players—readers—have to form a mental model of.
They have to approach it with intent. They have to start thinking about these relationships. And sure, in something like T. S. Eliot, or something else impenetrable, the actual model of what’s going on can be dense and complicated and hypertextual and referential. And in something like the moment in, I don’t know, Arkham Asylum, or a game like Tomb Raider and Uncharted, you might just get this vast, incredibly intense, very juicy, high-feedback moment of a clump of narrative.
I read a sentence; I get a clump of narrative. I watch a cutscene; I get a clump of narrative. Then I have to try to arrange these puzzle pieces that I’ve gotten.
T. S. Eliot’s puzzle pieces are really weird-shaped and in complex, arbitrary arrangements, whereas something like Arkham is probably pretty linear. It’s pretty easy for me to take them and just put them end to end and put them in order. And that’s fine. We’re used to consuming narrative in order.
But that isn’t the way we solve narrative puzzles, because that isn’t how narrative and stories and characters and writing work. We actually, our goal is to shape it into this other puzzle, which makes us think of how these symbols interrelate, how they connect.
This is what writing horrible English term papers is actually about. When they said, “Oh, well, tell me something interesting about this,” the teacher is actually asking you to put together the puzzle pieces in an interesting, fresh way, to talk about the relationships between them that are not “A led to B, B led to C, C led to D.”
These pieces connect. In a simple story, you might have a very straightforward plot, where it turns out it’s all about love, or it’s all about heroism, and maybe there’s a loose end in the plot. In something more complex, these pieces might fit together in a myriad of ways, leaving the piece open to interpretability, making it art, if you want to use such a highfalutin term.
Why do I bore you with all of that? Because there are a lot of ways to build an experience. There are a lot of kinds of experience, so many kinds of experience that lately I tend to think of games this way.
This is the expanded version of that interaction loop, because there are so many experiences here.
If I start out with the intent of performing an action, first I need to develop a heuristic, form my objective that might involve high or low agency, that I’m going to play with perhaps in the service of a goal.
Hopefully, there’s an affordance that moves to a verb that provides an input that I decide to do an action. Then black boxes explode into all of these different fields. I tend to call them ludic structures, ludic artifacts if we’ve built them, ludic structures if not.
Music and the weather and the stock market are ludic structures. They are structures that have the characteristics that enable us to play with them. That is, I think, why we play musical instruments: because they have a game system built into them, the musical system itself. That’s why we play the stock market. I think language is wise that way.
Some of these things are about possibility space, and some of them are about complexity problems, and some of them are about our body. They lead down into realms like simulation and the inner life of objects, like that game Mountain, if any of you played that, which felt like it was entirely about experience except I think, when you dig into it, it’s actually about whether or not the mountain is actually in there. It’s about the inner life, the simulation.
It leads you to all the social problems that we just spent all that time talking about: hierarchy, othering, identity. These are the things that lie at the root of the toxicity problems that we’ve been talking about today.
That means they open up all these fields. From there we can move on into how we even tell people what’s going on: state views and imperfect information and metaphor models and the myth-making that players do around these things, all of that leading towards empathy and dynamics and art and play and blah, blah, blah.
And that means that all of a sudden, if you’re really serious about the craft of making games, and you really want to own the experience, then you’re going to be at the university for a really long time.
[Audience laughter.]
I’m not joking. I have books on my shelf for every single one of these fields, and yes, I read them. I swear.
Because that’s the thing. Once you start uncorking a cultural object like this, there’s no bottom.
There are a lot of ways to build experience. That is why today we’ve been able to move pretty seamlessly from discussions of identity to discussions of tribal membership. As folks were talking about, when in a co-op team, of course you do better when you’re together, and my head goes, “Multi-level selection theory!” Then we move on to something else and some other field pops into mind and I go, “Oh yeah, that’s a use of cinematography!”
Because that’s what it is. It’s enormous.
Then, on top of that, you’re designing the user experience. You’ve got to deal with the fact that since that very first step, as a player comes and forms their own intent, they might use your game for something else. Happens all the time.
“I am now going to play Super Mario to get over my breakup.”
“I am now going to dive into this game and play it blindfolded to prove that I can.”
“I am now going to beat Portal in less than thirteen seconds.”
“I am now going to decompile the game and write cheats for it.”
Those are all intents players can form. They’re using the object. There are even affordances for them to do this stuff. And they’re not in our control.
So, for me, UX, the discipline, is about being the axle for this wheel. It’s about getting this stuff to actually turn, getting this learning loop to actually happen. A lot of the other fields provide the things to learn, but the process—the process is where the user experience ends up living.
Fortunately, the preceding talk was about a racing game, so it makes for a perfect example. I swear I didn’t plan this.
If we think about a really tight-loop action game, driving a car, their black box is all in your hands. It’s all about haptics and feel. “Feel” is the term that got used.
If you look back at the older driving games, which they conveniently illustrated, the highest-order problem other than winning was, “I’m going to go as fast as I can down the optimal curved path for this track.”
That’s what it was about. Meaning, here’s a track. I need to figure out this line. That was the math puzzle at the heart. This, by the way, is an NP-hard math puzzle. It’s called the minimum cut.
But anyway, this used to be what a racing game meant in Gran Trak 10 or something.
Of course, you immediately have to change that path because you have to modify it in order to avoid other cars. Both of these are cognitive problems. They’re both causing cognitive loads. They’re puzzles I need to solve.
We only have so much bandwidth as players, so we run the risk of overloading. Then comes the third problem, and the third problem is haptic. It’s autonomic. It’s dealing with the fact that, oh my God, there are way too many buttons on this goddamn controller.
Because, let’s face it, there are way too many buttons on our controllers.
So in most modern racing games, this is the game. That’s why teams from Forza can stand up and say, “It’s about the feel.”
It didn’t used to be. Gran Trak was not a game about the feel. Gran Trak was a game about solving that path problem.
What ends up happening in this new haptic loop is that we have to solve different kinds of problems if we’re solving pathfinding versus the haptics, the interface, the autonomic responses, the reflexes.
The best practices here start dealing with actual physical limitations, like the fact that the average human reaction time is more or less 265 milliseconds, even though our perceptual frame rate is actually somewhere sub-15 milliseconds. But it’s possible to train reflexes under 265 milliseconds if you push people through repeated patterns of physical behavior. We can get that autonomic nervous system working for us.
Some of these are already instinctual. We get to see that when we stick our hand in a fire and our hand gets pulled back by that nervous system before the brain registers pain, because our brain is actually a distributed system.
We can react faster than our brain can react. Pain sensation takes about 265 milliseconds. It takes about 200 milliseconds before your hand inches away from the flame.
Today most racing games, therefore, have cut the path problem. They actually paint it on the road for you. They give you a ghost track and they say, “Just follow that.”
Why? Because the steering is so hard. It is enough of a challenge, and enough of a game in its own right, that in order to improve the player experience, we make a choice and we say, “We want the players to play this game, not that one.”
That’s at the heart of UX: picking the right problem, getting people to play the right problem, even when there’s more than one in the game.
So that loop, that particular racing action, helps reflexes. It helps training reaction time. It has a particular objective. There can be a lot of verbs: brakes and gearshifts and the wheel and lots of stuff there. But I’m going to use another term, which is that it doesn’t have a lot of play to it.
Now, I don’t mean you can’t play it. Obviously, you can play it. But think of that mobile and you poke it. You can only poke it so far. It only wiggles so much. It’s actually a fairly constrained game. It’s not that unpredictable, I guess is the way I would put it.
Contrast it with Minecraft. Way fewer buttons, way more wiggle.
Inputs do not mean it has a ton of play.
One way to think very mathematically about play is the possibility space. That right there is the very game-grammar-y kind of flow diagram of resources and whatnot through tic-tac-toe.
Tic-tac-toe only has about 255,000 possible games, but our brain is so good at pattern matching and building intuitive relationships and understandings that we very quickly realize, “Wait a minute, this is rotationally symmetric and mirror symmetric,” so we can collapse these 255,000 possible combinations down to 765, which means there’s only 26,000 games of tic-tac-toe, and that means it’s boring.
That’s how good our brain is at putting these kinds of patterns together when they get taught.
Tic-tac-toe is basically a tool for teaching younger kids how to collapse possibility spaces, how to learn, “Oh, it doesn’t matter which corner I start in. It means all of these consequences. Oh, I should grab the middle.”
We learn to collapse. We learn to find isomorphisms in the graph of the tree.
But play, like a poem, is there too. It’s there in writing.
One of these sentences has more “play” to it than the other. One of these just has more space inside it, even though it uses a lot fewer words, because we can’t help but read the second one and start walking through this crazy choice architecture. “What happened exactly? Why is this going on?” It just has more space inside of it.
One way to think about this that ties it back to UX, I guess, is that we can think of the agency that we as players have, the number of buttons on the controller or whatever. These are inputs. They’re verbs. But then there’s the stuff inside.
That sentence: you read it, you don’t have a lot of input buttons on that sentence, so to speak, but it’s got a lot of space inside it. It has a multitude of possible answers within it. It has complexity inside it.
When we look at something like a UI, we want the button to have a really straightforward answer to a really straightforward question. That’s what a great UI button does. I’ve got a really easy question. I want to hit “go,” and the button says, “I’m it!”
That’s great UI.
Great UX is often about saying, “I know you’ve got a lot of choices today, but go that way.” It’s about collapsing down a whole bunch of possible inputs into one answer.
Games are easily just a reversal of that.
We can’t go that straightforwardly, because games can do a lot of different things. Games can be teaching us many, many different kinds of things, very varied lessons.
Some games can be highly interactive but not have much to say thematically. Some games can be extremely interactive and let us do whatever we want, be completely creative. That means as games have different purposes, we end up clumping them into genres, talking about the ways in which particular kinds of games accomplish their purposes.
These are different kinds of learning. Some games are about teaching us to understand people. Some of them are about helping us understand complex webs of relations. Some of them are about getting us to behave the way we want: “No, you stay in your channel and you behave this way.” Some of them are about building reflexes. And that’s fine.
Games can do many, many things. We just need to be aware, as we build our user experiences, that we end up using vastly different tools in vastly different fields in order to accomplish these different kinds of purposes.
In this case, some of these, you can go back to Greek rhetoric for, like that horrendous word anacoenosis. Do you like that word? That was anacoenosis.
[Audience laughter.]
Let’s just look at one quadrant here, which is actually the dominant mode of games today: here’s a game where you don’t actually have tons of verbs, and you also don’t have a lot of agency.
Most games are entertainment. Entertainment is noble. It serves an important purpose.
But what happens when we entertain? We actually seek out black boxes and challenges people already tend to know, and we do it on purpose. We also start to seek out stories to put into those games that are also the ones people already kind of know. “Your princess is in another castle.” Rescue the princess.
Because these are not about incurring heavy cognitive load. They’re about acculturation. They’re about trying to get you to see the world in a particular way.
So we could look at the ways in which rhetorical devices like repeating something multiple times help you as a player learn that particular pattern over and over again.
In Howling Dogs, which is a really wonderful Twine game, we get this kind of repetitive action. It’s there for a thematic point. Or there’s the use of anaphora, which is where I say the exact same sentence structure multiple times, because the exact same sentence structure multiple times really helps to reinforce the point that I’m going to say multiple times.
Old-school rhetorical devices.
In games like these, we make use of those other rhetorical techniques. We bring up opposing viewpoints just in order to present an argument against someone and have them go away. We use labeling in order to narrow ideas down into frames so that they can be iconified and tidily filed in a particular box in our minds.
We intentionally create characters that people can relate to in particular ways in order to build up a certain kind of sympathy, because when we are in sympathy and identify with someone else, we are more likely to agree with their actions.
This, by the way, is why encyclopedia salesmen used to make a living, only today I believe they all work selling you solar on the phone. The more they can seem like you, the more likely you are to go along.
This is what Gone Home does, quite intentionally. It builds up that empathy.
Of course, we also do the opposite. I mean, name a shooter that doesn’t have bad guys in it. MMOs, if you start really thinking about an MMO, are kind of colonialist and disturbing. Wait a minute, these orcs have a civilization and we’re just killing them all and raiding them too? Whoa.
But we do this in games all the time, and not just with orcs. Games do this with women, as we all know. Othering. Putting into a tidy box. “Oh no! How are the breast physics?”
[Audience laughter.]
The funny thing is, here we are dealing with games about war and force projection, but we don’t have to do that with a story like that. All Quiet on the Western Front, the movie, the book, is about empathy, not about acculturation. You don’t walk out going, “Rah, rah!” for your side at the end of All Quiet on the Western Front.
So it isn’t that war stories naturally fall into it. It’s that the overall experience structure falls into this pattern.
One of the tricks of this quadrant is that it tries to tell you this is the only way this could have gone. If you think about that, that’s actually pretty much how we design most triple-A games. In practice, there’s only really one way in which the experience could have gone, and it leads to the end, to the correct achievement and trophy, and hopefully to you buying the sequel.
These games are about control.
Even sandbox games can be critiqued on this basis. SimCity was famously critiqued as an overly liberal take on city planning.
One of my favorite examples of this is actually September 12th, if you haven’t played it. This was a news game about terrorism. It’s quite simple. You see terrorists; you can lob a missile at them, but they do collateral damage, and anybody who is standing nearby turns into a new terrorist. The only way to win is not to play the game.
High causality.
So in some ways we spend a huge amount of time talking about that as the discipline of UX, that exercise of control. But in a lot of ways, the hardest games to build UX for fall at the exact opposite quadrant: the quadrant of systems that are so rich that people can’t actually give you solutions like Arkham has a solution. A bot can play Arkham. But if you’re up at the top end, those are the games that fry computers.
You can’t build a bot that will beat it perfectly.
That’s actually the toughest challenge for UX: multiplicity of verbs, multiplicity of goals, and no clear paths. It’s full of problems like the ones you guys have spent much of the day discussing. How do we deal with toxicity? How do we deal with multiplayer? What happens if you’ve got a lot of games inside one sandbox world? How do you build the UX for that? How do I guide people to the right starting point?
I’ll tell you my approach.
Games are made of games.
Those big sandbox games, designing something like Ultima Online—which, by the way, celebrated its twentieth anniversary running last week, for anybody who feels old now—
[Audience applause.]
We designed it down to when you get on a boat, the guy who is the tillerman on the ship tells you shaggy-dog sea stories, all the way down to, “Oh no, there are supposed to be eight necromantic reagents, but we left one out on purpose, so the quest cannot be beaten,” all the way up to, “Here’s the macroeconomic structure of the world and the essential physics and resources that go into it.”
These are all games. The only way I can get a handle on dealing with giant worlds is a checklist.
Do you guys like checklists? I like checklists. So I’m just going to give you my checklist.
Remember I mentioned the tiniest game is a button, and the biggest game is the whole thing? What I do when tuning a game is I go and ask these questions of every button, every loop, every system, every metasystem, every metagoal, every metagame, and for that matter, further out into the outside of game, into the community, the cheat systems, the forums, the society that will form around the game.
These ones are all about, hey, is this actually a substantial enough problem?
Don’t worry, we’re not done yet, because then I have to ask if there is a decent enough UX for it. Every button, every affordance. Is there an affordance for running your own player-run shop in Star Wars Galaxies? That’s a different problem. Are there enough forums for winning the Galactic Civil War? Separate problem. Is there an affordance for it?
I do that for every system in the game.
Then I do it again and see, okay, is it actually going to engage? Is there going to be flow? Can you actually be captivated by this?
This starts dealing with things like movement over time for the player’s learning experience. Does it have rising and falling pacing? Turns out that the curve for good gameplay pacing is the same one as that one they taught you when you were stuck in your high school literature class, and they showed you that curve that goes vroom-vroom.
Turns out, that’s also the rhythm of when you insert bosses in a gameplay experience, because we’re human and we use the same patterns underneath.
Even after that, I keep asking, because games actually run on players, not on computers. So I need to ask myself all of these questions.
Can you become a chess grandmaster? Can you mentor somebody? Can you gloat over somebody? Gloating is important. In limited doses, but it’s important.
Does it promote tribes? Does it prevent tribes from hating each other?
I have to ask all of these questions of every button and every loop, because games are made out of games.
If you think that’s exhausting, it is. But it’s an incredibly useful tool for me.
At Metaplace, I worked on this little pirate ship fighting game. UX folk, what is the most important part of this little drawing? Because I’ll tell you that the designer and the artist who worked on this ship, in my opinion, got it wrong.
What’s the most important part?
The wake.
If that doesn’t work, the player learns nothing. They cannot steer the ship. The ship could look like a rock, okay? But if it looks like a boat, because it needs to have an affordance that it’s steerable, the wake is critical for people to learn about the controls.
And guess what? The artist started out by building a pre-rendered wake loop. Like, no, it needs to be dynamic.
I’m going to get super concrete here, because I’ve been making board games lately. Hopefully, this is getting Kickstarted before the end of the year.
It’s this little party card game, and I designed the game system nine years before I managed to put it on cards, because its first UX was on wooden discs that looked like that, which is horrendously bad. We playtested it, and everybody said, “Wow, this sucks.”
The game is about building rainbows, so you want to put together sets, but you also want to make sure you don’t repeat any color more than once. Once I’ve started repeating orange, it’s the only color I get to repeat from then on in that hand.
You can picture what happens when you stack these. “Am I repeating more than one color in that stack?” is really hard to read when it’s these discs with these weird cube shapes on them.
Moved it to cards, everything turned around. Black box identical. UX completely different. This happens all the time when you’re working in system design, completely reskinning something.
I’m going to give you a game that you can all just take home, because you probably have all the pieces you need.
If you just take the red cards one through four out of a deck, you’ll have two of each, hearts and diamonds. And you take the black cards one through four out of a deck, and you’ll have two of each. Then you lay them out so that you’re playing four stacks.
On your move, you can play a higher card than whatever is there. A spot starts out empty, so you can put down a four over an empty spot. You can put down anything higher than empty, but nothing can go on top of a four.
Or, if there are multiple cards on the stacks, you can move any card from on top of one of them onto a different stack, as long as it’s higher than whatever is there.
Your last possible move is, if something is topped with a four, you can take all those cards, give them back, and you have to replace it with a one. You have to have the one in order to do this move.
This game is ridiculously simple. This is a really, really tiny black box. Anybody want to guess how many orders of magnitude larger than tic-tac-toe it is?
Yeah, it’s a couple of orders of magnitude larger.
You get to win either by getting one, two, three, and four of any color on top, or if all of the tops are your color, or if the other player can’t make a move.
Now, I didn’t even design this with cards originally. I designed it with discs, which means you couldn’t see what was underneath. Same with cards; you couldn’t see what was underneath. But that isn’t what I was actually working on designing.
I was designing this, which is actually quite a complex math problem. I’m designing this underlying system, and what I want players to do is build heuristics, build an intuitive understanding of the possibility landscape of this. I don’t want them to be able to explain it to me. I just want them to understand it intuitively, because that’s what game systems design tends to be about.
When this was played on cards, you looked down at the top of the cards and all you could see was the top. There’s a four on top, there’s a two on top, a red four on top, a black four on top. That’s all you can see.
It’s hiding three quarters of the state space from you.
I changed it to this eventually. I went over to the craft store, bought a bunch of wood stuff, took a hacksaw to it, played with one of the staining pens, and made an almighty mess. I changed it into physical castle pieces.
This is UX design. I’m not changing things underneath. The mathematical structure is identical. But the legibility of it, the guidance, the affordance is all very different. Now I see all of the context that previously used to be hidden. All of the state space is now visible to me.
It’s not that you couldn’t have calculated what it was before. You could. You could look and see what cards you have left, you can look and see how many cards are there, you can look and see what cards the other player has in their hand. You could figure out what was under the different stacks. You could play a memory game and you could count it.
But that’s not what the game was about. That is not the lesson I wanted to be teaching.
It’s a hard problem of its own, and it’s a different thing. It’s like finding the perfect path on the racetrack when what I want you to do is master the steering wheel. It’s a separate problem.
As soon as you can actually see all of the different castle pieces, I am now guiding players to the right kind of problem solving. I’m guiding them towards the problem that I want them to work on, because frankly, I’m an artist, and that is my theme. I want them to think about that.
I want them to think about the shapes and the ways in which this castle can be built, and maybe which noble family is going to end up the king. By making this choice, now I have different forms of play, because as soon as I did this, people would start just playing with the blocks.
Now there are whole new game systems for players to form their own intents with in their own ways. In fact, by accident, I made a game of how does this get back in the box again? Turns out there’s only one way to fit all those wood pieces back in the box.
[Audience laughter.]
Free game!
It is harder to read. I accept that tradeoff. Plus, I can always stick numbers on the sides of the pieces if I have to.
The point here is, if I’m wearing all the hats, sometimes I need to make sacrifices on some of the hats. Somebody has to sit at that fulcrum point. Somebody has to say, “This is more important than that.”
For me, the thing that will end up being most important is: what is the intent of the game? What is it that this game is trying to get across, to teach? What is this lesson?
It doesn’t matter whether the game is about reflexes. It doesn’t matter if the game is about a deep, complex plot and mastering all of the intricacies of it. It doesn’t matter if the game is about creativity. That’s why we start with vision statements.
For Galaxies, it was “live in the Star Wars universe.” That leads to a very different place than “be a Jedi.” That’s my filter. That’s my artistic statement. That’s why I bother doing what I do.
I make games, honestly, because it lets me play with all of these fields, because I have the attention span of a magpie sometimes. I really need to move across to different things, move across the fields. I hate getting stuck in one. Apologies to everybody who works in AAA.
[Audience laughter.]
Disassembling games into these little bits, and then being able to talk to individuals on the teams who are specialists in these fields, and knowing just enough about their fields to, A, probably annoy them, but B, be able to communicate with them in that language, helps me steer the game towards its actual intent, its actual purpose.
So I end up going, well, is that experience design? I don’t know. I’m lucky enough that I direct, so I don’t have to say whether it is or not. But yeah, I’m not in AAA.
In the end, game system design is about creating those problems for the user to solve, whether they are vast, intricate plots and complex webs of relationships, whether they are deep, complicated mathematical systems. Games are about presenting problems, about challenging the users with certain kinds of complexity, and it’s really about revealing the complexity.
It’s about spilling this toolbox in front of them and going, “Poof. What do you make of that?”
Those are the great games.
“Oh, it turns out I can make a dragon out of that.”
“It turns out I can make a racing game out of that.”
UI is about saying it should be really, really easy to put the pieces together. The tools I use just need to work. They need to be basically invisible.
And UX, for me, is about guiding you to the right problems. It’s about staying true to the soul, the spirit of what this artwork is for, what it’s about.
A perfectly valid answer is so that the player can take it and do something with it that we never expected. In fact, that’s actually my favorite answer, personally. It’s one of my favorite things to give people.
That means in some ways UX is about hiding the complexity away, guiding people towards the right pieces of it. It means that more than anyone else, except maybe that director, it’s you in this room who have to be able to think about what this game is for, who it is for, why it is for them, and what the complexity is that it is trying to teach that sits at the heart of it.
The beauty of that is you’ve got an awful lot of books to read and a whole lot of disciplines to go dig into. But the beauty of that is, in the end, it means you’re the ones, fundamentally, who get to reach out and touch the player.
You’re the ones who present them with the choices, let them choose, and in that choice you change them. You change them because they learn. They develop. You change people.
That, I think, is awesome power, and as we’ve learned today, a pretty awesome responsibility too. So go forth, and please use your powers wisely.
[Audience applause.]
I don’t know if there’s time or not. Ten minutes? Look at that!
AUDIENCE MEMBER: I have a question. Sorry, I just have to say that when you said touching the player, twelve years of my life I’ve been playing Ultima Online—
RAPH KOSTER: Thank you!
AUDIENCE MEMBER: And that game has absolutely made me want to work in video games. The work you’ve done has absolutely shaped the person I am today, so I just wanted to say, really, thank you for that.
RAPH KOSTER: We’ll never know what you could have been instead, right?
[Audience laughter.]
Twelve years? Oh my God!
AUDIENCE MEMBER: I played Ultima Online for thousands of hours, and honestly… So actually I do have a question, and I’m not just geeking out.
In Ultima Online, the game was always so immersive. The UI has always been so meta. I’ve always felt that it was the best example of meta-UI. The inventory system, where you have to rummage through your bag to find something, or someone’s stolen something from you, you had to think to yourself, “Is it actually in my bag?”
Or the spellbook system, or the fact that when you run out of town and you don’t know if you’re going to die or not. That risk was the core essence of the game, and immersion was the core essence of the game.
I feel like after a few iterations of Ultima Online, a lot of those aspects disappeared. For example, the inventory system changed to something that looks like a bag but had cells. Or they started forgiving the player and not making it a dangerous game anymore. There was this thing where you could just resurrect yourself and get out of the way.
It’s such an amazing game, and I wonder, do you think those changes to the UI design through iterations were actually the reason the game became less popular after a while?
RAPH KOSTER: No, I don’t. But the way I would put this is, we went into UO with a bag of themes in our head. We were going to go build an alternate world on the internet, and therefore we were going to simulate everything we could think of to simulate that we could do on a server that was dramatically less powerful than your Fitbit.
That implied a lot of limitations. But if you are true to that soul, that theme, that point, then yes, you have to rummage through bags and not use inventory Tetris gridded UI.
Is inventory Tetris gridded UI better UI? Probably. Is it even better UX for an inventory? Yes. But does it fit the UX of what the game is about? No.
That’s the thing. When you create a game that’s meant to live on, the next person who takes it over has different themes in their mind. If you’ve built a game expansive enough that it can accommodate that, then they can say, “You know what? For me, Ultima Online has always been about guilds,” or “it’s always been about gear,” or whatever.
We’ve actually seen over the years, there was a time period where UO became about gear. It was not about gear when we started. Very explicitly.
So that artistic point can change. The way I think of it is, Bob Dylan had something in mind when he wrote, I don’t know, “All Along the Watchtower,” but a live team like Jimi Hendrix, or the Indigo Girls version, or the Michael Hedges version, might take it in pretty damn different directions, and it can still be a great song with a very, very different experience.
So I just think of it as, my UO was the one you played between 1997 and 1998.
AUDIENCE MEMBER: It’s the best one!
[Audience laughter.]
RAPH KOSTER: Hendrix’s version is better than Dylan’s. It happens.
AUDIENCE MEMBER: Whenever I give a presentation about UI, I always put a slide in where I show the inventory, and I say that UX is not also just about measuring the usability and the accessibility of the game. It’s about measuring the pleasure, and what you designed gave the player pleasure.
RAPH KOSTER: I did not design that piece. Richard Garriott did.
AUDIENCE MEMBER: It’s still fantastic.
RAPH KOSTER: It was taken from the older Ultimas. Richard was also the one who came to us and said, “The UX and the UI in this game is a mess. Your hands are on this mouse button, your feet are on the other one,” which was brilliant and in one stroke solved UI all the way across the game.
AUDIENCE MEMBER: Thank you so much.
RAPH KOSTER: Yeah, sure.
AUDIENCE MEMBER: Hi.
RAPH KOSTER: Hi.
AUDIENCE MEMBER: Thank you for the inspiring talk. When you described your approach to designing systems, going to different levels from most granular to an almost meta level, where eventually you’re dealing with how people interact in society, that got me thinking about problems outside of gaming. Problems in the world.
Do you think we could use that approach, and how can we use that approach, to perhaps tackle other things in our world?
RAPH KOSTER: Yes. I do a lot of consulting these days. I’ve actually done a fair amount of consulting for non-game projects because so many of the social lessons taken from massively multiplayer apply.
When we started out with MUDs and massively multiplayer, we were consciously thinking about—I’ll be blunt—we were crazy. We thought we were building Snow Crash, the metaverse, the Matrix. So we actually thought about all of that from the beginning.
Part of the reason why we kept player killing in UO was an ethical decision. If we put in systems to control this behavior, what will happen in twenty years when the internet doesn’t have a player-killing toggle on Twitter?
I’m serious. I was that naive and crazy when we were doing UO, and you can actually find essays from the ’90s where we’re explicitly saying things like, “We need to declare the rights of avatars now, so that later, when your online identity is fragmented and owned by corporations, you have rights beyond tort law.”
We were actually thinking that big all the way back then.
So yes, I do think that. In fact, I’ll probably never write it, so if one of you wants to write it, go for it. I’ve had in the back of my head for a while now that there needs to be a book out there called Fix the Rules.
Because when I look at things like gerrymandering, proportional representation, the way in which the stock market works, they’re actually fairly simple game design problems. I think that game design trains us to see systems in a way that the ordinary person does not do, including—especially including—politicians.
So I actually think that we as a discipline have an enormous responsibility, because we’re the only group that’s been training ourselves to think this way.
So yes. Go forth, change the world, please.
AUDIENCE MEMBER: That’s incredible. Thank you.
[Audience applause.]