A Better Gamepad Keyboard: Part 1, A Survey of Extant Gamepad Keyboards

(This article is also available as a devblog on Gamasutra.)

Slashcards: Learn Japanese! Gameplay

Slashcards is a game about learning language. And part of rehearsing new language knowledge is responding with English equivalents of pronunciation. The most thorough way of testing players is for them to input that pronunciation letter by letter. So, one interface challenge I embraced for Slashcards’ design was to make a game-friendly gamepad-compatible onscreen keyboard.

Part 1: A survey of extant gamepad typing interfaces.

In an effort to avoid re-inventing the wheel — and falling into old ruts — I did some research on to typing on consoles. Perhaps the most naive approach of all was to make a matrix of buttons where each button corresponded to a key on the keyboard. The user would then navigate through the matrix like any kind of on-screen GUI, going from GUI element to GUI element by pressing the d-pad or left stick in the corresponding direction. This has remained ubiquitous ever since 8-bit consoles. Here’s The Legend of Zelda (NES, 1986):

The Legend of Zelda

…and here’s the Xbox 360, 20 years layer: (around its launch, ~2006)

XBox 360 ABC Keyboard

The Xbox 360 keyboard originally defaulted to an ABC layout. I guess ABC-order is defensible insofar as we all know the alphabet, so, given a letter, we would intuit the neighboring left and right letters. But what about the letters above and below? It’s frustrating that, for example, P is to the left of L. Moreover using the wrap-around topology of the keyboard — pressing left on the left edge to go to the right edge, and vice versa — would similarly be a convenience only available to those who carefully studied this unnatural layout.

Eventually — perhaps from release, I don’t recall — you could change the layout of the Xbox 360 on-screen keyboard to a QWERTY layout, and leverage the familiarity every modern person has from thousands of hours of typing. Now you could hunt-and-peck like you were typing on your laptop as you would while you ate cereal or nudged some stir-frying onions with the other hand. I believe this is now the default layout on the Xbox One.

Xbox Keyboard QWERTY

Sony has offered a bizarre multi-step interface, whereby users hunt-and-peck a letter family on the left. Then a number of options including auto-complete suggestions are presented on the right. Finally a user selects one of those options by again navigating across them. The result is a mixed bag. Letters that were far from each other in the above keyboards are closer in terms of button press-count, but previously-adjacent letters might be five or six button-presses in the PSP system. Maybe this compromise plus the autocomplete suggestions turns out to give more words per minute — I remember my time typing on the PSP as an exercise in frustration where I had to relearn the system every time I encountered it. Either way, the bottom line is that any design that relies on autocomplete suggestions is not going to be appropriate for Slashcards.

PSP keyboard

Fortunately for PSP fans (and the PSP’s successor, the Vita) eventually the PSP also offerred a full keyboard layout (known as the “fullscreen keyboard” on the PSP.

A far better performing option is the approach taken by the Wii and the PS4 on-screen keyboards. They offer a QWERTY keyboard whereby the player can more or less point to a key to select it. (The PS4 requires players to tilt, not exactly point, but the result feels responsive and intuitive.)

PS4 accelerometer keyboard

Steam Big Picture mode has an interesting hierarchical keyboard. The user presses a direction with the left stick and selects one of the four action buttons to select a character.

Steam Big Picture Keyboard

The more you use this keyboard, the faster you’ll type — and the skill ceiling is far higher than the hunt-and-peck keyboards above. It also has the virtue of being alphabetical but without the compromise of arbitrary rows that computer-keyboard-lookalike layouts have. At every letter you can easily see if your next layer is 1) in the current button group, 2) counter-clockwise (previous to the current letter) or 3) clockwise (after the current letter.) This is the first system that really tempted me towards implementation in Slashcards. I struggled mightily to compress it to a size that wouldn’t be so demanding of screen real estate. I couldn’t come up with a workable solution but you’ll see that it has something in common with the Japanese input solution I devised.

My first thought was to take the extant QWERTY hunt-and-peck keyboard that we see on the Xbox/PS and add another cursor. Players could hunt-and-peck with the left stick and right stick. This seemed obvious enough that I wondered if this had been attempted before — sure enough, a Google search hit showed me that the approach is explored to some extent in a Microsoft research paper.

Their research finds study participants went from 5.8 words per minute on the single-stick hunt-and-peck QWERTY keyboard to 6.4 words per minute with a dual-stick split hunt-and-peck QWERTY keyboard. A 10% gain is still far from the kind of quantum leap I’m looking for.

In the end, I found that none of these options would work to type in even a few letters in a real-time action game situation. Slashcards demands better!–or rather, a more precisely-designed solution.

In the next part, I’ll show how I mixed and matched the best of the above approaches to iterate towards a functional, consistent, and radically more efficient on-screen gamepad keyboard.

Slashcards Unity Quick Tip 2: Bringing a face to life

This is a continuation of Quick Tip 1, which discusses the coding approach to the faces in Slashcards: Learn Japanese.

Character still.

Let’s bring our character to life.

Plop her in your fantasy world and what would she be doing? She’d be looking around, thinking and reacting to what she saw. That means she’s blinking. She’s raising her eyebrows or furrowing them as she thinks about this or that. She’s pursing her lips and relaxing them. And none of this is happening in a rigid, repeating pattern. It’s all semi-random. Sometimes she blinks three times in three seconds. Sometimes once in ten seconds. Something will worry her for just the briefest moment…etc.

For my character, I’ve broken facial activity into four processes:

  • Blinking
  • Looking-at
  • Eyebrow movement/shape
  • Eye shape

Moreover, the character has a Mood and a PointOfInterest.

Blinking is a coroutine that blinks every so often (from .5 to 5 seconds). It’ll blink more often if the mood is Afraid or Sad.

LookingAt is a coroutine that aims the pupils at the PointOfInterest if there is one. Otherwise it just picks a different thing/point in space to gaze at every several seconds.

Eyebrow movement/shape shifts the eyebrows up and down at random intervals. The eyebrows’ shapes are determined by the mood.

And eye shape — angry, relaxed, afraid, and so on…

Eye shapes by deviantartist sharkie19.  You can imagine how you might break out eyebrow, pupil, and eye shape.

…are controlled by a coroutine that varies them according to the mood. Certain moods use different sets of eyeshapes. For example, in Mood.Relaxed, the character might momentarily be pensive/worried — an eyeshape also used in Mood.Afraid — but it won’t use that expression as often.

Slashcards facial animation

Crucially, this approach shows the player that the character has an inner life. Keeping the control in code makes it easier to vary. And the varied behavior means the designer/animator doesn’t have to handcraft every little reaction.

Slashcards Unity Quick Tip 1: Fast Faces

One thing I wanted to achieve in Slashcards (check out the playable preview at itch) was characters that had some character. I wanted them to feel alive. One game that does that superbly well is The Legend of Zelda: Wind Waker.

Because of the specific artistic choices in that game, the characters faces were basically drawn on as if cartoons. That permitted a ton of expression with a bit less work than, say, modeling a humanoid face, rigging it, and using motion capture or hand-crafted animation for each little expression.

To make expressive faces, you first and foremost need eyes and eyebrows. Eye shape and eyebrow position tell you just about everything you need to know about someone’s emotional state. I wanted to be able to control the eye pupil position by script, so rather than have canned images fr looking left, looking right, etc., I chose to have the pupil be a separate image from the white of the eye. And that meant masking the eye, which means shaders, which means more draw calls…

Two eye brows, two eyes with two pupils, and we’re already up to six draw calls per character, and we haven’t even gotten to the mouth, yet. The trick to keeping this cheap is (in Unity) using MaterialPropertyBlocks.

The drawback of MPBs is that you have to set them in code. And if you want to see the results in the editor, you’ll need to have a script that sets their values in OnValidate.

So the recipe is: use the same texture, use the same shader, and to vary parameters across different facial elements, use MateriapPropertyBlocks. Next up, I’ll get into how they’re animated.

looks painful..
Hm, looks painful…

Slashcards: Learn Japanese! Alpha Preview

Learning your first kana

So, the Slashcards 7DRL morphed away from a roguelike RPG and more into an action game with some light RPG progression elements.  Slashcards: Learn Japanese!: 一緒に勉強しよう! is an action-RPG where your mastery of reading (and writing!) Japanese is your character’s power. You’ve been summoned to go on an epic journey that will have you battling monsters, mastering the power of the Slashcards, and rescuing the denizens of a land accursed.

The focus of this project is to make the otherwise tedious task of rote learning (memorization) a seamless and central part of an action game.

Even if you plow through the dialog and introductions, and go in blind, the game will still give you hints, let you review in-game, and you can finish a level having really learned something.

Enemies have characters or kana right on their bodies. If you scope out enemies you “know” well, you can prioritize your attack — or step back to find some safety so you can review in-game.

(in-game review – you’re seeing 上手 being written mid-stroke animation)

Or just barrel in and give it a shot — a miss will give you a hint to the right answer, if you survive being stunned (and you probably will.)

Attack an enemy and the faster and more accurately you respond, the more damage you’ll do, the more experience you’ll earn, and the better you’ll be able to evade the monsters that are still chasing you! And by “responding”, I mean choosing from multiple choices, typing in English or Japanese, or even drawing a character stroke-by-stroke.

I’m a learner myself. So I’ve gotten help putting together the lessons, the level order, and I’ve even male + female voice recordings for the words and letters in the game. They’re not all in there yet, but most of them are.

On top of that, Slashcards has been built for multiplayer co-op and competitive modes from the very beginning. So you can adventure across the land with your friends…

some co-op gameplay

…or try one of the hectic versus modes (with CPU opponents, if you want!)

Bingo Battle!

There’s tons more for me to talk about (and to work on), but I’m hoping I’ll be able to update this regularly.

I’m anxious to get this rolling on Greenlight, but first I’d like to get a working demo going. Do me a favor give the preview a try and let me know what you think!

>> Download the pre-release demo: Slashcards: Learn Japanese. <<

You can play the first couple levels and try one of the versus modes, Bingo Battle. You’ll be treated to three original music tracks and one recycled one from an old game of mine.