3 Ways to Train Your VR Players to Drive a Non-human Body
Today we’ll be talking about how we as builders of virtual worlds can give our players new virtual bodies. Too many current games use our hands as hands. They’re missing out on all the fun! I don’t want at hand at all, I want a set of razor-sharp clippers. I don’t want two arms — I want three! Guess what, some neuroscientists already tried that out and it totally works. I don’t want to be a human at all, I want to be a dinosaur stomping around!
This is a transcript of the video from the series, Embodied Reality.
Turns out we can figure out how to control three arms in only a few minutes of training. Researchers at the Stanford Virtual Human Interaction Lab put players in a body with an arm protruding out of the center of their chest. They were asked to grab boxes off of shelves. They controlled the third arm with a small flick of their wrist. In only 2 ½ minutes of play they could grab boxes *faster* than doing the task with two arms. What!? They drove the body with three arms better than with two. We spent our whole lives with two arms but In two and a half minutes we can control three arms better! This is wild!
In VR, we can give players a virtual body that they can *quickly* figure out how to control. Throw the term “homuncular flexibility” into Google Scholar and you’ll find out more than you could ever want to know. I read the dry academic papers so you don’t have to. There are links at the bottom to some of my favorite studies.
These ideas of driving a novel body is already getting integrated into VR content. Life of Us, a multi-player room-scale experience got a lot of buzz at Sundance last month. Life of Us takes you along an epic evolutionary timescale from the era of single-celled organisms to dinosaurs to a human today and a future robo-bodies. You inhabit the body of a bird, and flap your wings to soar along. As an ape you wander around with your ape friend and throw the ape babies around. As a tadpole you blow bubbles by singing into the mic. They modulate your voice to make it sound like you’re underwater. Nice touch, y’all! High fives!
As a pterodactyl you can yell to breathe fire. I love that they mapped audio input to be a controller. So smart!! Think of *all* the potential inputs you have at your disposal and brainstorm what input you might map to controlling the virtual bodies in your experience. I’m hoping Life of Us adds more interactivity with these animal bodies before it releases later this year.
The key to making these shenanigans of driving a non-human body possible is high-fidelity motion tracking. Current hand controllers can map motions of the wrist. As hardware develops to capture more and more of the body’s motion, we will have many more tools in our toolbox. Let’s think forward into the future a tiny bit to a world where we have full-body motion capture available on mainstream hardware.
Imagine your player begins on level one with a body plan similar to ours, with two arms and two legs and one head. Say a kitty. It’s different, but not too different. Kitties have two arms and legs, but they also have a tail! A tail! I always wanted a tail! As researchers from University College London demonstrated, players can learn to control the tail with slight sways of the hips. These motions are quite natural and don’t require much training.
To speed the learning process along, you can have a friend in the same body model the motion. Give your player who’s newly inhabiting a kitty body a NPC kitty friend to show her the ropes. If your player sees her kitty friend slap things with her tail and win points, that will trigger her to think “Oh right, I have a tail!” and immediately start figuring out how to move her tail.
Another technique to speed along training is to use a virtual mirror to show them how the motions of their real body translate to their new virtual body. It can be challenging to realize you’re in a new body if you can’t see yourself.
The last trick is to make a duplicate avatar so that they’re looking at themselves from a third-person perspective. The cow you look over at standing in the field is actually you. When you pick up your hand, she picks up her hoof.
When the player can see their virtual body, they figure out how to control it much faster. A lot of the process happens automatically. Yep, it really is that simple. Participants in the study with the third arm often figure out how to move it without any instruction. What’s weirder, they say things like “I have no idea what I did to move it. I just did.” Y’all brains are so cool!!
As your player becomes more and more comfortable learning to drive a novel body, you can increase the difficulty by making the body less like a human one. Player can progress from kitty with four legs to ant with six to Lobster with eight — plus two pincers you have to figure out how use to snip the wittle sprigs of parsley off the stems. — What? You never fantasized about being a lobster chef cooking up a nice vegetarian bouillabaisse? But I digress.
You can keep the sense of play from novel sensations going by deviating further and further from human. How cool would it be to drive a body of a millipede with all their tiny legs, or a snake wiggling across a sand dune?! Someone please go build this for me so I can play it!
Today we talked about getting creative with mapping inputs to control brand new appendages like tails and third arms. Make sure players can see their actions so they can quickly learn new gestures and actions. And give players NPC friends who model the actions so they can mimic them.
I want to try out as many of VR experiences as I can get my hands on. If you know of some that already make use of this technique, please send them my way on twitter at @crystaldbeasley. Can’t wait to hear from y’all!
More episodes are available either in transcript for here on Medium or as a video series on YouTube at Embodied Reality.
