The VR Interface of Dino Frontier

September 18, 2017

Dino Frontier recently launched and I want to share some of the cool new things we made for it. First things first, here’s our launch trailer to help you familiarize yourself with the game.

Dino Frontier is a PlayStation VR game where players build and manage a frontier settlement in a world where the wild west and Jurassic collide.

Players assume the role of Big Mayor. They build a town, manage settlers, gather resources, domesticate dinosaurs, fight bandits, and more. All this is done with a top-down perspective at tabletop scale.

Controls

For Dino Frontier we wanted to create a control scheme that was comfortable, powerful, and intuitive. As a second wave VR game we wanted to satisfy core gamers eager for new content. But we also wanted to be accessible to players who may be trying VR for the first time.

I’d like to think we achieved those lofty goals. But don’t take my word for it! Here are some reviewer quotes.

The tutorial turned me into an expert in just five minutes, and I can safely say that this tactile-focused experience is the best control scheme I’ve ever encountered in a strategy game. 
— GameCritics

The best part, though? At any time you can instantly and seamlessly resize the tabletop world. You just reach down, grab the earth, and stretch. It’s pinch-to-zoom controls applied to VR, and it rules. 
— Destructoid

Playing Dino Frontier frequently made me feel like Tony Stark operating one of his computers from the Iron Man franchise. […] The controls feel incredible. 
— DaddyDayDream

Camera

Two games I’ve previously worked on are Planetary Annihilation and Supreme Commander. If you’re familiar with either one then you’ll be familiar with strategic zoom. Which is the ability for players to zoom way out to see the whole battlefield.

Supreme Commander Screenshot Supreme Commander Screenshot Supreme Commander Screenshot

Dino Frontier has something similar. Players have full control over the tabletop. They can freely translate, rotate, and scale the playing surface.

This might sound like a recipe for nausea and motion sickness. With a naive implementation you’d be right! But using a few clever tricks it’s incredibly comfortable.

Table Operations

Translate

Rotate

Scale

Now let’s tie them all together to quickly move around the map.

Input

Dino Frontier requires two motion controllers. For PSVR this means two Move controllers. They have a trigger, big oblong face button, and four little buttons.

PlayStation Move Controllers

We only use the trigger and the face button. Two buttons per controller. The little buttons are too confusing. Especially when wearing a VR headset which prevents players from looking down at their controllers.

The trigger is used for context sensitive world interactions. Pinch, grab, open, use, etc.

The face buttons are used for table manipulation. Press and hold the face button on one controller to pan. Hold both face buttons to zoom or rotate. It works just like a touch screen; pinch to zoom.

VR Sickness

A great VR experience makes players feel like they’re in another world. A bad experience gives them a headache, nausea, and the spins. Designing for comfort is of the utmost importance.

There are multiple ways VR can cause discomfort. Some well understood and some we’re still figuring out. The basic cause is a mismatch between what your body feels and your eyes see.

In Dino Frontier player sit on their couch and look down at a tabletop. Using the controllers they can move, spin, and scale that table.

Doing this in an empty, ethereal space is discomforting. There’s no frame of reference. Is the table moving left or did I move right? You can’t tell, but your inner ear lets you know it’s not happy!

Our solution is to give the player a frame reference to ground themselves in. We use a background environment which is fixed in “real world” space so to speak. The player can move the table around all they want but those rocks stay in place.

When moving the table we turn on a subtle, fixed grid. This is especially important if the player is zoomed in and unable to see the background rocks.

Careful observers may note how the grid is positioned at table height, renders on top of everything, and fades out in the center of view. It provides peripheral stability.

Finally, we run at 90 frames per second. Running at 90 instead of 60 makes a huge difference in comfort. The bigger camera movements a game has the more important 90 is. This is far easier said than done!

Polish

One of the things that makes our controls feel crisp is our use of anchor points. When pressing the face buttons to move the table the hands establish an anchor point on the table.

As the player scales or rotates the hand stays attached to that anchor point. Pay close attention to the relative position of the trees in the follow image.

We apply momentum to table motion on release. This also helps the player navigate across the map quickly.

You may have noticed that all table motion happens on the ground plane. We keep the table at a fixed relative height. There’s a separate control for raising or lowering the table to a comfortable height.

User Interface

VR is a bold new frontier. Rules and best practices have not been established. Even more so in the simulation, tabletop-scale space.

World Interactions

Dino Frontier is a hands based game. The player has two motion controllers and two in-game hands. VR hands provide a visceral, intuitive experience.

World interactions are context sensitive and all use a single button; the trigger. Remember this is a two button game and one button is dedicated to the table controls!

We tried to have as little UI as possible. Interactions are far more literal than typical button filled video games.

Settlers aren’t selected with a cursor. They’re pinched with a hand.

Jobs are assigned by physically dropping settlers on a role site.

Promoting a settler to sheriff isn’t done by clicking a promote button. You physically assign a sheriff badge.

Watering trees is done by watering them with a watering can.

Palette

One of the more effective VR tools is the palette. Tilt Brush is the perfect example.

Tilt Brush Palette

The player holds a palette with their left hand and uses the right to select elements.

We have something kinda similar. Our problem is that our two buttons are already in use. We don’t have any free buttons to control the palette!

Our solution is put game buttons in “real world” space. These buttons are off to the side but always available. The player can be zoomed in or zoomed out and they’ll be in a known position and fixed size. Grabbing a button grabs a palette.

One of these palettes is the Build Palette. It holds buildings the player can place in the world. Hovering over a building produces a popup with details.

To build a Saloon you grab the Build Palette, pinch the Saloon, drop it in place, and thwack it with a hammer.

Popups

I hate words. Especially in video games. Sadly, you can’t always get around having a few.

When we do need words they appear on world space popups.

Sometimes those popups have buttons.

Toolbars

Normal sim games have an omnipresent toolbar which shows resource counts. That doesn’t work in VR.

Our solution is an always available wrist watch. It displays experience, resources, and objectives.

Misc Features

Here are a few random features worth a mention.

Tutorial

As previously mentioned I hate words. We heavily rely on animated gesture hints to teach the player mechanics.

In 20 seconds the player is taught how to pan the camera, zoom the camera, pickup a settler, and assign the settler to a task. All without using a single word. These gesture hints appear throughout the game when the player encounters a new mechanic.

Settlers

Settlers are semi-autonomous actors that have jobs and needs. Jobs are visually represented by their costume. Lumberjack, hunter, sheriff, etc.


Dino Fronter Screenshot Dino Fronter Screenshot Dino Fronter Screenshot

Needs are represented with Sims-like emoticons. Hungry, sleepy, sad, etc.


Dino Fronter Screenshot Dino Fronter Screenshot

Health and needs can be observed with a hover or pinch. To see even more information we use a palm gesture.

Mounts

All trained dinosaurs can be ridden by settlers. Making a rider is as simple as placing a settler on a free dino. Pinch them apart to dismount.

Conclusion

Making a tabletop scale VR game with motion controls requires solving a huge number of new design problems. There is no blueprint to follow.

Every type of interaction is potentially new to players. You have to consider both how they work and how to teach them to the player. We cut features that functioned but were too hard to communicate.

This post covers a few of the problems we encountered and how we solved them. I hope you learned a thing or two.

Thanks for reading.

Bonus! Technical Details

Here are a few implementation details if you want to create a similar control scheme.

In Dino Frontier it feels like you’re playing on a tabletop and it feels like you’re moving that table around. In truth the table is fixed at (0,0,0) and never moves.

When the player moves the table one way they actually counter-move the camera the opposite way. It makes the math a little funky, but it’s the right thing to do.

When I say we move the camera I actually mean a “camera rig”. The Unity render camera is attached to a child of an HMD transform. The HMD is a child to the rig. Somewhere in between are probably magical offsets and rotations related to platform tracking sensors.

Pseudocode

Here is some very approximate Unity pseudo code.

This doesn’t handle constraining movement to the ground plane. It doesn’t handle offsetting the camera y-position to give the illusion of a table at fixed height. It doesn’t handle momentum. It won’t work as is. But it’ll help you get started.

Everything is based around anchors. When the player presses a button it establishes an anchor in table space. When the hand moves we reset the camera rig and calculate how to counter transform the rig such that the hand appears fixed to its anchor.

if (leftPressedThisFrame)
    tableAnchorLeft = leftPositionInTableSpace
if (rightPressedThisFrame)
    tableAnchorRight = rightPositionInTableSpace

if (isTranslating) {
    // Translation only
    cameraRig.localPosition = Vector3.zero
    Vector3 anchorInCamera = cameraRig.InverTransformPoint(table.transformPoint(tableTranslateAnchor)) * cameraScale
    Vector3 currentInCamera = cameraRig.InverseTransformPoint(translateHand.worldPos) * cameraScale

    Vector3 offset = (anchorInCamera - currentInCamera)
    cameraRig.localPosition = cameraRig.TransformDirection(offset)
}
else if (isScaling) {
    // Scale
    worldAnchorDiff = worldFromTable(tableAnchorLeft) - worldFromTable(tableAnchorRight.worldPos)
    rawDiff = (right.worldPos - left.worldPos) / cameraScale
    camera.scale = (worldAnchorDiff.magnitude / rawDiff.magnitude)

    // Translation
    cameraRig.localPosition = Vector3.zero
    tableTranslateAnchor = (tableAnchorLeft + tableAnchorRight) / 2
    Vector3 currentWorld = (left.worldPos + right.worldPos) / 2

    Vector3 anchorInCamera = cameraRig.InverTransformPoint(table.transformPoint(tableTranslateAnchor)) * cameraScale
    Vector3 currentInCamera = cameraRig.InverseTransformPoint(translateHand.worldPos) * cameraScale

    Vector3 offset = (tableAnchorInCamera - currentInCamera).x_z(localheight - cameraEye.localPosition.y * cameraScale);
    cameraRig.localPosition = offset;

    // Rotation
    currentLeftInCamera = cameraRig.InverseTransformPoint(left.world) * cameraScale
    currentRightInCamera = cameraRig.InverseTransformPoint(right.world) * cameraScale
    diff = currentRightInCamera - currentLeftInCamera

    tableAnchorLeftInCamera = cameraRig.InverseTransformPoint(table.TransformPoint(tableAnchorLeft)) * cameraScale
    tableAnchorRightInCamera = cameraRig.InverseTransformPoint(table.TransformPoint(tableAnchorRight)) * cameraScale
    anchorDiff = tableAnchorRightInCamera - tableAnchorLeftInCamera

    angle = Vector3.Angle(anchorDiff, currentDiff)
    if (Dot(Cross(currentDiff, anchorDiff), Vector3.up) < 0)
        angle = -angle;

    rotateCenter = (worldLeft + worldRight) / 2
    cameraRig.RotateAround(rotateCenter, Vector3.up, angle)
}