Projects

Project Gotan: 'Garden - Mixed Reality' on Google Play | 2016

Position: Producer

Introduction

Garden is a mixed reality (MR) experience on the Project Tango mounted with the Durovis Dive headset. It allows the player to walk around freely in large spaces and avoid obstacles, while immersed in virtual reality (VR). It achieves this using real-time 3d reconstruction to bring the real world into virtual space. The real world is represented using minecraft-style voxels for easy rendering and texturing. We spawn game elements on tables and on the floors using the scanned geometry to recognize flat features. We also allow users to interact with these objects using just their hands and feet. With these features combined, we enable users to intuitively explore and interact with an alternate, virtual space as they do in the real world.

Game concept

The player is put into a virtual garden where they must kill worms that are destroying the garden and replant trees by filling in wireframes using the voxels created by their hands and feet.

Real-Time 3D reconstruction

Real-time 3D reconstruction is ongoing all the time, from the moment the player starts the experience till the headset comes off. This is done for several reasons. Firstly, it ensures moving objects in the real world can be seen by the player in the virtual world. Secondly, the player can move to new areas as they please. Thirdly, the user’s hands and feet can be detected as our 3D reconstruction algorithm adds and removes voxels as objects appear and disappear in the world. Lastly, if the virtual world drifts away from the real world in position and orientation, the player still can accurately avoid real world objects. We employ a simple point voting scheme over a 3D grid to determine the addition of voxels. This ensures that the 3D reconstruction is lightweight and efficient, and ensures that our games runs at 60 fps for user comfort in VR.

Hand and Feet Detection

The Project Tango (Yellowstone) depth sensor returns a point cloud of the environment at 5Hz. Each point cloud has only ~8K - 9K points. The low refresh rate and resolution makes it difficult to do reliable hand tracking. The player’s hands and feet appear in the world as voxels however, and we detect these voxels. We define trigger volumes (wireframes) that fire events when a voxel appears within it. These volumes are only active when the player comes close. Otherwise, it will move itself to “avoid” the voxel(s) that appear in it. This prevents any voxels that was previously unscanned from falsely activating the trigger (“stray” voxels). Finally, only when the player uses hands or feet to create voxels in the trigger volumes will the game event fire.

Feature detection

We need to spawn our game elements on the floors and table as part of the game concept. Our virtual world’s 3D grid is divided into groups of voxels called chunks. We select a column of chunks on the x-z plane. With the column, we perform feature detection on each chunk (from top to bottom) to find flat surfaces by calculating the ratio of exposed faces with upward normals over the total number of exposed faces. When we find a chunk that meets the criteria, we spawn the game element on the highest and most central voxel. We find that this helps in finding good spots to spawn with fewer “stray” voxels appearing in the trigger volumes.

Conclusion

We have created an immersive, untethered MR experience that allows the player to walk around the virtual world freely without bumping into real world obstacles. We allow the user interact with virtual objects intuitively without additional controllers as well. We bring in real world objects into the virtual world as voxels and place additional virtual objects on top of them. With this, we allow players to experience an alternative virtual space that resembles the real environment they are in.

 

Download from Google Play: https://goo.gl/Ngchpf  Try it out!



 

Project Hypnos VR: A VR Movie Project - <Imago> | 2015

 

Position: Producer/VR Designer/Marketing Manager

Nobody gets to see your dreams. As you experience life through Sam’s eyes, you are confronted with the dynamics of a family still coping with your devastating accident. Your sister, discouraged and alone, tries to reach out for some hope of connection, some affirmation that her best friend is still there, only to be yet again disappointed.

Little does she know the rich inner world of Sam's mind.

<Imago> is a 360 degree VR film experience that brings viewers into film like never before. Through drama, dance, and art, experience reality in an entirely new way.
For more info, see here: Current_VR_Project


 

 

Meltdown (Platform: CAVE +Makey Makey) | 2014

 

Position: Level Designer/3D Artist/Props Artist/Actor

The guest is escorted into the cave as a new employee of a nuclear power plant. An earthquake occurs and the guest is left in charge of rescuing two employees who were checking on the reactor at the time of the disaster by using a physical control console. The experience ends in one of three ways
1) the guest saves the employees and shuts the containment door
2) the guest fails to save the employees and locks them behind the containment door saving the city, but killing the two people and
3) the guest leaves the containment door open and the city explodes. 

p.s. A CAVE system is an immersive virtual reality interface usually consisting of an enclosed environment that surrounds the user with projected images(running 3 projectors to front and the side screens simultaneously)


 

 

Let's Smash Bugs! ( Chinese name: 全民打蟲蟲 ) (Platform: Xtion) | 2011-2012
Best Choice Award for the Taiwan Asus Xtion PRO Developer Challenge 2012

Position: Producer/Game Designer/Level Designer 

Motion-Sensing Physiotherapy and Entertainment was the goal of Let's Smash Bugs. I have 2 teammates in team to work on this project and devised the game to combine the concepts of physiotherapy with recreation and also psychological reconstruction. The simplicity in operation and the adorable visual style makes it easy for the public to get started with the game, and I used the time difference in hitting and speed difference of bugs to simulate the motion of physiotherapy in hand movement. I also took any type of players into consideration (the elderly, the disabled and children).

I was the producer, game designer and level designer for this project. My job was providing the game idea, supplying the detailed design, drawbacks checking, distribution of work, and integration and public marketing. By the project, we won the Best Choice Award for the Taiwan Asus Xtion PRO Developer Challenge 2012.


 

 

Blow Dude Blow ( Chinese name: 粉褲褲 ) (Platform: Jam-O Drum) | 2014

 

Position: Producer/Game Designer/2D Artist

Delay death for the falling man! Direct your balloon to blow the man towards the holes in the ground so he doesn't crash. Watch out though -- blowing too hard can bounce him back directly towards you again! A four-player Jam-O-Drum experience.

p.s. Jam-O Drum is the circular desk with drum pads embedded in a shared tabletop surface which only exists in Carnegie Mellon University's Entertainment Technology Center.


 

 

Nanodroid Rescue (Platform: Oculus Rift + PS Move) | 2014

 

Position: Experience Designer/3D Artist 

Nanodroid Rescue is the game that the player has to play as the nanodroid which means nano robot then accesses into a injured girl's stomach to kill the virus and bacteria. Oculus Rift is a good way to experience death, because people who are still alive may not have the experience being hit by the car or jumping from height. Therefore, what I want in this game is, giving player a chance to understand those two feelings, being hit by car and jumping from height.

In the beginning of this game, there is a intro which controls player's view of point. We need a injured girl so definitely we need to tell the player how the girl be injured. I designed the car accident, bus smash, and make the players know they fly away by eye view in Oculus Rift. After that, the player become the nanodroid and go into girl's stomach through her throat. Jumping from height happens at this part, a vertical "tunnel" from mouth to the stomach. This tunnel becomes my way to let players feel the height, just like jumping out of the window from 10 meters height.

In Nanodroid Rescue, I also created the models, textures and animations of White Blood Cell, Red Blood Cell, Bacteria, the whole city for the first scene and the bus which smashs the girl. According to the use of virtual world in Oculus Rift, I just want not only to make my players experience the death and but also have fun in gameplay.


 

 

Paint Arcade (Platform: Kinect) | 2014

Position: Game Designer/2D Artist/Props Artist

Granted the feeling of freedom, the guest is able to choose colours, create drawings and paint pictures that animate on a canvas.


 

 

Tank Race + Maze Escape + Shoot Gun (Platform: PC) | 2012

Position: Level Designer/Producer

This is the game designed by using Unreal Development Kit. The game includes different kinds of gameplay such as car racing, third-person and first-person shooting.

Facebook Twitter Instagram Blog LinkedIn share
online portfolio