Heroes

A Duet in Mixed Reality

This is a dance piece presented in three media. You first encounter the piece on a screen as an ambient, abstract art piece. Next you view the piece as a dynamic and personal 360 video in a virtual reality headset. Finally you enter a room wearing an augmented reality headset and can explore and interact with the dance in an unbounded and natural way.

In the VR experience you travel through the iconic Ace Hotel in L.A. You are brought on stage amongst the dancers, they accompany you into the lobby, grow in scale, and finally join you on the roof.

The AR experience is built on extensive motion capture of the dance and detailed photogrammetry of the stunning Spanish Gothic architecture of the Ace Hotel. The dance and architecture is presented as dollhouse-scale vignettes around the room which you can interact with via movement and voice.

My Role & Responsibilities

I assisted as a developer on the project, primarily working on integration and networking for the AR (HoloLens) experience.

There are moments in the AR experience where you can interact with the dancers with your hands. We wanted a more natural interaction than the HoloLens provides. I created a system which uses a separate 3d camera in the room to track your hands and allows you to “hold” the holographic dancers.

We needed to show the piece continuously throughout the weeklong festival. This meant it was necessary to streamline device management, troubleshooting, and audience communication. I developed an Android app which keeps tabs on all the HoloLens devices and lets us know various system details such as which app is running and how much battery is left. It also shows us where in the experience the audience member is, so we can direct their attention to different parts of the room in case they get lost.

Collaborators

• Melissa Painter Creative Director
• Thomas Wester Creative Technologist
• Helios Dance Theater Choreography and Dancers
• Moving Picture Company Motion Capture and VR