Work Human Computer Interaction: Surface and Gesture Interfaces

2D Sensing – Surfaces
Through a grant from Microsoft, and in cooperation with the USC School for Cinematic Arts, we are currently researching surface computing as an interface for authoring emotionally rich content. We are particularly interested in fluid interfaces that maximize the creativity of the user, allowing the user to control form and expression. Thus, we are exploring families of interface rhetoric that:

  • do not impose rigid order of operation sequences or abstract grammars,
  • do not interrupt the user with information, warnings, or questions,
  • and do not impose strong design aesthetics or emotional distraction.

The creative workflow today, whether it is a 3D modeling pipeline for a game, or a digital recording studio, is a collaborative, emotionally expressive space. Our aim is to create calm interfaces that do not seize the spotlight from the content. Such interfaces must enable authors to collaborate, rapidly generate content ideas, and keep track of multiple variations.

3D Sensing – Gestures
Gestures are an important part of communication, lending emphasis, magnitude, indication, and direction to spoken language. More subtle gestures can also communicate tone, emotion, confusion, and other rich qualities of interaction.

Our research lab is currently investigating human-computer interactions made possible through depth sensing cameras, full body motion capture systems, and other tracking systems. Virtual humans in virtual reality training scenarios can be made to recognize and respond to user gestures. Multiple virtual humans might also be puppeteered in near real-time to create unique and customized training. Other applications include novel game play, interactive data visualizations, physical therapy, and rehabilitation.