Evan A. Suma, Belinda Lange, Skip Rizzo, David Krum, and Mark Bolas

Project Email Address: faast@ict.usc.edu

NEW!

Download FAAST 1.0

Note: the new version of FAAST only supports discrete mouse events at this time.  If you need continuous mouse control, you may continue to use the previous version, available here.  We are currently working on adding improved mouse control to the new version, and will release an update soon.

You may also view our online video gallery, which contains videos that demonstrate FAAST’s capabilities, as well as interesting applications that use the toolkit.

Recent News

March 30, 2012
We are excited to announce the release of FAAST 1.0, which now supports both skeleton trackers from OpenNI and Microsoft Kinect for Windows!  New features also include: a new graphical user interface for building sophisticated gestures, VRPN streaming for up to 4 skeletons, and the ability output complicated macros directly to applications via keyboard, mouse, and VRPN button commands.

Summary

FAAST is middleware to facilitate integration of full-body control with games and VR applications using either OpenNI or the Microsoft Kinect for Windows skeleton tracking software. FAAST includes a custom VRPN server to stream up to four user skeletons over a network, allowing VR applications to read the skeletal joints as trackers using any VRPN client. Additionally, the toolkit can also emulate keyboard input triggered by body posture and specific gestures. This allows the user add custom body-based control mechanisms to existing off-the-shelf games that do not provide official support for depth sensors.

FAAST is free to use and distribute for both commercial and noncommercial purposes.  However, you must still abide by the licensing terms of any third party software you install for skeleton tracking (either OpenNI software or Microsoft Kinect for Windows).  Please see the websites of these libraries for more information.

E. Suma, B. Lange, A. Rizzo, D. Krum, and M. Bolas, “FAAST: The Flexible Action and Articulated Skeleton Toolkit,” Proceedings of IEEE Virtual Reality, pp. 247-248, 2011.

FAAST is currently available for Windows only.

Support

FAAST can easily be integrated with existing game engines, VR toolkits, and other applications using a networked client-server architecture provided by the Virtual Reality Peripheral Network (VRPN).  We have also developed a plugin for the Unity3D engine, which we will be posting online soon.  FAAST is also officially supported by several commercial toolkits, including 3DVIA Studio and the Vizard toolkit from WorldViz.

Installation

To run FAAST, you first need to install a skeleton tracker.  Both the OpenNI and Microsoft Kinect for Windows implementations are supported.  Only one tracker needs to be installed, and you can select the one you want to use at run-time.

Option 1: Microsoft Kinect for Windows

To use the Microsoft skeleton tracker, you must download and install the Kinect for Windows SDK.

Option 2: OpenNI

To use the OpenNI skeleton tracker, you must download and install the following software packages directly from our website:

1. PrimeSense Package (includes OpenNI, NITE, and PrimeSensor Module)
2. Microsoft Kinect Driver

Alternatively, if you want to install everything individually, you need to visit the OpenNI website and download each separate component (for advanced users only).

1. OpenNI Unstable Build for Windows x86 (32-bit) v1.5.2.23
2. PrimeSense NITE Unstable Build for Windows x86 (32-bit) v1.5.2.21
   
3. Hardware sensors drivers
   • Microsoft Kinect: SensorKinect091-Bin-Win32-v5.1.0.25.msi (available on GitHub)
   • PrimeSensor: PrimeSensor Module Unstable Build for Windows x86 (32-bit) v5.0.3.4

Switching Between Trackers

You can have both the OpenNI and Microsoft software installed at the same time.  However, when you plug in your Kinect, only one set of drivers will be detected.  Therefore, to switch between the two implementations, you will need to uninstall the currently detected driver, disconnect the sensor, reinstall the driver from the other software library, and then reconnect the sensor.

Additional Setup

FAAST should then run out-of-the-box; no additional installation or setup is necessary. If you encounter an error on startup, you may also need to install the Microsoft Visual C++ 2010 Redistributable Package.

Skeleton Usage

FAAST streams up to four user skeletons over a VRPN server.  The four skeletons are identified as as Tracker0, Tracker1, Tracker2, and Tracker3.  For example, to read from the first skeleton, you should connect to “Tracker0@ip_address” in your VRPN client (“Tracker0@localhost” if running on the same machine as the client). The server automatically starts when the toolkit connects to a sensor.  For each skeleton, a total of 24 skeleton joint transformations (including position and rotation) are streamed as sensors. Corresponding to the OpenNI framework, the joints are ordered as follows:

Note: The joint positions and orientations can be streamed in either the world coordinate system or local coordinate system (each joint relative to the parent).  Positions are reported in meters relative to the sensor’s position and orientations are reported as quaternions, in accordance with the VRPN standard units.

Input Emulator Usage

FAAST includes a graphical user interface for designing custom gestures and mapping to a series of input events.  Multiple input and output events can be specified either simultaneously or in sequence.  Any event listed immediately after a previous event will be considered simultaneous (e.g. moving two hands forward at once).  Two or more events separated by a time will be treated as sequential, allowing for more complicated movements over a period of time (e.g. drawing a specific symbol in the air).

More detailed documentation, along with a video tutorial, will be posted soon.