Research Study: Spatio-temporal Motion Tracking with Unsynchronized Cameras

Markerless motion capture.

We present a new spatio-temporal method for markerless motion capture. We reconstruct the pose and motion of a character from a multi-view video sequence without requiring the cameras to be synchronized and without aligning captured frames in time. By formulating the model-to-image similarity measure as a temporally continuous functional, we are also able to reconstruct motion in much higher temporal detail than was possible with previous synchronized approaches. By purposefully running cameras unsynchronized we can capture even very fast motion at speeds that off-the-shelf but high quality cameras provide.

We have introduced a spatio-temporal approach to articulated motion tracking from unsynchronized multi-view video. Unlike previous approaches that rely on synchronized input video, our method makes use of the additional temporal resolution to successfully track fast moving actors with low frame-rate cameras. It also enables setting up simpler and cheaper capture setups, as there is no need anymore for hardware based synchronization and high frame rate cameras.