Integration of a Time-of-Flight Camera into a Mixed Reality System for Handling Dynamic Scenes, Moving Viewpoints and Occlusions in Real-Time

Bogumil Bartczak, Ingo Schiller, Christian Beder, Reinhard Koch

2008 · 28 citations · 14 references

Concepts

Abstract

We present a novel approach to mixed reality applications. The key characteristics of the presented system are the use of a automatically generated static environment model and a time-of-flight camera device. The combination of both allows the correct handling of mutual occlusion between real and virtual content on the fly, which is not possible with the currently applied approaches. Typically expensive studio setups with complex camera tracking installations and multi-camera approaches in combination with chroma-keying facilities are used. Our system is rather inexpensive, compact, mobile, flexible and provides convenient calibration procedures. The use of a background model not only eliminates the need for chroma-keying in mixed reality production, it moreover supports planing and alignment of virtual content. Based on depth information the system is generating appropriate depth maps in realtime, making the approach suitable for 3D-TV productions. The presented paper discusses all key elements of mixedreality applications based on our approach. This includes camera pose tracking, correct real-time handling between interacting virtual and real content and the fast environment model building. 1.

References

14