ComputingVisualizationReal-time RenderingRenderingVolume Visualization3D RenderingAugmented RealityMixed RealityVirtual RealityVolume RenderingVolumetric RenderingCamera CalibrationCamera Self-calibrationDynamic CalibrationEnd-to-end DelayEnd-to-end LatencyLow-latency RenderingMultimedia StreamingObject Segmentation
While recent work explored streaming volumetric content on-demand, there is little effort on live volumetric video streaming that bears the potential of bringing more exciting applications than its on-demand counterpart. To fill this critical gap, in this paper, we propose MetaStream, which is, to the best of our knowledge, the first practical live volumetric content capture, creation, delivery, and rendering system for immersive applications such as virtual, augmented, and mixed reality. To address the key challenge of the stringent latency requirement for processing and streaming a huge amount of 3D data, MetaStream integrates several innovations into a holistic system, including dynamic camera calibration, edge-assisted object segmentation, cross-camera redundant point removal, and foveated volumetric content rendering. We implement a prototype of MetaStream using commodity devices and extensively evaluate its performance. Our results demonstrate that MetaStream achieves low-latency live volumetric video streaming at close to 30 frames per second on WiFi networks. Compared to state-of-the-art systems, MetaStream reduces end-to-end latency by up to 31.7% while improving visual quality by up to 12.5%.
35
ORB: An efficient alternative to SIFT or SURF
Ethan Rublee, Vincent Rabaud, Kurt Konolige et al. · 2011 · 10.4K citations
3D is here: Point Cloud Library (PCL)
Radu Bogdan Rusu, Steve Cousins · 2011 · 4.8K citations
Understanding Convolution for Semantic Segmentation
Panqu Wang, Pengfei Chen, Ye Yuan et al. · 2018 · 2K citations