MapEval: Towards Unified, Robust and Efficient SLAM Map Evaluation Framework

Abstract

Evaluating massive-scale point cloud maps in Simultaneous Localization and Mapping (SLAM) still remains challenging due to three limitations: lack of unified standards, poor robustness to noise, and computational inefficiency. We propose MapEval, a novel framework for point cloud map assessment. Our key innovation is a voxelized Gaussian approximation method that enables efficient Wasserstein distance computation while maintaining physical meaning. This leads to two complementary metrics: Voxelized Average Wasserstein Distance (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">AWD</monospace>) for global geometry and Spatial Consistency Score (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SCS</monospace>) for local consistency. Extensive experiments demonstrate that MapEval achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$100$</tex-math></inline-formula>-<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$500$</tex-math></inline-formula> times speedup while maintaining evaluation performance compared to traditional metrics like Chamfer Distance (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CD</monospace>) and Mean Map Entropy (<monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MME</monospace>). Our framework shows robust performance across both simulated and real-world datasets with million-scale point clouds.