Image‐to‐Geometry Registration: a Mutual Information Method exploiting Illumination‐related Geometric Properties

TLDR

Image‑to‑geometry registration, inspired by multi‑modal medical imaging, typically relies on mutual information to align data from different sensors such as CT, X‑ray, and PET. The study aims to use mutual information between an image and rendered geometry, combined with illumination‑related geometric cues, to drive an iterative registration algorithm. The authors employ an iterative optimization framework that computes mutual information between the image and rendered geometry, integrating surface normals, ambient occlusion, and reflection directions to enhance registration. The approach shows that illumination‑related geometric properties can be leveraged for mutual‑information‑based registration, achieving robust performance across diverse real cases and allowing easy extension.

Abstract

Abstract This work concerns a novel study in the field of image‐to‐geometry registration. Our approach takes inspiration from medical imaging, in particular from multi‐modal image registration. Most of the algorithms developed in this domain, where the images to register come from different sensors (CT, X‐ray, PET), are based on Mutual Information , a statistical measure of non‐linear correlation between two data sources. The main idea is to use mutual information as a similarity measure between the image to be registered and renderings of the model geometry, in order to drive the registration in an iterative optimization framework. We demonstrate that some illumination‐related geometric properties, such as surface normals, ambient occlusion and reflection directions can be used for this purpose. After a comprehensive analysis of such properties we propose a way to combine these sources of information in order to improve the performance of our automatic registration algorithm. The proposed approach can robustly cover a wide range of real cases and can be easily extended.