A Mixed-Reality System for Breast Surgical Planning

Abstract

One quarter of women who undergo breast lumpectomy to treat early-stage breast cancer in the United States undergo a repeat surgery due to concerns that residual tumor was left behind. This has led to a significant increase in women choosing mastectomy operations in the United States. We have developed a mixed-reality system that projects a 3D “hologram” of images from a breast MRI onto a patient using the Microsoft HoloLens. The goal of this system is to reduce the number of repeated surgeries by improving surgeons' ability to determine tumor extent. We are conducting a pilot study in patients with palpable tumors that tests a surgeon's ability to accurately identify the tumor location via mixed-reality visualization during surgical planning. Although early results are promising, it is critical but not straightforward to align holograms to the breast and to account for tissue deformations. More work is needed to improve the registration and holographic display at arm's-length working distance. Nonetheless, first results from breast cancer surgeries have shown that mixed-reality guidance can indeed provide information about tumor location, and that this exciting new use for AR has the potential to improve the lives of many patients.