Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification

TLDR

CNNs achieve high performance in computer vision but incur high computational cost, hindering deployment in embedded systems despite algorithmic and hardware optimizations. We aim to improve image classification by adding an optical computing layer before electronic processing, minimizing added electronic cost. We design an optical convolutional layer using an optimized diffractive optical element and evaluate it in simulations of a learned optical correlator and an optoelectronic two‑layer CNN. Simulations and an optical prototype show that our hybrid systems match electronic CNN accuracy while substantially reducing computational cost.

Abstract

Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.