Science Advances · 2023 · 50 citations · 63 references
EngineeringComputer ScienceHigh-performance ComputingComputer VisionMachine VisionPhotonicsIntegrated PhotonicsOptical ImagingOptical Signal ProcessingPhotonic Integrated CircuitsUltrafast OpticsHigh-performance Computing OptimizationHigh-performance Computing SystemsIntegrated Photonic DevicesOptical ComputingProcessing
Ultrafast dynamic machine vision in the optical domain can provide unprecedented perspectives for high-performance computing. However, owing to the limited degrees of freedom, existing photonic computing approaches rely on the memory's slow read/write operations to implement dynamic processing. Here, we propose a spatiotemporal photonic computing architecture to match the highly parallel spatial computing with high-speed temporal computing and achieve a three-dimensional spatiotemporal plane. A unified training framework is devised to optimize the physical system and the network model. The photonic processing speed of the benchmark video dataset is increased by 40-fold on a space-multiplexed system with 35-fold fewer parameters. A wavelength-multiplexed system realizes all-optical nonlinear computing of dynamic light field with a frame time of 3.57 nanoseconds. The proposed architecture paves the way for ultrafast advanced machine vision free from the limits of memory wall and will find applications in unmanned systems, autonomous driving, ultrafast science, etc.
63
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio et al. · Proceedings of the IEEE · 1998 · 58.2K citations · Full text
Vision meets robotics: The KITTI dataset
Andreas Geiger, Philip Lenz, Christoph Stiller et al. · The International Journal of Robotics Research · 2013 · 9.7K citations
Cramming More Components Onto Integrated Circuits
Gordon E. Moore · Proceedings of the IEEE · 1998 · 5.8K citations
Recurrent neural network based language model
Tomáš Mikolov, Martin Karafiát, Lukáš Burget et al. · 2010 · 5.4K citations