近日，美国康奈尔大学的Peter L.McMahon课题组成功实现了多层非线性光神经网络图像感应。相关成果已于2023年3月23日在国际权威学术期刊《自然—光子学》上发表。

该研究报道了一种基于商业图像增强器作为光学-光学非线性激活函数的非线性多层光学神经网络（ONN）编码器，用于图像感应。这个非线性的ONN在多个代表性任务上优于相似大小的线性光学编码器，包括机器视觉基准测试、流式细胞术图像分类和三维打印真实场景中物体的识别。对于机器视觉任务，特别是那些具有不相干宽带照明的任务，他们的概念可以大大减少相机分辨率和电子后处理复杂性的要求。总之，使用ONN进行图像预处理可实现使用更少的像素、更少的光子、更高的吞吐量和更低的延迟进行准确操作的图像感应应用。

据悉，光学成像通常用于工业和学术界的科学和技术应用。在图像感应中，通过对数字图像进行计算分析来执行测量，例如测量物体的位置或轮廓。一种新兴的图像感应范式依赖于光学系统，它们不是进行成像，而是通过提取显著特征将图像光学压缩到低维空间中作为编码器。然而，这些编码器的性能通常受到它们的线性限制。

附：英文原文

Title: Image sensing with multilayer nonlinear optical neural networks

Author: Wang, Tianyu, Sohoni, Mandar M., Wright, Logan G., Stein, Martin M., Ma, Shi-Yuan, Onodera, Tatsuhiro, Anderson, Maxwell G., McMahon, Peter L.

Issue&Volume: 2023-03-23

Abstract: Optical imaging is commonly used for both scientific and technological applications across industry and academia. In image sensing, a measurement, such as of an object’s position or contour, is performed by computational analysis of a digitized image. An emerging image-sensing paradigm relies on optical systems that—instead of performing imaging—act as encoders that optically compress images into low-dimensional spaces by extracting salient features; however, the performance of these encoders is typically limited by their linearity. Here we report a nonlinear, multilayer optical neural network (ONN) encoder for image sensing based on a commercial image intensifier as an optical-to-optical nonlinear activation function. This nonlinear ONN outperforms similarly sized linear optical encoders across several representative tasks, including machine-vision benchmarks, flow-cytometry image classification and identification of objects in a three-dimensionally printed real scene. For machine-vision tasks, especially those featuring incoherent broadband illumination, our concept allows for a considerable reduction in the requirement of camera resolution and electronic post-processing complexity. In general, image pre-processing with ONNs should enable image-sensing applications that operate accurately with fewer pixels, fewer photons, higher throughput and lower latency.

DOI: 10.1038/s41566-023-01170-8

Source: https://www.nature.com/articles/s41566-023-01170-8