美国华盛顿大学Sheng Wang等研究人员合作发现，一个基础模型可用于跨九种模式的生物医学物体的联合分割、检测和识别。2024年11月18日，《自然—方法学》杂志在线发表了这项成果。

研究人员提出了一个生物医学基础模型BiomedParse，可以跨越九种成像模式联合进行分割、检测和识别。这种联合学习提高了各个任务的准确性，并启用了新应用，例如通过文本描述分割图像中所有相关物体。为了训练BiomedParse，研究人员通过利用现有数据集中的自然语言标签或描述创建了一个大型数据集，包含超过600万个图像、分割掩码和文本描述的三元组。

研究人员展示了BiomedParse在九种成像模式下的图像分割任务中优于现有方法，且在处理不规则形状物体时的改进更为显著。研究人员还展示了BiomedParse可以同时分割并标注图像中的所有物体。总之，BiomedParse是一个适用于所有主要成像模式的生物医学图像分析的全能工具，为高效、准确的基于图像的生物医学发现铺平了道路。

研究人员表示，生物医学图像分析是生物医学发现的基础。整体图像分析包括如分割、检测和识别等相互依赖的子任务，传统方法通常将这些任务分开处理。

附：英文原文

Title: A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities

Author: Zhao, Theodore, Gu, Yu, Yang, Jianwei, Usuyama, Naoto, Lee, Ho Hin, Kiblawi, Sid, Naumann, Tristan, Gao, Jianfeng, Crabtree, Angela, Abel, Jacob, Moung-Wen, Christine, Piening, Brian, Bifulco, Carlo, Wei, Mu, Poon, Hoifung, Wang, Sheng

Issue&Volume: 2024-11-18

Abstract: Biomedical image analysis is fundamental for biomedical discovery. Holistic image analysis comprises interdependent subtasks such as segmentation, detection and recognition, which are tackled separately by traditional approaches. Here, we propose BiomedParse, a biomedical foundation model that can jointly conduct segmentation, detection and recognition across nine imaging modalities. This joint learning improves the accuracy for individual tasks and enables new applications such as segmenting all relevant objects in an image through a textual description. To train BiomedParse, we created a large dataset comprising over 6 million triples of image, segmentation mask and textual description by leveraging natural language labels or descriptions accompanying existing datasets. We showed that BiomedParse outperformed existing methods on image segmentation across nine imaging modalities, with larger improvement on objects with irregular shapes. We further showed that BiomedParse can simultaneously segment and label all objects in an image. In summary, BiomedParse is an all-in-one tool for biomedical image analysis on all major image modalities, paving the path for efficient and accurate image-based biomedical discovery.

DOI: 10.1038/s41592-024-02499-w

Source: https://www.nature.com/articles/s41592-024-02499-w