基于深度學習的病理

計算機視覺/深度學習/醫學影像 (COMPUTER VISION/ DEEP LEARNING/ MEDICAL IMAGING)

In the last part, we started an introductory discussion on the present state of Deep Learning in histopathology, we discussed Histopathology, Digital Histopathology, the possibilities of Machine Learning in the area, the various applications, followed by a detailed discussion of the challenges involved in working with Digital Microscopic Slide Images and in the application of Deep Learning Algorithms to them.

在最后一部分中 ，我們開始就組織病理學中深度學習的現狀進行介紹性討論，討論了組織病理學，數字組織病理學，該領域機器學習的可能性，各種應用程序，然后詳細討論了其中涉及的挑戰與數字顯微幻燈片圖像配合使用，并在它們中應用深度學習算法。

In this blog, we shall be discussing in greater detail the applicability of Deep Learning to Histopathology from a methodological perspective along with the tasks it helps accomplish using relevant work for illustration.

在這個博客中，我們將從方法論的角度更詳細地討論深度學習在組織病理學中的適用性，以及使用相關工作進行說明有助于完成深度學習的任務。

The applicability of deep learning can be studied in terms of the tasks it performs or in terms of the learning paradigm, which is the classification we shall be using in this writeup. The different learning algorithms, viz a viz Deep Learning for histopathology, along with the tasks are visualized in the following overview.

可以根據深度學習執行的任務或學習范式來研究深度學習的適用性，這是我們在本文中將使用的分類。 在下面的概述中將可視化不同的學習算法，即組織病理學的深度學習，以及任務。

Based on these, a number of DL models have been proposed in the literature that are traditionally based on convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), auto-encoders(AEs) and other variants.

基于這些，在文獻中已經提出了許多基于卷積神經網絡(CNN)，遞歸神經網絡(RNN)，生成對抗網絡(GAN)，自動編碼器(AE)和其他變體的DL模型。 。

監督學習 (Supervised Learning)

Among the supervised learning techniques, we identify three major canonical deep learning models based on the nature of tasks that are solved in digital histopathology: Classification, Regression, and Segmentation.

在有監督的學習技術中，我們根據在數字組織病理學中解決的任務的性質，確定三種主要的規范深度學習模型：分類，回歸和分段。

監督分類 (Supervised Classification)

It can be further subdivided into local and global level classification. Local level classification entails identifying cells or nuclei in patches of the whole slide image. Deep Learning has proven extensively successful in pixel-wise prediction by sliding window approach over image patches that are annotated by pathologists as regions containing objects of interest( cells/nuclei) or background.One of the most prominent works in local classification came in 2019 when Qaiser et al1 in their paper used Persistent Homology Profiles as distinguishing features in order to segment colon tumor regions by classifying patches as tumor regions or normal ones. Persistent Homology profiles are compact mathematical feature representations of a region that are distinctive as well as robust to scale, perturbations in input data, dimension, and coordinates.They used PHP of training dataset in combination with features extracted using CNN and then employed Random Forest regressions on them separately followed by a multi-stage ensemble strategy for the final classification. This hybrid approach proved to be both accurate and highly efficient wrt inference speed.In global level classification, most of the published work focusses on a patch-based classification approach for whole-slide level disease prediction task. It can involve both patch level localization as well as whole slide level classification or grading of disease. The main disadvantage of these methods is the relatively long computational time required to carry out a dense patch-wise prediction over an entire WSI. Different works have approached this problem in different ways, some using heuristic sampling strategies to more recent ones using task-driven visual attention based coarse processing.

它可以進一步細分為本地和全局級別分類。 局部水平分類需要識別整個玻片圖像的斑塊中的細胞或核。 深度學習已證明，通過在圖像切片上滑動窗口方法在病理學家注釋為包含感興趣對象(細胞/細胞核)或背景的區域上進行滑動窗口方法，在像素方向預測方面取得了廣泛的成功.2019年，當局部分類中最杰出的作品之一是Qaiser等 1在他們的論文中使用持久性的同源性概況如由分類補丁腫瘤區域或正常者為了區分功能段結腸腫瘤區域。 持久同源性輪廓是一個區域的緊湊數學特征表示，具有獨特性，規模魯棒性，輸入數據，尺寸和坐標的擾動，將訓練數據集PHP與使用CNN提取的特征結合使用，然后采用隨機森林回歸分別對它們進行分類，然后采用多階段合奏策略進行最終分類。 事實證明，這種混合方法既準確又高效。在全局級別分類中 ，大多數已發表的工作都集中在基于補丁的分類方法上，以進行全滑坡級別的疾病預測任務。 它可能涉及補丁級別定位以及整個幻燈片級別分類或疾病分級。 這些方法的主要缺點是在整個WSI上執行密集的逐塊預測所需的計算時間相對較長。 不同的作品以不同的方式解決了這個問題，有些作品采用啟發式采樣策略 ，而最近的作品采用任務驅動的基于視覺注意力的粗加工。

Xu et al(2019)2Xu et al(2019)2

Xu et al2 in their work adaptively select a sequence of coarse regions from the raw image by a hard visual attention algorithm, and then for each such region, it is able to investigate the abnormal parts based on a soft-attention mechanism. A recurrent network is then built on top to classify the image region and also to predict the location of the image region to be investigated at the next time step. This way, only a fraction of pixels need to be investigated for the classification

Xu et al2在他們的工作中通過硬視覺注意算法自適應地從原始圖像中選擇了一系列粗糙區域，然后針對每個這樣的區域，可以基于軟注意機制來研究異常部位。 然后在頂部建立一個遞歸網絡 ，以對圖像區域進行分類，并預測下一個時間步驟要研究的圖像區域的位置。 這樣，只需要調查一小部分像素即可進行分類

Advantages of using visual attention-based models for Whole Slide Image global classification task are:

對整個幻燈片圖像全局分類任務使用基于視覺注意的模型的優點是：

• The model tries to learn only the most relevant diagnostically useful areas for disease prediction as it enforces a region selection mechanism.
  該模型嘗試執行區域選擇機制，因此僅嘗試學習對疾病預測最相關的診斷有用區域。
• The model complexity is independent of the size of WSI.
  模型的復雜性與WSI的大小無關。

Another recent work for global classification by Halicek, Martin, et al.3 perform patch-based localization and whole slide classification for Squamous Cell Carcinoma(SCC) and Thyroid Cell Carcinoma using CNN using an entirely different approach. A ground-truth binary mask of the cancer area was produced from each outlined histology slide. The WSIs and corresponding ground-truths were down-sampled by a factor of four using nearest-neighbor interpolation. The downsampled slides were then broken into patches of 101 x 101 size. To ensure generalization the number of image patches was augmented by 8x by applying 90-degree rotations and reflections to develop a more robust diagnostic method. Additionally, to establish a level of color-feature invariance and tolerance to differences in H&E staining between slides, the hue, saturation, brightness, and contrast of each patch were randomly manipulated to make a more rigorous training paradigm before being fed to the Inception-v4 model for detecting head and neck cancer.

Halicek，Martin等人[3]進行的全球分類的另一項最新工作是采用完全不同的方法，使用CNN對鱗狀細胞癌(SCC)和甲狀腺細胞癌進行基于補丁的定位和整個玻片分類。 從每個概述的組織學幻燈片中得出了癌區域的真相二元掩模。 使用最近鄰插值對WSI和相應的地面真相進行4倍下采樣。 然后將降采樣后的幻燈片分成101 x 101大小的小塊。 為了確保通用性，通過應用90度旋轉和反射以開發更可靠的診斷方法，圖像補丁的數量增加了8倍 。 此外，為了確定幻燈片之間的顏色特征不變性和對H＆E染色差異的容忍度，在將每個補丁的色相，飽和度，亮度和對比度進行隨機處理以形成更嚴格的訓練范式后，再將其送入Inception-用于檢測頭頸癌的v4模型。

監督回歸 (Supervised Regression)

In this method, we focus on directly regressing the likelihood of pixel being the center of an object for detection or localization of objects. Regression, unlike classification, gives us a continuous value, usually probability score instead of simply a class label as output. Regression helps in better detection by enforcing topological constraints such as assigning higher probability values to pixels near the object center.Regression also helps with challenges faced in cell/nuclei detection arising due to highly irregular appearance and them occurring as overlapping clumps resulting in problems separating them. Deep regression models proposed in the literature are mainly based on either CNN or Fully Convolutional Network(FCN) architectures.

在這種方法中，我們專注于直接回歸以像素為對象中心以檢測或定位對象的可能性。 回歸與分類不同，回歸為我們提供了一個連續的值，通常是概率得分，而不是簡單地將類標簽作為輸出。 回歸通過加強拓撲約束(例如為對象中心附近的像素分配更高的概率值)來幫助更好地進行檢測;回歸還有助于解決由于高度不規則外觀而導致的細胞/核檢測面臨的挑戰，并且它們以重疊團塊的形式出現，從而導致分離它們的問題。 文獻中提出的深度回歸模型主要基于CNN或完全卷積網絡(FCN)架構。

The paper by Graham et al? on HoVer-Net is one of the most seminal works in the entire area of research. It proposes a unified FCN model for simultaneous nuclear instance segmentation and classification. It leverages the instance-rich information encoded within the vertical and horizontal distances of nuclear pixels to their centers of mass. These distances are then utilized to separate clustered nuclei, resulting in an accurate segmentation, particularly in areas with overlapping instances.Then, for each segmented instance, the network predicts the type of nucleus via a devoted up-sampling branch. The network is composed of three parallel branches that are used for three different tasks. We have corresponding ground truth annotations of the data for each of the three branches.

Graham等人在HoVer-Net上發表的論文是整個研究領域中最具開創性的著作之一。 它為核實例的同時分割和分類提出了一個統一的FCN模型。 它利用了在核像素到其質心的垂直和水平距離內編碼的實例豐富的信息。 然后利用這些距離來分離聚集的原子核，從而實現精確的分割，尤其是在實例重疊的區域中。然后，對于每個分割的實例，網絡都會通過專用的向上采樣分支來預測原子核的類型。 該網絡由用于三個不同任務的三個并行分支組成。 對于三個分支中的每個分支，我們都有相應的數據地面真實性注釋。

• The Nuclear Pixel(NP) branch predicts whether or not a pixel belongs to the nuclei or background,

  核像素(NP)分支預測像素是否屬于原子核或背景，

• whereas the Horizontal-Vertical(HoVer) branch predicts the horizontal and vertical distances of nuclear pixels to their centers of mass. Here the colors represent the gradation of the distance of each nuclear pixel from the center of mass.

  而Horizo??ntal-Vertical(HoVer)分支預測核像素到其質心的水平和垂直距離。 在這里，顏色表示每個核像素到質心的距離的漸變。

• Blue represents positive distance up to +1 and means the pixel lies on the left side of the COM in case of the horizontal map and above the COM when it comes to the vertical mapping. Similarly, red represents negative distance up to -1 and means the pixel lies on the Right/bottom side of COM accordingly.
  藍色表示正值，最遠為+1，表示像素在水平圖的情況下位于COM的左側，而在垂直圖的情況下位于COM的上方。 同樣，紅色表示負距離，最大為-1，表示像素相應地位于COM的右/下側。

• Then, the Nuclear Classification(NC) branch(optional) predicts the type of nucleus for each pixel.

  然后， 核分類(NC)分支(可選)預測每個像素的核類型。

Graham et al(2019)?Graham等(2019)?

In particular, the NP and HoVer branches jointly achieve nuclear instance segmentation by first separating nuclear pixels from the background (NP branch) and then separating touching nuclei (HoVer branch). This is the same model that was used for the localization and clustering of tissues step for modeling Whole Slide Images as graphs for subsequent learning using Graph Neural Networks, as discussed in the previous post.

特別是， NP和HoVer分支首先通過將核像素與背景分離(NP分支)，然后分離接觸核( HoVer分支 )，共同實現核實例分割。 這是用于組織的定位和聚類步驟的模型，該步驟用于將整個幻燈片圖像建模為圖形，以供以后使用Graph Neural Networks學習，如先前的文章所述。

In the next and final part of our discussion on Deep Learning in Histopathology, we shall be discussing Supervised Segmentation, Weakly Supervised and Unsupervised Learning methodologies in the context of Digital Histopathology, with appropriate application and relevant literature.

在關于組織病理學深度學習的討論的下一部分和最后一部分中，我們將討論數字組織病理學背景下的監督分割，弱監督和無監督學習方法，并結合適當的應用和相關文獻。

PS: I have linked technically important terms to respective resources explaining them.

PS：我已將技術上重要的術語鏈接到解釋它們的相應資源。

翻譯自: https://towardsdatascience.com/deep-learning-in-histopathology-35c0294d38eb

基于深度學習的病理

總結

以上是生活随笔為你收集整理的基于深度学习的病理_组织病理学的深度学习（第二部分）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。