檢測和語義分割

有關深層學習的FAU講義 (FAU LECTURE NOTES ON DEEP LEARNING)

These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. Try it yourself! If you spot mistakes, please let us know!

這些是FAU YouTube講座“ 深度學習 ”的 講義 。 這是演講視頻和匹配幻燈片的完整記錄。 我們希望您喜歡這些視頻。 當然，此成績單是使用深度學習技術自動創建的，并且僅進行了較小的手動修改。 自己嘗試！ 如果發現錯誤，請告訴我們！

導航 (Navigation)

Previous Lecture / Watch this Video / Top Level / Next Lecture

上一個講座 / 觀看此視頻 / 頂級 / 下一個講座

Instance segmentation can also be used for video editing. Image created using gifify. Source: YouTube實例分割也可以用于視頻編輯。 使用gifify創建的圖像 。 資料來源： YouTube

Welcome back to deep learning! Today, we want to talk about the last part of object detection and segmentation. We want to look into the concept of instance segmentation.

歡迎回到深度學習！ 今天，我們要討論對象檢測和分割的最后一部分。 我們想研究實例分割的概念。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

So, let’s have a look at our slides. You see this is already the last part. Part five and now we want to talk about instance segmentation. We do not just want to detect where pixels with cubes are instead of pixels of cups. We want to really figure out which pixels belong to what cube. This is essentially a combination of object detection and semantic segmentation.

因此，讓我們看一下幻燈片。 您已經看到這已經是最后一部分了。 第五部分，現在我們要討論實例分段。 我們不只是要檢測帶有立方體的像素在哪里，而不是檢測杯子的像素。 我們要真正找出哪些像素屬于哪個立方體。 這實質上是對象檢測和語義分割的組合。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Examples for potential applications are information about occlusion, counting the number of elements belonging to the same class, detecting object boundaries for example of gripping objects in robotics. This is very important and there are examples in the literature for simultaneous detection and segmentation, DeepMask, SharpMask, and Mask RCNN in [10].

潛在應用的示例包括有關遮擋的信息，計算屬于同一類的元素的數量，檢測對象邊界(例如在機器人技術中抓取對象)。 這非常重要，文獻[10]中提供了用于同時檢測和分割的示例，DeepMask，SharpMask和Mask RCNN。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Let’s look at [10] in a little more detail. We essentially go back to the storage. We combine object detection and the segmentation. We use RCNN for object detection. It essentially solves the instance separation. Then, the segmentation refines the bounding boxes per instance.

讓我們更詳細地研究[10]。 我們實質上是回到存儲。 我們結合了對象檢測和分割。 我們使用RCNN進行對象檢測。 它實質上解決了實例分離。 然后，分割會細化每個實例的邊界框。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

The workflow is a two-stage procedure. You have the region proposal that proposes the object bounding boxes. Then, you have the classification using a bounding box regression and the segmentation in parallel. So, you have a multi-task loss that essentially combines the pixel-classification loss of the segmentation, the box loss, and the class loss for producing the right class/bounding box. So, you have these three terms that are then combined in a multi-task loss.

工作流程分為兩個階段。 您具有提議對象邊界框的區域提議。 然后，您可以使用包圍盒回歸和并行分割進行分類。 因此，您有一個多任務損失，該損失實際上將分割的像素分類損失，框損失和產生正確的類/邊界框的類損失結合在一起。 因此，您具有這三個術語，然后將它們組合成多任務損失。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Let’s look in some more detail into the two-stage procedure. You have two different options here for two-stage networks. You can have a joint branch that is working on the ROIs and then splits at a later stage into the segmentation of the mask and the class and bounding box prediction, or you can split early. Then, you run that in two separate networks, In both versions, you have this multi-task loss and that combines the pixel-wise segmentation loss, the box loss, and the class loss.

讓我們更詳細地研究兩階段過程。 對于兩階段網絡，您有兩個不同的選擇。 您可以擁有一個在ROI上工作的聯合分支，然后在稍后階段拆分為蒙版的分割以及類和邊界框的預測，或者您可以盡早拆分。 然后，您在兩個單獨的網絡中運行該文件，在這兩個版本中，您都有這種多任務丟失，并且將逐像素分段丟失，框丟失和類丟失合并在一起。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Let’s have a look at some examples. These are results again from mask RCNN. You can see that to be honest these are quite impressive results. So, there are really difficult cases. You identify where the persons are and you also see that the different persons, of course, are different instances. So, very impressive results!

讓我們看一些例子。 這些也是掩碼RCNN的結果。 您可以看到，說實話，這些都是令人印象深刻的結果。 因此，確實有困難的情況。 您確定人員所在的位置，并且還可以看到不同的人員當然是不同的實例。 因此，非常令人印象深刻的結果！

Mask RCNN is also suited to support autonomous driving. Image created using gifify. Source: YouTube.Mask RCNN也適用于支持自動駕駛。 使用gifify創建的圖像 。 資料來源： YouTube 。

So let’s summarize what we’ve seen so far. The segmentation is commonly solved by architectures analyzing the image and subsequently refining the coarse results. Fully convolutional networks preserve the spatial layout and enable arbitrary input sizes with pooling.

因此，讓我們總結一下到目前為止所看到的。 通常通過架構分析圖像并隨后完善粗略結果來解決分割問題。 完全卷積網絡保留空間布局，并通過池化實現任意輸入大小。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

We can use object detectors and implement them as a sequence of region proposals and classification. Then this leads essentially to the family of RCNN-type of networks. Alternatively, you can go to single-shot detectors. We looked at YOLO which is a very common and very fast technique such as YOLO9000. We looked into RetinaNet if you really have a scale dependency and you want to detect on many different scales like for the example of histological slice processing. So, object detection and segmentation are closely related and combinations are common as you have seen here for the purpose of instance segmentation.

我們可以使用對象檢測器并將其實現為區域建議和分類的序列。 然后，這實質上導致了RCNN型網絡家族。 或者，您可以轉到單次檢測器。 我們介紹了YOLO，這是一種非常常見且非常快速的技術，例如YOLO9000。 如果您確實具有比例依賴性，并且想要在許多不同的尺度上進行檢測，例如組織學切片處理示例，我們就研究了RetinaNet。 因此，對象檢測和分段密切相關，并且組合是常見的，如您在此處為實例分段的目的所見。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

Let’s look at what we still have to talk about in this lecture. Coming up very soon are methods to relieve the burden of labeling. So, we will talk about weak annotation. How we can generate labels? This then also leads to the concept of self-supervision which is a very popular topic right now. It’s been very heavily used in order to generate better networks. The methods are able to reuse also sparsely or even completely unlabeled data. We will look into some of the more advanced methods. One idea that I want to show to you later is the use of known operators. How we can integrate knowledge into networks? Which properties does this have? and we also demonstrate some ideas on how we could potentially make parts of networks reusable. So, there are exciting things still coming up.

讓我們看看在本講座中我們仍然要談論的內容。 很快就會出現減輕標簽負擔的方法。 因此，我們將討論弱注釋。 我們如何生成標簽？ 然后，這也導致了自我監督的概念，這是當前非常流行的話題。 為了生成更好的網絡，它已被大量使用。 這些方法還可以稀疏地使用，甚至可以完全重用未標記的數據。 我們將研究一些更高級的方法。 我想稍后向您展示的一個想法是使用已知的運算符。 我們如何將知識整合到網絡中？ 有哪些屬性？ 我們還將展示一些有關如何使網絡的某些部分可重用的想法。 因此，仍然有令人興奮的事情發生。

CC BY 4.0 from the 深度學習講座中 Deep Learning Lecture.CC BY 4.0下的圖像。

I have some comprehensive questions for you like “What is the difference between semantic and instance segmentation?”, “What is the connection to object detection?”, “How can we construct a network which accepts arbitrary input sizes?”, “What is ROI pooling?”, “How can we perform backpropagation through an ROI pooling layer?”, “What are typical measures for the evaluation of segmentations?”, or for example I could ask you to explain a method for instance segmentation.

我對您有一些綜合性的問題，例如“語義和實例分割之間的區別是什么？”，“與對象檢測的聯系是什么？”，“我們如何構建可以接受任意輸入大小的網絡？”，“什么是ROI池？”，“我們如何通過ROI池層進行反向傳播？”，“評估細分的典型方法是什么？”，或者例如，我可以要求您解釋實例細分的方法。

I have a couple of further readings in terms of links. So, there is this awesome website by Joseph Redmond the creator of Yolo. I think this is a really nice library that is called darknet. You can also study Joseph’s Redmon’s CV I have the link here. I think if you follow this kind of layout, this will definitely jumpstart your career. Please take your time and also look at the references below. we selected really very good state-of-the-art papers and we can definitely recommend having a look at them. So thank you very much for listening to this lecture and I hope you liked our short excursion to the more applied fields like segmentation and object detection. I hope that this turns out to be useful for you and I also hope that we will see again in one of our next videos. So, thank you very much and goodbye!

關于鏈接，我還有一些進一步的讀物。 因此，Yolo的創建者Joseph Redmond提供了一個很棒的網站。 我認為這是一個非常不錯的庫，稱為darknet 。 您也可以研究Joseph的Redmon的簡歷，我在這里具有鏈接 。 我認為，如果您遵循這種布局，那肯定會Swift啟動您的職業生涯。 請花一些時間，并查看下面的參考資料。 我們選擇了非常好的最新技術論文，我們絕對可以建議您看一看。 因此，非常感謝您收聽本講座，希望您喜歡我們對更廣泛應用的領域(如分段和對象檢測)的短暫訪問。 我希望這對您有用，也希望我們在下一個視頻中再次看到。 因此，非常感謝，再見！

If you liked this post, you can find more essays here, more educational material on Machine Learning here, or have a look at our Deep LearningLecture. I would also appreciate a follow on YouTube, Twitter, Facebook, or LinkedIn in case you want to be informed about more essays, videos, and research in the future. This article is released under the Creative Commons 4.0 Attribution License and can be reprinted and modified if referenced. If you are interested in generating transcripts from video lectures try AutoBlog.

如果你喜歡這篇文章，你可以找到這里更多的文章 ，更多的教育材料，機器學習在這里 ，或看看我們的深入 學習 講座 。 如果您希望將來了解更多文章，視頻和研究信息，也歡迎關注YouTube ， Twitter ， Facebook或LinkedIn 。 本文是根據知識共享4.0署名許可發布的 ，如果引用，可以重新打印和修改。 如果您對從視頻講座中生成成績單感興趣，請嘗試使用AutoBlog 。

翻譯自: https://towardsdatascience.com/segmentation-and-object-detection-part-5-4c6f70d25d31

檢測和語義分割

總結

以上是生活随笔為你收集整理的检测和语义分割_分割和对象检测-第5部分的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。