多目標分類的混淆矩陣

After training a machine learning classifier, the next step is to evaluate its performance using relevant metric(s). The confusion matrix is one of the evaluation metrics.

訓練完機器學習分類器后，下一步就是使用相關指標評估其性能。 混淆矩陣是評估指標之一。

A confusion matrix is a table showing the performance of a classifier given some truth values/instances (supervised learning kind of).

混淆矩陣是一個表，顯示在給定一些真值/實例(監督學習的情況)的情況下分類器的性能。

But calculating of confusion matrix for object detection and instance segmentation tasks is less intuitive. First, it is necessary to understand another supporting metric: Intersection over Union (IoU). A key role in calculating metrics for object detection and instance segmentation tasks is played by Intersection over Union (IoU).

但是，用于對象檢測和實例分割任務的混淆矩陣的計算不太直觀。 首先，有必要了解另一種支持指標：聯盟交叉口(IoU)。 Intersection over Union(IoU)在計算對象檢測和實例分割任務的度量標準中扮演著關鍵角色。

聯合路口(IoU) (Intersection over Union (IoU))

IoU, also called Jaccard index, is a metric that evaluates the overlap between the ground-truth mask (gt) and the predicted mask (pd). In object detection, we can use IoU to determine if a given detection is valid or not.

IoU (也稱為Jaccard索引 )是一種度量，用于評估地面真假蒙版( gt )與預測蒙版( pd )之間的重疊。 在對象檢測中，我們可以使用IoU來確定給定的檢測是否有效。

IoU is calculated as the area of overlap/intersection between gt and pd divided by the area of the union between the two, that is,

將IoU計算為gt和pd之間的重疊/交叉區域除以兩者之間的并集區域，即

Diagrammatically, IoU is defined as shown below:

IoU的示意圖如下所示：

Fig 1 (Source: Author)圖1(來源：作者)

Note: IoU metric ranges from 0 and 1 with 0 signifying no overlap and 1 implying a perfect overlap between gt and pd.

注意： IoU度量值的范圍是0和1，其中0表示沒有重疊，而1表示gt和pd之間存在完美的重疊。

A confusion matrix is made up of 4 components, namely, True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN). To define all the components, we need to define some threshold (say α) based on IoU.

混淆矩陣由4個分量組成，即正正(TP)，正負(TN)，誤正(FP)和誤負(FN) 。 要定義所有組件，我們需要基于IoU定義一些閾值(例如α) 。

Fig 2 : Confusion matrix over 2 classes: 1 and 2 (Source: Author)圖2：2個類別(1和2)上的混淆矩陣(來源：作者)

• True Positive (TP) — This is an instance in which the classifier predicted positive when the truth is indeed positive, that is, a detection for which IoU ≥ α.
  真實正值(TP)-在這種情況下，分類器在事實確實為正值時預測為正值，也就是說，檢測到IoU≥α。
• False Positive (FP) — This is a wrong positive detection, that is, a detection for which IoU < α.
  假陽性(FP)-這是一個錯誤的陽性檢測，即IoU <α的檢測。
• False Negative (FN) — This is an actual instance that is not detected by the classifier.
  假陰性(FN)-這是分類器未檢測到的實際實例。
• True Negative (TN) — This metric implies a negative detection given that the actual instance is also negative. In object detection, this metric does not apply because there exist many possible predictions that should not be detected in an image. Thus, TN includes all possible wrong detection that were not detected.
  真負數(TN)-真實實例也為負數時，此度量標準表示負數檢測。 在對象檢測中，此度量標準不適用，因為存在許多不應在圖像中檢測到的可能預測。 因此，TN包括未檢測到的所有可能的錯誤檢測。

These concepts can intuitively be understood with some diagrammatic examples (let's consider the IOU threshold, α = 0.5)

這些概念可以通過一些示例直觀地理解(讓我們考慮IOU閾值，α= 0.5)

Fig 3: LEFT: False Negative (FN) , RIGHT: True Positive (TP), Credits: Photo by Daniel Jensen on Unsplash無花果3：左：假陰性(FN)，右：真陽性(TP)，鳴謝： 丹尼爾·詹森 ( Daniel Jensen )攝影： Unsplash Daniel Jensen on Daniel Jensen攝影： UnsplashUnsplash

Remark : By the definition of IoU threshold, Fig 3 RIGHT turns to be FP if we choose threshold above 0.86 and Fig 4 RIGHT becomes a TP if we choose IoU threshold below 0.14

備注：根據IoU閾值的定義，如果選擇高于0.86的閾值，則圖3 RIGHT變為FP，如果選擇低于0.14的IoU閾值，圖4 RIGHT變為TP

Other metrics that can be derived from confusion matrix includes:

可以從混淆矩陣得出的其他指標包括：

• Precision is the ability of a classifier to identify only relevant objects. It is the proportion of correct positive predictions and is given by

  精度是分類器僅識別相關對象的能力。 它是正確肯定預測的比例，由下式給出

• Recall is a metric which measures the ability of a classifier to find all the relevant cases (that is, all the ground-truths). It is the proportion of true positive detected among all ground-truths and is defined as

  召回率是衡量分類器查找所有相關案例(即所有真實情況)的能力的度量。 它是在所有地面真相中檢測到的真實正數的比例，定義為

F? score is harmonic mean of precision and recall.

F?得分是精度和查全率的調和平均值。

例 (Example)

Consider the following image with the ground truths (dark blue) and classifier detections (red). Through observation can you be able to tell the number of TP, FP and FN?

考慮下面的圖像，其中包含地面實況(深藍色)和分類器檢測(紅色)。 通過觀察，您可以分辨出TP，FP和FN的數量嗎？

Fig 5 (Source : Fuji-SfM dataset (cited in the reference section))圖5(來源：Fuji-SfM數據集(在參考部分中引用))

Python實現 (Python Implementation)

In Python, a confusion matrix can be calculated using Shapely library. The following function ( evaluation(ground,pred,iou_value) → 6-value tuple for TP, FP, FN, Precision, Recall, F?) can be used to determine confusion matrix for above image (Fig 5)

在Python中，可以使用Shapely庫計算混淆矩陣。 可以使用以下函數( evaluation(ground,pred,iou_value)值evaluation(ground,pred,iou_value) → TP，FP，FN，Precision，Recall，F?的 6值元組)來確定上述圖像的混淆矩陣(圖5)

Parameters:

參數：

• ground — is n × m × 2 array where n is number of the ground truth instances for the given image, m is the number of (x,y) pairs sampled on the circumference of the mask.

  ground —是n × m × 2的數組，其中n是給定圖像的地面真實實例的數量， m是在蒙版圓周上采樣的(x，y)對的數量。

• pred is p × q × 2 array where p is the number of detections, and q is the number of (x,y) points sampled for the prediction mask

  pred是p×q×2數組，其中p是檢測次數， q是為預測掩碼采樣的(x，y)點數

• iou_value is the IoU threshold

  iou_value是IoU閾值

For Fig 5 and IoU threshold, α = 0.5, evaluation(ground,pred,iou_value) →

對于圖5和IoU閾值，α= 0.5， evaluation(ground,pred,iou_value) →

TP: 9 FP: 5 FN: 0 GT: 10
Precall: 0.643 Recall: 1.0 F1 score: 0.783

Thank you for reading :-)

感謝您的閱讀:-)

Jordi Gene-Mola, Ricardo Sanz-Cortiella, Joan R. Rosell-Polo, Josep-Ramon Morros, Javier Ruiz-Hidalgo, Verónica Vilaplana, & Eduard Gregorio. (2020). Fuji-SfM dataset [Data set]. Zenodo. http://doi.org/10.528/zenodo.3712808

Jordi Gene-Mola，Ricardo Sanz-Cortiella，Joan R.Rosell-Polo，Josep-Ramon Morros，Javier Ruiz-Hidalgo，VerónicaVilaplana和Eduard Gregorio。 (2020)。 Fuji-SfM數據集[數據集]。 Zenodo。 http://doi.org/10.528/zenodo.3712808

Everingham, M., Eslami, S. A., Van Gool, L., Williams, C. K., Winn, J., and Zisserman, A. The pascal visual object classes challenge: A retrospective. International journal of computer vision, 111(1):98–136, 2015.

馬薩諸塞州埃弗林漢姆(Everingham)，路易斯安那州埃斯拉米(Eslami)，路易斯安那州范古爾(Van Gool)，威廉姆斯(CK)，溫恩(J.Winn)和澤西曼(A.Zisserman)。 國際計算機視覺雜志，111(1)：98-136，2015年。

Fawcett, T. An introduction to ROC analysis. Pattern recognition letters, 27(8):861–874, 2006.

Fawcett，T。ROC分析簡介。 模式識別字母，27(8)：861–874，2006年。

翻譯自: https://towardsdatascience.com/confusion-matrix-and-object-detection-f0cbcb634157

多目標分類的混淆矩陣

總結

以上是生活随笔為你收集整理的多目标分类的混淆矩阵_用于目标检测的混淆矩阵的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。