0 概述

• 論文名稱：“Richer Convolutional Features for Edge Detection”
• 論文鏈接：https://openaccess.thecvf.com/content_cvpr_2017/papers/Liu_Richer_Convolutional_Features_CVPR_2017_paper.pdf
• 縮寫：RCF

這一篇文論在我看來，是CVPR 2015年 HED網(wǎng)絡(luò)（holistically-nested edge detection）的一個改進(jìn)，RCF的論文中也基本上和HED網(wǎng)絡(luò)處處對比。

在上一篇文章中，我們依稀記得HED模型有這樣一個圖：

其中有HED的五個side output的特征圖，下圖是RCF論文中的圖：

我們從這兩個圖的區(qū)別中來認(rèn)識RCF相比HED的改進(jìn)，大家可以看一看圖。

揭曉答案：

• HED是豹子的圖片，但是RCF是兩只小鳥的圖片（手動狗頭）
• HED中的是side output的輸出的特征圖，而RCF中是conv3_1,conv3_2，這意味著RCF似乎把每一個卷積之后的輸出的特征圖都作為了一個side output。

沒錯，HED選取了5個side output，每一個side output都是池化層之前的卷積層輸出的特征圖；而RCF則對每一次卷積的輸出特征圖都作為side output，換句話說 最終的side output中，同一尺寸的輸出可能不止一個。

如果還沒有理解，請看下面章節(jié)，模型結(jié)構(gòu)。

1 模型結(jié)構(gòu)

RCF的backbone是VGG模型：

從圖中可以看到：

• 主干網(wǎng)絡(luò)上分成state1到5，stage1有兩個卷積層，stage2有兩個卷積層，總共有13個卷積層，每一次卷積輸出的圖像，再額外接入一個1x1的卷積，來降低通道數(shù)，所以可以看到，圖中有大量的21通道的卷積層。
• 同一個stage的21通道的特征圖經(jīng)過通道拼接，變成42通道或者是63通道的特征圖，然后再經(jīng)過一個1x1的卷積層，來把通道數(shù)降低成1，再進(jìn)過sigmoid層，輸出的結(jié)果就是一個RCF模型中的side output了

2 損失函數(shù)

這里的損失函數(shù)其實和HED來說類似：

首先整體來看，損失函數(shù)依然使用二值交叉熵

其中

表示 negative的像素值，表示positive的像素值。一般來說輪廓檢測任務(wù)中，positive的樣本應(yīng)該是較少的，因此的值較小，因此損失函數(shù)中第一行，y=0也就是計算非輪廓部分的損失的時候，就會增加一個較小的權(quán)重，來避免類別不均衡的問題。

損失函數(shù)中有兩個常數(shù)，一個是

,這個就是權(quán)重常數(shù)，默認(rèn)為1.1；另外一個是。論文中的描述為：Edge datasets in this community are usually labeled by several annotators using their knowledge about the presences of objects and object parts. Though humans vary in cognition, these human-labeled edges for the same image share high consistency. For each image, we average all the ground truth to generate an edge probability map, which ranges from 0 to 1. Here, 0 means no annotator labeled at this pixel, and 1 means all annotators have labeled at this pixel. We consider the pixels with edge probability higher than η as positive samples and the pixels with edge probability equal to 0 as negative samples. Otherwise, if a pixel is marked by fewer than η of the annotators, this pixel may be semantically controversial to be an edge point. Thus, whether regarding it as positive or negative samples may confuse networks. So we ignore pixels in this category.

大意就是：一般對數(shù)據(jù)集進(jìn)行標(biāo)注，是有多個人來完成的。不同的人雖然有不同的意識，但是他們對于同一個圖片的輪廓標(biāo)注往往是具有一致性。RCF網(wǎng)絡(luò)最后的輸出，是由5個side output融合產(chǎn)生的，因此你這個RCF的輸出也應(yīng)該把大于

的考慮為positive，然后小于的考慮為negative。 其實這一點我自己在復(fù)現(xiàn)的時候并沒有考慮，我看網(wǎng)上的github和官方的代碼中，都沒有考慮這個，都是直接交叉熵。。。我這就也就多此一舉的講解一下論文中的這個的含義

3 pytorch部分代碼

對于這個RCF論文來說，關(guān)鍵就是一個模型的構(gòu)建，另外一個就是損失函數(shù)的構(gòu)建，這里放出這兩部分的代碼，來幫助大家更好的理解上面的內(nèi)容。

3.1 模型部分

下面的代碼在上采樣部分的寫法比較老舊，因為這個網(wǎng)上找來的pytorch版本估計比較老，當(dāng)時還沒有Conv2DTrans這樣的函數(shù)封裝，但是不妨礙大家通過代碼來學(xué)習(xí)RCF。

class RCF(nn.Module):def __init__(self):super(RCF, self).__init__()#lr 1 2 decay 1 0self.conv1_1 = nn.Conv2d(3, 64, 3, padding=1)self.conv1_2 = nn.Conv2d(64, 64, 3, padding=1)self.conv2_1 = nn.Conv2d(64, 128, 3, padding=1)self.conv2_2 = nn.Conv2d(128, 128, 3, padding=1)self.conv3_1 = nn.Conv2d(128, 256, 3, padding=1)self.conv3_2 = nn.Conv2d(256, 256, 3, padding=1)self.conv3_3 = nn.Conv2d(256, 256, 3, padding=1)self.conv4_1 = nn.Conv2d(256, 512, 3, padding=1)self.conv4_2 = nn.Conv2d(512, 512, 3, padding=1)self.conv4_3 = nn.Conv2d(512, 512, 3, padding=1)self.conv5_1 = nn.Conv2d(512, 512, kernel_size=3,stride=1, padding=2, dilation=2)self.conv5_2 = nn.Conv2d(512, 512, kernel_size=3,stride=1, padding=2, dilation=2)self.conv5_3 = nn.Conv2d(512, 512, kernel_size=3,stride=1, padding=2, dilation=2)self.relu = nn.ReLU()self.maxpool = nn.MaxPool2d(2, stride=2, ceil_mode=True)self.maxpool4 = nn.MaxPool2d(2, stride=1, ceil_mode=True)#lr 0.1 0.2 decay 1 0self.conv1_1_down = nn.Conv2d(64, 21, 1, padding=0)self.conv1_2_down = nn.Conv2d(64, 21, 1, padding=0)self.conv2_1_down = nn.Conv2d(128, 21, 1, padding=0)self.conv2_2_down = nn.Conv2d(128, 21, 1, padding=0)self.conv3_1_down = nn.Conv2d(256, 21, 1, padding=0)self.conv3_2_down = nn.Conv2d(256, 21, 1, padding=0)self.conv3_3_down = nn.Conv2d(256, 21, 1, padding=0)self.conv4_1_down = nn.Conv2d(512, 21, 1, padding=0)self.conv4_2_down = nn.Conv2d(512, 21, 1, padding=0)self.conv4_3_down = nn.Conv2d(512, 21, 1, padding=0)self.conv5_1_down = nn.Conv2d(512, 21, 1, padding=0)self.conv5_2_down = nn.Conv2d(512, 21, 1, padding=0)self.conv5_3_down = nn.Conv2d(512, 21, 1, padding=0)#lr 0.01 0.02 decay 1 0self.score_dsn1 = nn.Conv2d(21, 1, 1)self.score_dsn2 = nn.Conv2d(21, 1, 1)self.score_dsn3 = nn.Conv2d(21, 1, 1)self.score_dsn4 = nn.Conv2d(21, 1, 1)self.score_dsn5 = nn.Conv2d(21, 1, 1)#lr 0.001 0.002 decay 1 0self.score_final = nn.Conv2d(5, 1, 1)def forward(self, x):# VGGimg_H, img_W = x.shape[2], x.shape[3]conv1_1 = self.relu(self.conv1_1(x))conv1_2 = self.relu(self.conv1_2(conv1_1))pool1 = self.maxpool(conv1_2)conv2_1 = self.relu(self.conv2_1(pool1))conv2_2 = self.relu(self.conv2_2(conv2_1))pool2 = self.maxpool(conv2_2)conv3_1 = self.relu(self.conv3_1(pool2))conv3_2 = self.relu(self.conv3_2(conv3_1))conv3_3 = self.relu(self.conv3_3(conv3_2))pool3 = self.maxpool(conv3_3)conv4_1 = self.relu(self.conv4_1(pool3))conv4_2 = self.relu(self.conv4_2(conv4_1))conv4_3 = self.relu(self.conv4_3(conv4_2))pool4 = self.maxpool4(conv4_3)conv5_1 = self.relu(self.conv5_1(pool4))conv5_2 = self.relu(self.conv5_2(conv5_1))conv5_3 = self.relu(self.conv5_3(conv5_2))conv1_1_down = self.conv1_1_down(conv1_1)conv1_2_down = self.conv1_2_down(conv1_2)conv2_1_down = self.conv2_1_down(conv2_1)conv2_2_down = self.conv2_2_down(conv2_2)conv3_1_down = self.conv3_1_down(conv3_1)conv3_2_down = self.conv3_2_down(conv3_2)conv3_3_down = self.conv3_3_down(conv3_3)conv4_1_down = self.conv4_1_down(conv4_1)conv4_2_down = self.conv4_2_down(conv4_2)conv4_3_down = self.conv4_3_down(conv4_3)conv5_1_down = self.conv5_1_down(conv5_1)conv5_2_down = self.conv5_2_down(conv5_2)conv5_3_down = self.conv5_3_down(conv5_3)so1_out = self.score_dsn1(conv1_1_down + conv1_2_down)so2_out = self.score_dsn2(conv2_1_down + conv2_2_down)so3_out = self.score_dsn3(conv3_1_down + conv3_2_down + conv3_3_down)so4_out = self.score_dsn4(conv4_1_down + conv4_2_down + conv4_3_down)so5_out = self.score_dsn5(conv5_1_down + conv5_2_down + conv5_3_down)## transpose and crop way weight_deconv2 = make_bilinear_weights(4, 1).cuda()weight_deconv3 = make_bilinear_weights(8, 1).cuda()weight_deconv4 = make_bilinear_weights(16, 1).cuda()weight_deconv5 = make_bilinear_weights(32, 1).cuda()upsample2 = torch.nn.functional.conv_transpose2d(so2_out, weight_deconv2, stride=2)upsample3 = torch.nn.functional.conv_transpose2d(so3_out, weight_deconv3, stride=4)upsample4 = torch.nn.functional.conv_transpose2d(so4_out, weight_deconv4, stride=8)upsample5 = torch.nn.functional.conv_transpose2d(so5_out, weight_deconv5, stride=8)### center cropso1 = crop(so1_out, img_H, img_W)so2 = crop(upsample2, img_H, img_W)so3 = crop(upsample3, img_H, img_W)so4 = crop(upsample4, img_H, img_W)so5 = crop(upsample5, img_H, img_W)fusecat = torch.cat((so1, so2, so3, so4, so5), dim=1)fuse = self.score_final(fusecat)results = [so1, so2, so3, so4, so5, fuse]results = [torch.sigmoid(r) for r in results]return results

3.2 損失函數(shù)部分

def cross_entropy_loss_RCF(prediction, label):label = label.long()mask = label.float()num_positive = torch.sum((mask==1).float()).float()num_negative = torch.sum((mask==0).float()).float()mask[mask == 1] = 1.0 * num_negative / (num_positive + num_negative)mask[mask == 0] = 1.1 * num_positive / (num_positive + num_negative)mask[mask == 2] = 0cost = torch.nn.functional.binary_cross_entropy(prediction.float(),label.float(), weight=mask, reduce=False)return torch.sum(cost)

參考文章：

https://blog.csdn.net/a8039974/article/details/85696282

https://gitee.com/HEART1/RCF-pytorch/blob/master/functions.py

https://openaccess.thecvf.com/content_cvpr_2017/papers/Liu_Richer_Convolutional_Features_CVPR_2017_paper.pdf

總結(jié)

以上是生活随笔為你收集整理的轮廓检测_轮廓检测| Richer Convolutional Features | CVPR | 2017的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。