論文：Spatial As Deep: Spatial CNN for Traffic Scene Understanding

代碼：https://github.com/XingangPan/SCNN

參考：

數據集：CULane

Overview

一般的 CNNs 通常是通過堆疊卷積層來實現的。然而這種方式不能有效的獲取各行各列像素檢的空間關系。而這種空間關系對于類似車道線檢測這種具有“形狀先驗強但是外觀一致性若”的語義目標來說是很重要的。

在這篇論文中，作者提出了一種名為 SCNN(Spatial CNN) 的網絡結構，它將常規的 layer-by-layer convolutions 換成了 to slice-byslice convolutions，這使得信息可以跨行和跨列傳遞。這種 SCNN 特別適用于連續形狀結構或者大型的目標，這類目標空間關系強但外觀線索少。

作者在車道線檢測數據集上和實例分割數據集上進行了實驗。實驗結果表明了，SCNN 可以顯著提高系統的性能。本文同時獲得了 TuSimple Benchmark Lane Detection Challenge 的第一名，with an accuracy of 96.53%.

如圖中，車道線是一種 long continuous shape and might be occluded 的目標，人類可以很容易通過上下文信息來填充被遮擋部分，但一般的 CNNs 不行。

In a layer-by-layer CNN, a convolution layer receives input from the former layer, applies convolution operation and nonlinear activation, and sends result to the next layer. This process is done sequentially. Similarly, SCNN views rows or columns of feature maps as layers and applies convolution, nonlinear activation, and sum operations sequentially, which forms a deep neural network. In this way information could be propagated between neurons in the same layer.

CULane

作者還提供了 CULane 數據集，在這之前的一些數據集，例如 Tusimple（6408 imgs)，場景簡單且對于磨損的車道線沒有進行標注，而這種車道線人肉眼是很容易被推斷補齊的。

To collect data, we mounted cameras on six different vehicles driven by different drivers and recorded videos during driving in Beijing on different days. More than 55 hours of videos were collected and 133,235 frames were extracted, which is more than 20 times of TuSimple Dataset. We have divided the dataset into 88880 for training set, 9675 for validation set, and 34680 for test set. These images were undistorted using tools in (Scaramuzza, Martinelli, and Siegwart 2006) and have a resolution of 1640 x 590

CULane 數據集對于遮擋和磨損的部分進行了估計，如上圖2、4；為了讓算法能夠識別出柵欄，只對柵欄一側進行標注，如上圖1。CULane 只標注了最需要關注的 4 條車道線。

SCNN

以 SCNN_D 為例，假定經過一系列卷積操作后， feature map size 為 $C$ x $H$ x $W$, 從 $H$ 維度出發，將feature map 切成 $H$ slices，然后第一個 slice 經過一個 $1$ x $w$ x $C$ 的卷積、非線性操作后加到下一個 slice 上（一般這個 $w$ 是個較大尺度的卷積核，作者實驗證明 $w=9$ 結果最理想）。這個操作繼續往下進行，直到最后一個 slice 也被更新了。圖中的標記符 D, U, R, L 分別表示 SCNN that is downward, upward, rightward, and leftward 四個方向進行上述操作。

作者提到，同常規卷積相比 SCNN 有三個優點：

Computational efficiency. 這是相比于 MRF/CRF來說的。

Message as residual.
Flexibility. Usually, the top hidden layer contains information that is both rich and of high semantics,thus is an ideal place to apply SCNN

訓練測試相關

In both tasks, we train the models using standard SGD with batch size 12, base learning rate 0.01, momentum 0.9, and weight decay 0.0001. The learning rate policy is ”poly” with power and iteration number set to 0.9 and 60K respectively.

As shown in Fig.5 (b), for each lane marking whose existence value is larger than 0.5, we search the corresponding probmap every 20 rows for the position with the highest response. These positions are then connected by cubic splines, which are the final predictions.

During training, the line width of the targets is set to 16 pixels, and the input and target images are rescaled to 800x288. Considering the imbalanced label between background and lane markings, the loss of background is multiplied by 0.4.

評價指標

作者利用 intersection over union (IoU) 和閾值（0.3/0.5）來判斷一條車道線是否被檢測出來。

egin{equation}
label{a}
egin{split}
&Precision = frac{TP}{TP+FP} \
& Recall= frac{TP}{TP+FN}\
&F-measure = (1+eta^2)frac{Precision*Recall}{eta^2*Precision + Recall}\
&F-measure = frac{2} {frac{1}{Precision} + frac{1}{Recall}}, if eta =1
end{split}
end{equation}

Ablation Study

作者從 6 個角度證明了 SCNN 有多牛逼：

Effectiveness of multidirectional SCNN：同簡單添加常規卷積層相比 SCNN 更牛逼


Effects of kernel width w：w 大小等于9 時最理想

Spatial CNN on different positions：SCNN 結構作用在最后一個隱含層上比作用在輸出層上更理想

Effectiveness of sequential propagation：順序執行（一個切片更新后再去更新下一個切片）比平行更新效果好很多，這說明 a pixel does not merely affected by nearby pixels, but do receive information from further positions.

Comparison with state-of-the-art methods: 這里的 baseline 就是上面網絡(a) DeepLab.

Computational efficiency over other methods：這個，看下面表情。。。

作者還從 Semantic Segmentation on Cityscapes 角度做了實驗，這里就不累述了

SCNN for Tusimple

。。。

總結

以上是生活随笔為你收集整理的深度学习笔记（十四）车道线检测 SCNN的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。