目錄

前言：

SqueezeNet模型原理

Fire Module

SqueezeNet模型結構

SqueezeNet模型特點

SqueezeNet網絡實現(xiàn)

---

前言：

CNN網絡模型一直在追求識別成功率，從AlexNet到VGGNet模型，識別精度不斷提高，TOP-5錯誤率從15.3%下降到7.3%，但參數(shù)數(shù)量也越來越多，但參數(shù)數(shù)量也越來越多，從60M增加到140M。過多的參數(shù)降低了分布式訓練的效率，也給數(shù)據傳輸所需的網絡帶寬造成很大的負擔。如何在保證識別精度的情況下，對網絡參數(shù)進行壓縮是需要進一步研究的方向。

本篇介紹的SqueezeNet模型（壓縮模型）就是為了解決這一問題而提出的網路模型。本篇先闡述SqueezeNet模型的基本理論，然后解讀Caffe的實現(xiàn)，最后介紹Caffe環(huán)境下的訓練方法。

SqueezeNet模型原理

SqueezeNet設計目標不是為了得到最佳的CNN識別精度，而是希望在簡化網絡復雜度的同時保證網絡模型的識別精度。

SqeezeNet模型的設計用以下三個方法簡化網絡復雜度：

（1）替換3x3的卷積核為1x1的卷積核。從AlexNet模型發(fā)展到現(xiàn)在，因為設計上的簡潔和有效性，卷積核的大小都選擇3x3.SqueezeNet模型用1x1的卷積核替換3x3的卷積核可以讓網絡參數(shù)縮小9倍。但是為了不影響識別精度，只做了部分替換。

（2）減少輸入3x3卷積的輸入特征數(shù)量。將卷積層分解為squeeze層以及expand層，并封裝為一個Fire Module。

（3）在整個網絡后期進行下采樣，使得卷積層有較大的activation maps。

Fire Module

Fire Module是SqueezeNet的核心構件，其思想非常簡單。即將一個卷積層分解為一個Squeeze層和一個expand層，并各自帶上Relu激活曾。squeeze層包含全部都是1x1的卷積核，共有S個。expand層包含1x1核3x3的卷積核，其中1x1的卷積核有E1個，3x3的卷積核有E3個，要求滿足S<(E1+E3)。如下圖FireModule結構：

? ? ? ? ? ? ??

SqueezeNet模型結構

SqueezeNet的網絡模型結構如圖：

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

SqueezeNet模型共有九層Fire Module，中間穿插了三個MAX pooling層，最后一層用Average Pooling層替換全連接層是的參數(shù)大量減少。SqueezeNet網絡模型在最上層核下層各保留了一個卷積層，這樣做的目的是保證輸入輸出的大小可掌握。其他參數(shù)細節(jié)設置在下面caffe實現(xiàn)中會詳細介紹。下表給出了每層的維度：

? ? ? ? ? ? ?

SqueezeNet模型特點

（1）SqueezeNet比AlexNet的參數(shù)減少的50倍，模型大小只有4.8M，在性能好的FPGA上可以運行起來，并且能帶來與AlexNet相當?shù)淖R別精度。

（2）SqueezeNet證明了小的神經網絡也能達到很好的識別精度，這使得未來將嵌入式設備或移動設備植入神經網絡成為一種可能。

SqueezeNet網絡實現(xiàn)

# please cite: # @article{SqueezeNet, # Author = {Forrest N. Iandola and Matthew W. Moskewicz and Khalid Ashraf and Song Han and William J. Dally and Kurt Keutzer}, # Title = {SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and $<$1MB model size}, # Journal = {arXiv:1602.07360}, # Year = {2016} # } layer {name: "data" //data層type: "Data"top: "data"top: "label"include {phase: TRAIN}transform_param {crop_size: 227mean_value: 104mean_value: 117mean_value: 123}data_param {source: "examples/imagenet/ilsvrc12_train_lmdb" //訓練數(shù)據集batch_size: 32backend: LMDB} } layer {name: "data"type: "Data"top: "data"top: "label"include {phase: TEST}transform_param {crop_size: 227mean_value: 104mean_value: 117mean_value: 123}data_param {source: "examples/imagenet/ilsvrc12_val_lmdb"batch_size: 25 #not *iter_sizebackend: LMDB} } layer { //第一個卷積層，縮小輸入圖像，提取96維特征name: "conv1"type: "Convolution"bottom: "data"top: "conv1"convolution_param {num_output: 64kernel_size: 3stride: 2weight_filler {type: "xavier"}} } layer { //RELU層name: "relu_conv1"type: "ReLU"bottom: "conv1"top: "conv1" } layer {name: "pool1" //第一個Max Pooling層，降采樣，縮小一半type: "Pooling"bottom: "conv1"top: "pool1"pooling_param {pool: MAXkernel_size: 3stride: 2} } layer { //第一個fire模塊，模塊內先用squeeze層減少通道數(shù)，再用expand層增加通道數(shù)name: "fire2/squeeze1x1"type: "Convolution"bottom: "pool1"top: "fire2/squeeze1x1"convolution_param {num_output: 16kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire2/relu_squeeze1x1"type: "ReLU"bottom: "fire2/squeeze1x1"top: "fire2/squeeze1x1" } layer {name: "fire2/expand1x1"type: "Convolution"bottom: "fire2/squeeze1x1"top: "fire2/expand1x1"convolution_param {num_output: 64kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire2/relu_expand1x1"type: "ReLU"bottom: "fire2/expand1x1"top: "fire2/expand1x1" } layer {name: "fire2/expand3x3"type: "Convolution"bottom: "fire2/squeeze1x1"top: "fire2/expand3x3"convolution_param {num_output: 64pad: 1 //增加一個像素邊界，是的1x1和3x3filter對齊kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire2/relu_expand3x3"type: "ReLU"bottom: "fire2/expand3x3"top: "fire2/expand3x3" } layer {name: "fire2/concat"type: "Concat"bottom: "fire2/expand1x1"bottom: "fire2/expand3x3"top: "fire2/concat" } layer {name: "fire3/squeeze1x1"type: "Convolution"bottom: "fire2/concat"top: "fire3/squeeze1x1"convolution_param {num_output: 16kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire3/relu_squeeze1x1"type: "ReLU"bottom: "fire3/squeeze1x1"top: "fire3/squeeze1x1" } layer {name: "fire3/expand1x1"type: "Convolution"bottom: "fire3/squeeze1x1"top: "fire3/expand1x1"convolution_param {num_output: 64kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire3/relu_expand1x1"type: "ReLU"bottom: "fire3/expand1x1"top: "fire3/expand1x1" } layer {name: "fire3/expand3x3"type: "Convolution"bottom: "fire3/squeeze1x1"top: "fire3/expand3x3"convolution_param {num_output: 64pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire3/relu_expand3x3"type: "ReLU"bottom: "fire3/expand3x3"top: "fire3/expand3x3" } layer {name: "fire3/concat"type: "Concat"bottom: "fire3/expand1x1"bottom: "fire3/expand3x3"top: "fire3/concat" } layer {name: "pool3"type: "Pooling"bottom: "fire3/concat"top: "pool3"pooling_param {pool: MAXkernel_size: 3stride: 2} } layer {name: "fire4/squeeze1x1"type: "Convolution"bottom: "pool3"top: "fire4/squeeze1x1"convolution_param {num_output: 32kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire4/relu_squeeze1x1"type: "ReLU"bottom: "fire4/squeeze1x1"top: "fire4/squeeze1x1" } layer {name: "fire4/expand1x1"type: "Convolution"bottom: "fire4/squeeze1x1"top: "fire4/expand1x1"convolution_param {num_output: 128kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire4/relu_expand1x1"type: "ReLU"bottom: "fire4/expand1x1"top: "fire4/expand1x1" } layer {name: "fire4/expand3x3"type: "Convolution"bottom: "fire4/squeeze1x1"top: "fire4/expand3x3"convolution_param {num_output: 128pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire4/relu_expand3x3"type: "ReLU"bottom: "fire4/expand3x3"top: "fire4/expand3x3" } layer {name: "fire4/concat"type: "Concat"bottom: "fire4/expand1x1"bottom: "fire4/expand3x3"top: "fire4/concat" } layer {name: "fire5/squeeze1x1"type: "Convolution"bottom: "fire4/concat"top: "fire5/squeeze1x1"convolution_param {num_output: 32kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire5/relu_squeeze1x1"type: "ReLU"bottom: "fire5/squeeze1x1"top: "fire5/squeeze1x1" } layer {name: "fire5/expand1x1"type: "Convolution"bottom: "fire5/squeeze1x1"top: "fire5/expand1x1"convolution_param {num_output: 128kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire5/relu_expand1x1"type: "ReLU"bottom: "fire5/expand1x1"top: "fire5/expand1x1" } layer {name: "fire5/expand3x3"type: "Convolution"bottom: "fire5/squeeze1x1"top: "fire5/expand3x3"convolution_param {num_output: 128pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire5/relu_expand3x3"type: "ReLU"bottom: "fire5/expand3x3"top: "fire5/expand3x3" } layer {name: "fire5/concat"type: "Concat"bottom: "fire5/expand1x1"bottom: "fire5/expand3x3"top: "fire5/concat" } layer {name: "pool5"type: "Pooling"bottom: "fire5/concat"top: "pool5"pooling_param {pool: MAXkernel_size: 3stride: 2} } layer {name: "fire6/squeeze1x1"type: "Convolution"bottom: "pool5"top: "fire6/squeeze1x1"convolution_param {num_output: 48kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire6/relu_squeeze1x1"type: "ReLU"bottom: "fire6/squeeze1x1"top: "fire6/squeeze1x1" } layer {name: "fire6/expand1x1"type: "Convolution"bottom: "fire6/squeeze1x1"top: "fire6/expand1x1"convolution_param {num_output: 192kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire6/relu_expand1x1"type: "ReLU"bottom: "fire6/expand1x1"top: "fire6/expand1x1" } layer {name: "fire6/expand3x3"type: "Convolution"bottom: "fire6/squeeze1x1"top: "fire6/expand3x3"convolution_param {num_output: 192pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire6/relu_expand3x3"type: "ReLU"bottom: "fire6/expand3x3"top: "fire6/expand3x3" } layer {name: "fire6/concat"type: "Concat"bottom: "fire6/expand1x1"bottom: "fire6/expand3x3"top: "fire6/concat" } layer {name: "fire7/squeeze1x1"type: "Convolution"bottom: "fire6/concat"top: "fire7/squeeze1x1"convolution_param {num_output: 48kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire7/relu_squeeze1x1"type: "ReLU"bottom: "fire7/squeeze1x1"top: "fire7/squeeze1x1" } layer {name: "fire7/expand1x1"type: "Convolution"bottom: "fire7/squeeze1x1"top: "fire7/expand1x1"convolution_param {num_output: 192kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire7/relu_expand1x1"type: "ReLU"bottom: "fire7/expand1x1"top: "fire7/expand1x1" } layer {name: "fire7/expand3x3"type: "Convolution"bottom: "fire7/squeeze1x1"top: "fire7/expand3x3"convolution_param {num_output: 192pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire7/relu_expand3x3"type: "ReLU"bottom: "fire7/expand3x3"top: "fire7/expand3x3" } layer {name: "fire7/concat"type: "Concat"bottom: "fire7/expand1x1"bottom: "fire7/expand3x3"top: "fire7/concat" } layer {name: "fire8/squeeze1x1"type: "Convolution"bottom: "fire7/concat"top: "fire8/squeeze1x1"convolution_param {num_output: 64kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire8/relu_squeeze1x1"type: "ReLU"bottom: "fire8/squeeze1x1"top: "fire8/squeeze1x1" } layer {name: "fire8/expand1x1"type: "Convolution"bottom: "fire8/squeeze1x1"top: "fire8/expand1x1"convolution_param {num_output: 256kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire8/relu_expand1x1"type: "ReLU"bottom: "fire8/expand1x1"top: "fire8/expand1x1" } layer {name: "fire8/expand3x3"type: "Convolution"bottom: "fire8/squeeze1x1"top: "fire8/expand3x3"convolution_param {num_output: 256pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire8/relu_expand3x3"type: "ReLU"bottom: "fire8/expand3x3"top: "fire8/expand3x3" } layer {name: "fire8/concat"type: "Concat"bottom: "fire8/expand1x1"bottom: "fire8/expand3x3"top: "fire8/concat" } layer {name: "fire9/squeeze1x1"type: "Convolution"bottom: "fire8/concat"top: "fire9/squeeze1x1"convolution_param {num_output: 64kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire9/relu_squeeze1x1"type: "ReLU"bottom: "fire9/squeeze1x1"top: "fire9/squeeze1x1" } layer {name: "fire9/expand1x1"type: "Convolution"bottom: "fire9/squeeze1x1"top: "fire9/expand1x1"convolution_param {num_output: 256kernel_size: 1weight_filler {type: "xavier"}} } layer {name: "fire9/relu_expand1x1"type: "ReLU"bottom: "fire9/expand1x1"top: "fire9/expand1x1" } layer {name: "fire9/expand3x3"type: "Convolution"bottom: "fire9/squeeze1x1"top: "fire9/expand3x3"convolution_param {num_output: 256pad: 1kernel_size: 3weight_filler {type: "xavier"}} } layer {name: "fire9/relu_expand3x3"type: "ReLU"bottom: "fire9/expand3x3"top: "fire9/expand3x3" } layer {name: "fire9/concat"type: "Concat"bottom: "fire9/expand1x1"bottom: "fire9/expand3x3"top: "fire9/concat" } layer {name: "drop9" //最后一個fire模塊后，增加一個Dropout層type: "Dropout"bottom: "fire9/concat"top: "fire9/concat"dropout_param {dropout_ratio: 0.5 //丟棄率為50%} } layer { //第二個卷積層，為圖的每個像素預測1000個分類name: "conv10"type: "Convolution"bottom: "fire9/concat"top: "conv10"convolution_param {num_output: 1000kernel_size: 1weight_filler {type: "gaussian"mean: 0.0std: 0.01}} } layer {name: "relu_conv10"type: "ReLU"bottom: "conv10"top: "conv10" } layer { //average pooling層得到1000類name: "pool10"type: "Pooling"bottom: "conv10"top: "pool10"pooling_param {pool: AVEglobal_pooling: true} } layer { //softmax層，使用softmax函數(shù)歸一化為概率name: "loss"type: "SoftmaxWithLoss"bottom: "pool10"bottom: "label"top: "loss"#include {# phase: TRAIN#} } layer {name: "accuracy"type: "Accuracy"bottom: "pool10"bottom: "label"top: "accuracy"#include {# phase: TEST#} } layer {name: "accuracy_top5"type: "Accuracy"bottom: "pool10"bottom: "label"top: "accuracy_top5"#include {# phase: TEST#}accuracy_param {top_k: 5} }

?

總結

以上是生活随笔為你收集整理的SqueezeNet网络模型详解的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。