前言

最近打算重新跟著官方教程學習一下caffe，順便也自己翻譯了一下官方的文檔。自己也做了一些標注，都用斜體標記出來了。中間可能額外還加了自己遇到的問題或是運行結果之類的。歡迎交流指正，拒絕噴子！
官方教程的原文鏈接：http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/02-fine-tuning.ipynb

Fine-tuning a Pretrained Network for Style Recognition

在這個例子中，我們會一起探索一種在現實世界的應用中比較常用的方法：使用一個預訓練模型，并使用用戶自定義的數據集來微調網絡的參數。
這個方法的優點就是，既然預訓練的網絡已經事先使用很大的數據集訓練好了，那么網絡的中間層可以獲取大多數視覺上的語義（“semantics”）信息。有關這里說到的“語義”（“semantics”）這個詞，我們把它當一個黑盒子來看，就將它想象成一個非常強大且通用的視覺特征就行。更重要的是，我們只需要一個相對較小的數據集就足夠在目標任務上取得不錯的結果。
首先，我們需要準備好數據集。包括以下幾個步驟：（1）通過提供的shell腳本來獲取ImageNet ilsvrc的預訓練模型。（2）從整個Flickr style數據集中下載一個子數據集。（3）編譯下載好的Flickr數據集成Caffe可以使用的格式。

# 指定caffe路徑 caffe_root = '/home/xhb/caffe/caffe/' # this file should be run from {caffe_root}/examples (otherwise change this line)# 導入caffe import sys sys.path.insert(0, caffe_root + 'python') import caffe# 備注：我是在筆記本上用CPU跑的，所以改成了CPU模式 # caffe.set_device(0) # caffe.set_mode_gpu() caffe.set_mode_cpu()import numpy as np from pylab import * %matplotlib inline import tempfile# 對圖像格式等做一下處理 # Helper function for deprocessing preprocessed images, e.g., for display. def deprocess_net_image(image):image = image.copy() # don't modify destructivelyimage = image[::-1] # BGR -> RGBimage = image.transpose(1, 2, 0) # CHW -> HWCimage += [123, 117, 104] # (approximately) undo mean subtraction# clamp values in [0, 255]image[image < 0], image[image > 255] = 0, 255# round and cast from float32 to uint8image = np.round(image)image = np.require(image, dtype=np.uint8)return image

1.準備數據集

下載此例子需要的數據集。

• get_ilsvrc_aux.sh：下載ImageNet數據集的均值文件，標簽文件等。
• download_model_binary.py：下載預訓練好的模型。
• finetune_flickr_style/assemble_data.py：下載用于圖像風格檢測的訓練和測試的數據集，后面就簡稱style數據集。

我們在下面的練習中，會從整個數據集中下載一個較小的子數據集：8萬張圖片中只下載2000張，從20個風格類別中只下載5種類別。（如果想要下載完整數據集，修改下面對應的代碼成full_dataset=True即可。）

# Download just a small subset of the data for this exercise. # (2000 of 80K images, 5 of 20 labels.) # To download the entire dataset, set `full_dataset = True`. full_dataset = False if full_dataset:NUM_STYLE_IMAGES = NUM_STYLE_LABELS = -1 else:NUM_STYLE_IMAGES = 2000NUM_STYLE_LABELS = 5# This downloads the ilsvrc auxiliary data (mean file, etc), # and a subset of 2000 images for the style recognition task. # 備注：下面這段代碼我運行時將其注釋了，因為我事先在命令行下運行過腳本，事先下載好了要用到的幾個文件 ''' import os os.chdir(caffe_root) # run scripts from caffe root !data/ilsvrc12/get_ilsvrc_aux.sh !scripts/download_model_binary.py models/bvlc_reference_caffenet !python examples/finetune_flickr_style/assemble_data.py \--workers=-1 --seed=1701 \--images=$NUM_STYLE_IMAGES --label=$NUM_STYLE_LABELS # back to examples os.chdir('examples') ''' "\nimport os\nos.chdir(caffe_root) # run scripts from caffe root\n!data/ilsvrc12/get_ilsvrc_aux.sh\n!scripts/download_model_binary.py models/bvlc_reference_caffenet\n!python examples/finetune_flickr_style/assemble_data.py --workers=-1 --seed=1701 --images=$NUM_STYLE_IMAGES --label=$NUM_STYLE_LABELS\n# back to examples\nos.chdir('examples')\n"

定義weights，路徑指向我之前下載的使用ImageNet數據集預訓練好的權重，請確保這個文件要存在。

import os weights = os.path.join(caffe_root, 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel') assert os.path.exists(weights)

從ilsvrc12/synset_words.txt導入1000個ImageNet的標簽，并從finetune_flickr_style/style_names.txt導入5個style數據集的標簽。

# Load ImageNet labels to imagenet_labels imagenet_label_file = caffe_root + 'data/ilsvrc12/synset_words.txt' imagenet_labels = list(np.loadtxt(imagenet_label_file, str, delimiter='\t')) assert len(imagenet_labels) == 1000 print 'Loaded ImageNet labels:\n', '\n'.join(imagenet_labels[:10] + ['...'])# Load style labels to style_labels style_label_file = caffe_root + 'examples/finetune_flickr_style/style_names.txt' style_labels = list(np.loadtxt(style_label_file, str, delimiter='\n')) if NUM_STYLE_LABELS > 0:style_labels = style_labels[:NUM_STYLE_LABELS] print '\nLoaded style labels:\n', ', '.join(style_labels) Loaded ImageNet labels: n01440764 tench, Tinca tinca n01443537 goldfish, Carassius auratus n01484850 great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias n01491361 tiger shark, Galeocerdo cuvieri n01494475 hammerhead, hammerhead shark n01496331 electric ray, crampfish, numbfish, torpedo n01498041 stingray n01514668 cock n01514859 hen n01518878 ostrich, Struthio camelus ...Loaded style labels: Detailed, Pastel, Melancholy, Noir, HDR

2.定義網絡并運行

我們一開始先定義caffenet()函數，用來初始化CaffeNet結構（AlexNet的一個小的變種網絡）。該函數使用參數來指定數據和輸出類別的數量。

from caffe import layers as L from caffe import params as Pweight_param = dict(lr_mult=1, decay_mult=1) bias_param = dict(lr_mult=2, decay_mult=0) learned_param = [weight_param, bias_param]frozen_param = [dict(lr_mult=0)] * 2# 卷積層加ReLU單元 def conv_relu(bottom, ks, nout, stride=1, pad=0, group=1,param=learned_param,weight_filler=dict(type='gaussian', std=0.01),bias_filler=dict(type='constant', value=0.1)):conv = L.Convolution(bottom, kernel_size=ks, stride=stride,num_output=nout, pad=pad, group=group,param=param, weight_filler=weight_filler,bias_filler=bias_filler)return conv, L.ReLU(conv, in_place=True)# 全連接層加ReLU單元 def fc_relu(bottom, nout, param=learned_param,weight_filler=dict(type='gaussian', std=0.005),bias_filler=dict(type='constant', value=0.1)):fc = L.InnerProduct(bottom, num_output=nout, param=param,weight_filler=weight_filler,bias_filler=bias_filler)return fc, L.ReLU(fc, in_place=True)# 最大池化 def max_pool(bottom, ks, stride=1):return L.Pooling(bottom, pool=P.Pooling.MAX, kernel_size=ks, stride=stride)# caffenet網絡 def caffenet(data, label=None, train=True, num_classes=1000,classifier_name='fc8', learn_all=False):"""Returns a NetSpec specifying CaffeNet, following the original proto textspecification (./models/bvlc_reference_caffenet/train_val.prototxt)."""n = caffe.NetSpec()# 按照套路來，一層一層接下去n.data = dataparam = learned_param if learn_all else frozen_paramn.conv1, n.relu1 = conv_relu(n.data, 11, 96, stride=4, param=param)n.pool1 = max_pool(n.relu1, 3, stride=2)n.norm1 = L.LRN(n.pool1, local_size=5, alpha=1e-4, beta=0.75)n.conv2, n.relu2 = conv_relu(n.norm1, 5, 256, pad=2, group=2, param=param)n.pool2 = max_pool(n.relu2, 3, stride=2)n.norm2 = L.LRN(n.pool2, local_size=5, alpha=1e-4, beta=0.75)n.conv3, n.relu3 = conv_relu(n.norm2, 3, 384, pad=1, param=param)n.conv4, n.relu4 = conv_relu(n.relu3, 3, 384, pad=1, group=2, param=param)n.conv5, n.relu5 = conv_relu(n.relu4, 3, 256, pad=1, group=2, param=param)n.pool5 = max_pool(n.relu5, 3, stride=2)n.fc6, n.relu6 = fc_relu(n.pool5, 4096, param=param)# 訓練集還要加上一個Dropout，測試集就不需要；加上Dropout，以防止過擬合if train:n.drop6 = fc7input = L.Dropout(n.relu6, in_place=True)else:fc7input = n.relu6n.fc7, n.relu7 = fc_relu(fc7input, 4096, param=param)# 訓練集還要加上一個Dropout，測試集就不需要；加上Dropout，以防止過擬合if train:n.drop7 = fc8input = L.Dropout(n.relu7, in_place=True)else:fc8input = n.relu7# always learn fc8 (param=learned_param)fc8 = L.InnerProduct(fc8input, num_output=num_classes, param=learned_param)# give fc8 the name specified by argument `classifier_name`n.__setattr__(classifier_name, fc8)# 如果不是訓練模式，即測試模式，fc8接上一個softmax，輸出置信率if not train:n.probs = L.Softmax(fc8)# 如果給了label，建立loss和acc層，loss為損失函數，acc計算準確率if label is not None:n.label = labeln.loss = L.SoftmaxWithLoss(fc8, n.label)n.acc = L.Accuracy(fc8, n.label)# write the net to a temporary file and return its filenamewith tempfile.NamedTemporaryFile(delete=False) as f:f.write(str(n.to_proto()))return f.name

現在，我們來建立一個CaffeNet，輸入為沒打標簽的”dummy data”。這樣子，我們可以從外部設置它的輸入數據，也能看看它預測的ImageNet類別是哪個。

dummy_data = L.DummyData(shape=dict(dim=[1, 3, 227, 227])) imagenet_net_filename = caffenet(data=dummy_data, train=False) imagenet_net = caffe.Net(imagenet_net_filename, weights, caffe.TEST)

定義一個style_net函數，調用前面的caffenet函數，輸入的數據為Flickr style數據集。
這個新的網絡也有CaffeNet的結構，但是區別在輸入和輸出上：

• 輸入是我們下載好的Flickr style數據集，使用ImageData層將其讀入
• 輸出是一個超過20個類的分布，而不是原始的ImageNet類別的1000個類
• 分類層由fc8被重命名或fc8_flickr，以告訴Caffe不要從ImageNet預訓練模型中導入原始的fc8層

def style_net(train=True, learn_all=False, subset=None):if subset is None:subset = 'train' if train else 'test'source = caffe_root + 'data/flickr_style/%s.txt' % subsettransform_param = dict(mirror=train, crop_size=227,mean_file=caffe_root + 'data/ilsvrc12/imagenet_mean.binaryproto')style_data, style_label = L.ImageData(transform_param=transform_param, source=source,batch_size=50, new_height=256, new_width=256, ntop=2)return caffenet(data=style_data, label=style_label, train=train,num_classes=NUM_STYLE_LABELS,classifier_name='fc8_flickr',learn_all=learn_all)

使用style_net函數來初始化untrained_style_net，結構也是CaffeNet，但是輸入圖像是來自style數據集，權重是來自ImageNet預訓練模型。
在untrained_style_net調用forward函數來從style數據集獲取一個batch。

untrained_style_net = caffe.Net(style_net(train=False, subset='train'),weights, caffe.TEST) untrained_style_net.forward() style_data_batch = untrained_style_net.blobs['data'].data.copy() style_label_batch = np.array(untrained_style_net.blobs['label'].data, dtype=np.int32)

從一個batch的50張圖像中選取一整圖像輸入style net（前面使用style_net()函數定義的網絡，后面都簡稱style net）。這里我們任意選一張圖片，就選取一個batch中的第8張圖片。將圖片顯示出來，然后跑一邊ImageNet預訓練模型imagenet_net，接著顯示從1000個ImageNet類中預測的前5個結果。
下面，我們選了一張圖片，這張圖片中是有關海灘的，由于”sandbar”和”seashore”都是ImageNet-1000中的類別，所以網絡在預測這張圖片時預測結果還算合理。然而對于其他圖片，預測結果就不怎么好了，有時由于網絡沒能檢測到圖片中的物體，也可能不是所有圖片都包含ImageNet的100個類別中的物體。修改batch_index變量，它的默認值是8，也可改成0-49中的任意數值（因為一個batch就只有50個樣本），來看看預測結果。（如果不想使用這個batch的50張圖像，可以運行上面的cell重新導入一個新的batch到style_net）

def disp_preds(net, image, labels, k=5, name='ImageNet'):input_blob = net.blobs['data']net.blobs['data'].data[0, ...] = imageprobs = net.forward(start='conv1')['probs'][0]top_k = (-probs).argsort()[:k]print 'top %d predicted %s labels =' % (k, name)print '\n'.join('\t(%d) %5.2f%% %s' % (i+1, 100*probs[p], labels[p])for i, p in enumerate(top_k))def disp_imagenet_preds(net, image):disp_preds(net, image, imagenet_labels, name='ImageNet')def disp_style_preds(net, image):disp_preds(net, image, style_labels, name='style') batch_index = 8 image = style_data_batch[batch_index] plt.imshow(deprocess_net_image(image)) print 'actual label =', style_labels[style_label_batch[batch_index]] actual label = Melancholy

disp_imagenet_preds(imagenet_net, image) top 5 predicted ImageNet labels =(1) 69.89% n09421951 sandbar, sand bar(2) 21.75% n09428293 seashore, coast, seacoast, sea-coast(3) 3.22% n02894605 breakwater, groin, groyne, mole, bulwark, seawall, jetty(4) 1.89% n04592741 wing(5) 1.23% n09332890 lakeside, lakeshore disp_style_preds(untrained_style_net, image) top 5 predicted style labels =(1) 20.00% Detailed(2) 20.00% Pastel(3) 20.00% Melancholy(4) 20.00% Noir(5) 20.00% HDR

因為這兩個模型在conv1到fc7層之間使用的是相同的預訓練權重，所以我們也可以在分類層變成與ImageNet預訓練模型一樣之前，驗證在fc7上的激勵函數輸出。

diff = untrained_style_net.blobs['fc7'].data[0] - imagenet_net.blobs['fc7'].data[0] error = (diff ** 2).sum() assert error < 1e-8

刪除untrained_style_net以節約內存。imagenet_net先放一放，后面還會用到。

del untrained_style_net

3.訓練風格分類器

現在我們要創建一個函數solver來創建caffe的solver，我們可以用這個solver來訓練網絡。在這個函數中，我們將為各種用于訓練網絡、顯示、”snapshotting”的各種參數設置初值。請參考注釋理解各個參數的作用。你也可以試著自己修改一些參數，看看能不能取得更好的效果！

from caffe.proto import caffe_pb2def solver(train_net_path, test_net_path=None, base_lr=0.001):s = caffe_pb2.SolverParameter()# Specify locations of the train and (maybe) test networks.s.train_net = train_net_path# 設置了與測試相關的參數：test_interval：每訓練1000次測試一次；test_iter：測試中，每次迭代送入100個batch；if test_net_path is not None:s.test_net.append(test_net_path)s.test_interval = 1000 # Test after every 1000 training iterations.s.test_iter.append(100) # Test on 100 batches each time we test.# The number of iterations over which to average the gradient.# Effectively boosts the training batch size by the given factor, without# affecting memory utilization.s.iter_size = 1# 最大迭代次數s.max_iter = 100000 # # of times to update the net (training iterations)# Solve using the stochastic gradient descent (SGD) algorithm.# Other choices include 'Adam' and 'RMSProp'.s.type = 'SGD' # 迭代使用的優化算法：SGD——隨機梯度下降法，也可以試試其他的算法比如：Adam、RMSProp# Set the initial learning rate for SGD.s.base_lr = base_lr # SGD的初始學習率# Set `lr_policy` to define how the learning rate changes during training.# Here, we 'step' the learning rate by multiplying it by a factor `gamma`# every `stepsize` iterations.s.lr_policy = 'step's.gamma = 0.1s.stepsize = 20000# Set other SGD hyperparameters. Setting a non-zero `momentum` takes a# weighted average of the current gradient and previous gradients to make# learning more stable. L2 weight decay regularizes learning, to help prevent# the model from overfitting.s.momentum = 0.9s.weight_decay = 5e-4# Display the current training loss and accuracy every 1000 iterations.s.display = 1000 # 每迭代1000次，會在終端打印信息，包括訓練的loss值和準確率# Snapshots are files used to store networks we've trained. Here, we'll# snapshot every 10K iterations -- ten times during training.s.snapshot = 10000 # 每過10000次迭代，保存一次當前網絡s.snapshot_prefix = caffe_root + 'models/finetune_flickr_style/finetune_flickr_style' # 保存網絡的路徑# Train on the GPU. Using the CPU to train large networks is very slow. # s.solver_mode = caffe_pb2.SolverParameter.GPUs.solver_mode = caffe_pb2.SolverParameter.CPU # 原本這里應該是GPU模式，我在筆記本上跑，所以換成了CPU模式# Write the solver to a temporary file and return its filename.# 寫入臨時文件with tempfile.NamedTemporaryFile(delete=False) as f:f.write(str(s))return f.name

現在我們要調用上面定義好的solver來訓練style網絡的分類層。
不過，如果想要在命令行下調用solver來訓練網絡，也是可以的。指令如下：
build/tools/caffe train \ -solver models/finetune_flickr_style/solver.prototxt \ -weights models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel \ -gpu 0
補充：如果不適用gpu模式不加-gpu 0就可以了。
但是我們在這個例子中使用python來訓練網絡。
首先，定義一個run_solvers函數，這個函數會在循環中獲取solvers列表的每個元素，并一步一步迭代訓練網絡，同時記錄每次迭代的損失值和準確率。最后會將訓練好的權重保存到一個文件中。

def run_solvers(niter, solvers, disp_interval=10):"""Run solvers for niter iterations,returning the loss and accuracy recorded each iteration.`solvers` is a list of (name, solver) tuples."""blobs = ('loss', 'acc')loss, acc = ({name: np.zeros(niter) for name, _ in solvers}for _ in blobs)for it in range(niter):for name, s in solvers:s.step(1) # run a single SGD step in Caffeloss[name][it], acc[name][it] = (s.net.blobs[b].data.copy()for b in blobs)if it % disp_interval == 0 or it + 1 == niter:loss_disp = '; '.join('%s: loss=%.3f, acc=%2d%%' %(n, loss[n][it], np.round(100*acc[n][it]))for n, _ in solvers)print '%3d) %s' % (it, loss_disp) # Save the learned weights from both nets.weight_dir = tempfile.mkdtemp()weights = {}for name, s in solvers:filename = 'weights.%s.caffemodel' % nameweights[name] = os.path.join(weight_dir, filename)s.net.save(weights[name])return loss, acc, weights

接下來，運行創建好的solver來訓練那個用于風格識別的網絡。我們接下來會創建兩個網絡——一個（style_solver）的使用ImageNet預訓練網絡的權重初始化，另一個（scratch_style_solver）使用隨機初始化的權重。
在訓練過程中，我們會看到使用ImageNet預訓練好的權重的網絡，相比隨機初始化權重的網絡訓練得更快，準確率也更高。

niter = 200 # number of iterations to train# Reset style_solver as before. style_solver_filename = solver(style_net(train=True)) style_solver = caffe.get_solver(style_solver_filename) style_solver.net.copy_from(weights)# For reference, we also create a solver that isn't initialized from # the pretrained ImageNet weights. scratch_style_solver_filename = solver(style_net(train=True)) scratch_style_solver = caffe.get_solver(scratch_style_solver_filename)print 'Running solvers for %d iterations...' % niter solvers = [('pretrained', style_solver),('scratch', scratch_style_solver)] loss, acc, weights = run_solvers(niter, solvers) print 'Done.'train_loss, scratch_train_loss = loss['pretrained'], loss['scratch'] train_acc, scratch_train_acc = acc['pretrained'], acc['scratch'] style_weights, scratch_style_weights = weights['pretrained'], weights['scratch']# Delete solvers to save memory. del style_solver, scratch_style_solver, solvers Running solvers for 200 iterations...0) pretrained: loss=1.609, acc= 0%; scratch: loss=1.609, acc= 0%10) pretrained: loss=1.371, acc=46%; scratch: loss=1.625, acc=14%20) pretrained: loss=1.082, acc=58%; scratch: loss=1.641, acc=12%30) pretrained: loss=0.994, acc=58%; scratch: loss=1.612, acc=22%40) pretrained: loss=0.893, acc=58%; scratch: loss=1.593, acc=24%50) pretrained: loss=1.240, acc=52%; scratch: loss=1.611, acc=30%60) pretrained: loss=1.096, acc=54%; scratch: loss=1.621, acc=16%70) pretrained: loss=0.989, acc=50%; scratch: loss=1.591, acc=28%80) pretrained: loss=0.962, acc=68%; scratch: loss=1.593, acc=34%90) pretrained: loss=1.172, acc=56%; scratch: loss=1.606, acc=24% 100) pretrained: loss=0.849, acc=64%; scratch: loss=1.587, acc=30% 110) pretrained: loss=1.005, acc=52%; scratch: loss=1.587, acc=30% 120) pretrained: loss=0.870, acc=64%; scratch: loss=1.595, acc=24% 130) pretrained: loss=0.970, acc=62%; scratch: loss=1.590, acc=28% 140) pretrained: loss=0.908, acc=58%; scratch: loss=1.603, acc=18% 150) pretrained: loss=0.608, acc=76%; scratch: loss=1.614, acc=20% 160) pretrained: loss=0.816, acc=70%; scratch: loss=1.598, acc=26% 170) pretrained: loss=1.281, acc=52%; scratch: loss=1.622, acc=16% 180) pretrained: loss=0.870, acc=72%; scratch: loss=1.630, acc=12% 190) pretrained: loss=0.909, acc=66%; scratch: loss=1.609, acc=20% 199) pretrained: loss=1.086, acc=62%; scratch: loss=1.616, acc=18% Done.

對比兩個網絡訓練的loss和acc看看。可以看出使用ImageNet預訓練網絡的loss的下降速度非常快，而隨機初始化權重的網絡的訓練速度則慢很多。

plot(np.vstack([train_loss, scratch_train_loss]).T) xlabel('Iteration #') ylabel('Loss') Text(0,0.5,u'Loss')

plot(np.vstack([train_acc, scratch_train_acc]).T) xlabel('Iteration #') ylabel('Accuracy') Text(0,0.5,u'Accuracy')

再來看看迭代200次后的測試準確率。要預測的只有5個類，那么平均下來的準確率應該是20%左右。我們肯定期望，兩個網絡取得的結果都超過隨機狀況下的20%的準確率，另外，我們還期望使用ImageNet預訓練網絡的結果遠比隨機初始化網絡的結果好。讓我們拭目以待吧！

def eval_style_net(weights, test_iters=10):test_net = caffe.Net(style_net(train=False), weights, caffe.TEST)accuracy = 0for it in xrange(test_iters):accuracy += test_net.forward()['acc']accuracy /= test_itersreturn test_net, accuracy test_net, accuracy = eval_style_net(style_weights) print 'Accuracy, trained from ImageNet initialization: %3.1f%%' % (100*accuracy, ) scratch_test_net, scratch_accuracy = eval_style_net(scratch_style_weights) print 'Accuracy, trained from random initialization: %3.1f%%' % (100*scratch_accuracy, ) Accuracy, trained from ImageNet initialization: 51.4% Accuracy, trained from random initialization: 23.6%

端到端微調風格網絡

最后，我們再次訓練前面的兩個網絡，就從剛才訓練學習到的參數開始繼續訓練。這次唯一的區別是我們會以“端到端”的方式訓練參數，即訓練網絡中所有的層，從起始的conv1層直接送入圖像。我們將learn_all=True傳遞給前面定義的style_net函數，如此一來，在網絡中會給所有的參數都乘上一個非0的lr_mult參數。在默認情況下，是learn_all=False，所有預訓練層（conv1到fc7）的參數都被凍結了（lr_mult=0），我們訓練的只有分類層fc8_flickr。
請注意，這兩個網絡開始訓練時的準確率大致相當于之前訓練結束時的準確率。為了更科學一些，我們還要使用與之相同的訓練步驟，但結構不是端到端的，來確認我們的結果并不是因為訓練了兩倍的時間長度才取得更好的結果的。

end_to_end_net = style_net(train=True, learn_all=True)# Set base_lr to 1e-3, the same as last time when learning only the classifier. # You may want to play around with different values of this or other # optimization parameters when fine-tuning. For example, if learning diverges # (e.g., the loss gets very large or goes to infinity/NaN), you should try # decreasing base_lr (e.g., to 1e-4, then 1e-5, etc., until you find a value # for which learning does not diverge). base_lr = 0.001style_solver_filename = solver(end_to_end_net, base_lr=base_lr) style_solver = caffe.get_solver(style_solver_filename) style_solver.net.copy_from(style_weights)scratch_style_solver_filename = solver(end_to_end_net, base_lr=base_lr) scratch_style_solver = caffe.get_solver(scratch_style_solver_filename) scratch_style_solver.net.copy_from(scratch_style_weights)print 'Running solvers for %d iterations...' % niter solvers = [('pretrained, end-to-end', style_solver),('scratch, end-to-end', scratch_style_solver)] _, _, finetuned_weights = run_solvers(niter, solvers) print 'Done.'style_weights_ft = finetuned_weights['pretrained, end-to-end'] scratch_style_weights_ft = finetuned_weights['scratch, end-to-end']# Delete solvers to save memory. del style_solver, scratch_style_solver, solvers Running solvers for 200 iterations...0) pretrained, end-to-end: loss=0.851, acc=68%; scratch, end-to-end: loss=1.584, acc=28%10) pretrained, end-to-end: loss=1.312, acc=56%; scratch, end-to-end: loss=1.637, acc=14%20) pretrained, end-to-end: loss=0.802, acc=70%; scratch, end-to-end: loss=1.627, acc=16%30) pretrained, end-to-end: loss=0.786, acc=66%; scratch, end-to-end: loss=1.595, acc=22%40) pretrained, end-to-end: loss=0.748, acc=74%; scratch, end-to-end: loss=1.575, acc=24%50) pretrained, end-to-end: loss=0.818, acc=72%; scratch, end-to-end: loss=1.595, acc=34%60) pretrained, end-to-end: loss=0.773, acc=68%; scratch, end-to-end: loss=1.560, acc=26%70) pretrained, end-to-end: loss=0.617, acc=84%; scratch, end-to-end: loss=1.540, acc=28%80) pretrained, end-to-end: loss=0.561, acc=76%; scratch, end-to-end: loss=1.494, acc=46%90) pretrained, end-to-end: loss=0.824, acc=62%; scratch, end-to-end: loss=1.521, acc=30% 100) pretrained, end-to-end: loss=0.624, acc=80%; scratch, end-to-end: loss=1.482, acc=30% 110) pretrained, end-to-end: loss=0.586, acc=76%; scratch, end-to-end: loss=1.566, acc=32% 120) pretrained, end-to-end: loss=0.633, acc=72%; scratch, end-to-end: loss=1.547, acc=26% 130) pretrained, end-to-end: loss=0.547, acc=82%; scratch, end-to-end: loss=1.458, acc=28% 140) pretrained, end-to-end: loss=0.431, acc=80%; scratch, end-to-end: loss=1.469, acc=28% 150) pretrained, end-to-end: loss=0.514, acc=78%; scratch, end-to-end: loss=1.508, acc=32% 160) pretrained, end-to-end: loss=0.475, acc=82%; scratch, end-to-end: loss=1.440, acc=28% 170) pretrained, end-to-end: loss=0.490, acc=78%; scratch, end-to-end: loss=1.554, acc=40% 180) pretrained, end-to-end: loss=0.449, acc=80%; scratch, end-to-end: loss=1.470, acc=32% 190) pretrained, end-to-end: loss=0.367, acc=84%; scratch, end-to-end: loss=1.463, acc=34% 199) pretrained, end-to-end: loss=0.492, acc=82%; scratch, end-to-end: loss=1.364, acc=52% Done.

讓我們現在測試一下端到端微調模型。由于網絡中所有的層都參與到了訓練當中，所以我們期望這次的結果會比之前只讓分類層參與到訓練中的網絡取得更好的效果。

test_net, accuracy = eval_style_net(style_weights_ft) print 'Accuracy, finetuned from ImageNet initialization: %3.1f%%' % (100*accuracy, ) scratch_test_net, scratch_accuracy = eval_style_net(scratch_style_weights_ft) print 'Accuracy, finetuned from random initialization: %3.1f%%' % (100*scratch_accuracy, ) Accuracy, finetuned from ImageNet initialization: 54.4% Accuracy, finetuned from random initialization: 40.2%

先看看輸入的圖片，和它在端到端模型中的預測結果。

plt.imshow(deprocess_net_image(image)) disp_style_preds(test_net, image) top 5 predicted style labels =(1) 87.82% Melancholy(2) 6.10% Pastel(3) 5.66% HDR(4) 0.41% Detailed(5) 0.01% Noir

喔！預測結果相比之前好了不少。但是請注意，這個圖片是來自數據集的，所以網絡在訓練時就看過它的標簽了。
接下來，我們從測試集中取出一張圖片，看看端到端模型的預測結果如何。

batch_index = 1 image = test_net.blobs['data'].data[batch_index] plt.imshow(deprocess_net_image(image)) print 'actual label =', style_labels[int(test_net.blobs['label'].data[batch_index])] actual label = Pastel

disp_style_preds(test_net, image) top 5 predicted style labels =(1) 99.48% Pastel(2) 0.47% Detailed(3) 0.05% HDR(4) 0.00% Melancholy(5) 0.00% Noir

我們也可以看看這張圖片在scratch網絡中的預測結果。它也輸出了正確的結果，盡管置信率較另一個（使用預訓練權重的網絡）更低。

disp_style_preds(scratch_test_net, image) top 5 predicted style labels =(1) 46.02% Pastel(2) 23.50% Melancholy(3) 16.43% Detailed(4) 11.64% HDR(5) 2.40% Noir

當然我們還可以看看在ImageNet模型上的預測結果：

disp_imagenet_preds(imagenet_net, image) top 5 predicted ImageNet labels =(1) 34.90% n07579787 plate(2) 21.63% n04263257 soup bowl(3) 17.75% n07875152 potpie(4) 5.72% n07711569 mashed potato(5) 5.27% n07584110 consomme

總結

以上是生活随笔為你收集整理的Caffe官方教程翻译（7）：Fine-tuning for Style Recognition的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。