?

本博文介紹如何手把手制作自己的數據集，并使用SegNet網絡進行語義分割，如何訓練，如何測試自己的分割模型。

----------------------------------------------------------------------------------------------------------------------------------------------------------

感謝：

1.源碼參考cudnn5版本的caffe-segnet，https://github.com/TimoSaemann/caffe-segnet-cudnn5

2.作者的官方指導文檔：http://mi.eng.cam.ac.uk/projects/segnet/tutorial.html

3.一份使用指導：https://github.com/TqDavid/SegNet-Tutorial/tree/master/CamVid

4.如果想測試SegNet的效果，這里有官方的在線測試demo: http://mi.eng.cam.ac.uk/projects/segnet/#demo

5.一份segnet的預訓練model

https://github.com/alexgkendall/SegNet-Tutorial/blob/master/Example_Models/segnet_model_zoo.md，以及相對應的
segnet_basic_inference.prototxt
segnet_basic_solver.prototxt
segnet_basic_train.prototxt

6.參考了該博主的文章https://blog.csdn.net/caicai2526/article/details/77170223

7.以及該博主的文章https://blog.csdn.net/hjxu2016/article/details/77994925

8.對于已有mask和image,需要resize,可參考https://blog.csdn.net/u013249853/article/details/79827469

--------------------------------------------------------------------------------------------------------------------------------------------------------------

言歸正傳，開始正題。

下載上述caffe-segnet-cudnn5版本的代碼，其路徑后文稱之為caffe_root,以下的操作都在該caffe_root目錄下進行。

整體目錄包含如下文件（夾）：

一、制作自己數據集

??????? SegNet網絡需要兩個輸入，一個是原始圖像，三通道的;一個是mask label,也就是待分割目標的標簽，mask要求是單通道8位灰度圖，格式是uint8, 像素0表示背景，像素1表示目標的位置。 下面分別準備這兩樣東西。

???????? 1. 原始圖像就是你自己的照片了，注意一下：如果你不更改SegNet中參數，可以先保持與原作者的圖像尺寸一致，后面可以根據自己的情況在更改，一般上采樣2x，這里采用與原作者相同的圖像尺寸，（360， 480）彩色圖片，如果不是這樣，建議resize一下，博主自己的圖片是灰度的，resize前有一個轉成rgb的過程，python代碼函數；

??? # testdir為自己數據集路徑，樓主的圖是灰度圖，所以有一個轉換成rgb的過程，然后保存。
??? def resize_image_batch():
??????? file_list = os.listdir(testdir)
??????? for item in file_list:
??????????? #print(traindir + item)
??????????? imname = testdir + item
??????????? im = cv2.imread(imname, 0)
??????????? im = cv2.resize(im, (480, 360))
??????????? im_RGB = cv2.cvtColor(im, cv2.COLOR_GRAY2RGB)
??????????? print(im.shape)
??????????? print(im_RGB.shape)
??????????? #print(im)
??????????? print("============================")
??????????? #print(im_RGB[:,:,0])
??????????? #print(im_RGB[:,:,1])
??????????? #print(im_RGB[:,:,2])
??????????? print(im.dtype)
??????????? print(im_RGB.dtype)
??????????? cv2.imwrite('zht_test_use/'+ item, im_RGB)
??????????? #print("success!")

2.準備mask。

???? 首先，博主的mask來源是用labelme 得到的。labelme的使用方法，這里不再贅述。簡言之，對于每一張圖，我們可以得到其

json標注文件， 再通過，labelme_json_to_dataset命令，可以得到最終我們需要的label mask，我們所需的是文件夾下的label.png。

labelme_json_to_dataset批量生成數據集的shell腳本如下;

??? for item in $(find ./huaweishouji_20170720_train_360x480_try -iname "*.json");
??? do?? ?
???? echo $item;??? ?
???? labelme_json_to_dataset $item;
??? done

接下來將lable.png轉換成uint8，背景像素是0，目標像素是1的mask..convert_to_mask 代碼如下：

??? def convert_to_mask():
??????? file_list = os.listdir(traindir)
??????? for item in file_list:
??????????? item_list = os.listdir(traindir + item)
??????????? #print("len(item) : ", len(item_list))
??????????? for atom in item_list:
??????????????? if atom == "label.png":
??????????????????? np.set_printoptions(threshold='nan')
??????????????????? imname = traindir + item + "/" + atom
??????????????????? #print(imname)
??????????????????? im = io.imread(imname, 1)
??????????????????? print(imname)
??????????????????? #print(im[:, :])
??????????????????? print(im.shape)
??????????????????? print(im.dtype)
??????????????????? print("-------------after-----------------")
??????????????????? img = (im * 255).astype(np.uint8)
??????????????????? _, im_th= cv2.threshold(img, 0.0000000000000001, 1, cv2.THRESH_BINARY)
???????????????????? ?
??????????????????? #print(img.shape)
?????????????????? ?
??????????????????? print(im_th.shape)
??????????????????? print(im_th.dtype)
??????????????????? #print(im_th[ :, :])
??????????????????? print(item[:-5] + ".png")
??????????????????? cv2.imwrite(train_write_dir + item[:-5] + '.png', im_th )
?? ??? ??? ??? ??? ?#print(im[:,:,0])
??????????????????? #print(im[:,:,1])
??????????????????? #print(im[:,:,2])
??????????????????? #im_RGB = cv2.cvtColor(im, cv2.COLOR_GRAY2RGB)
??????????? #print(traindir + item)
??????????? #imname = traindir + item
??????????? #im = cv2.imread(imname, 0)
??????????? #im = cv2.resize(im, (480, 360))
??????????? #im_RGB = cv2.cvtColor(im, cv2.COLOR_GRAY2RGB)
?????????? ?
??????????? #print(im.shape)
??????????? #print(im_RGB.shape)
??????????? #print(im)
??????????? #print("============================")
??????????? #print(im_RGB[:,:,0])
??????????? #print(im_RGB[:,:,1])
??????????? #print(im_RGB[:,:,2])
??????????? #print(im.dtype)
??????????? #print(im_RGB.dtype)
??????????? #cv2.imwrite('huaweishouji_20170720_test_360x480/'+ item, im_RGB)
??? def check_mask():
??????? file_list = os.listdir(train_write_dir)
??????? for item in file_list:
??????????? np.set_printoptions(threshold='nan')
??????????? #item_list = os.listdir(test_write_dir)
??????????? imname = train_write_dir + item
??????????? #print("len(item) : ", len(item_list))
??????????? print(item)

check_mask（）是檢驗函數。驗證背景是0，目標是1，且為uint8編碼。

將上述原始圖像和mask，劃分出訓練集和測試集，分別放置在caffe_root/data/mydata/train和test下。mydata/下的結構是如圖，

，其中train/和test/ 結構如下：，image/和mask/下分別是上面得到的圖像和mask文件。注意圖像和mask名稱保持一致。

3.準備上面mydata/下的train.txt和test.txt，然后我們需要制作一個txt的列表，左邊是原圖的根目錄路徑，右邊是mask圖的根目錄路徑，中間以空格隔開，一定要注意，mask的路徑和原圖的路徑一定要對，列表形式如下：

/home/hjxu/caffe_examples/segnet_xu/data/test/image/8900_11800.tiff /home/hjxu/caffe_examples/segnet_xu/data/test/mask/8500_10200_ConfidenceMap.png
/home/hjxu/caffe_examples/segnet_xu/data/test/image/10100_9800.tiff /home/hjxu/caffe_examples/segnet_xu/data/test/mask/8900_11800_ConfidenceMap.png
/home/hjxu/caffe_examples/segnet_xu/data/test/image/8900_9000.tiff /home/hjxu/caffe_examples/segnet_xu/data/test/mask/9300_10200_ConfidenceMap.png
/home/hjxu/caffe_examples/segnet_xu/data/test/image/8900_10200.tiff /home/hjxu/caffe_examples/segnet_xu/data/test/mask/8900_9000_ConfidenceMap.png

具體的shell腳本

??? #!/usr/bin/env sh
??? DATA_train=/home/ccf/CCF/Cell_segnet/data/data_train_enhancement/train/image
??? MASK_train=/home/ccf/CCF/Cell_segnet/data/data_train_enhancement/train/mask
??? DATA_test=/home/ccf/CCF/Cell_segnet/data/data_train_enhancement/test/image
??? MASK_test=/home/ccf/CCF/Cell_segnet/data/data_train_enhancement/test/mask
??? MY=/home/ccf/CCF/Cell_segnet/data/data_train_enhancement
??? ?
??? ################################################
??? rm -rf $MY/train.txt
??? ?
??? echo "Create train.txt"
??? find $DATA_train/ -name "*.tif">>$MY/img.txt
??? find $MASK_train/ -name "*.tif">>$MY/mask.txt
??? paste -d " " $MY/img.txt $MY/mask.txt>$MY/train.txt
??? ?
??? rm -rf $MY/img.txt
??? rm -rf $MY/mask.txt
??? ?
??? ##################################################
??? rm -rf $MY/test.txt
??? ?
??? echo "Create test.txt"
??? find $DATA_test/ -name "*.tif">>$MY/img.txt
??? find $MASK_test/ -name "*.tif">>$MY/mask.txt
??? paste -d " " $MY/img.txt $MY/mask.txt>$MY/test.txt
??? ?
??? rm -rf $MY/img.txt
??? rm -rf $MY/mask.txt

可以適當修改，以適應自己的路徑。

二、訓練

????? 訓練之前，在訓練的時候我們可以根據自己訓練的要求更改分割的類型，segnet對原來是11中類型，在博主只有兩種類型，這就會遇到對網絡的修改，同時數據輸入的也是一樣原來的是360*480，網絡中的修改根據個人的要求以及效果進行修改，修改輸出層參數num_output為2，以及class-weighting，只有兩個。要注意的是上采樣upsample這個參數的修改，以及最后的class_weighting，對于class_weighting個數以及參數是根據自己的數據以及要求設定，輸出幾個類別class_weighting就有幾個，對于class_weighting參數的確定是根據訓練數據的mask中每一種類型的label確定的，就算方法：（all_label/class）/label,下面是計算的matlab算法代碼：

??? clear;
??? clc;
??? Path='C:\\Users\\xxxxx\\Desktop\\mask\\'
??? % save_mask_path='/home/ccf/CCF/Cell_segnet/data/mask_change/'
??? files=dir(Path);
??? ?
??? %element = [];
??? for k=3:length(files)
??????? k
??????? subpath=[Path,files(k).name];
??????? name=files(k).name;
??????? image=imread(subpath);
??????? I=image;
??????? name
??????? I
??????? img=uint8(zeros(360,480));
??????? [x,y]=find(I==0);
??????? for i=1:length(x)
??????????? img(x(i),y(i))=0;
??????? end
??????? [x,y]=find(I==1);
??????? for i=1:length(x)
??????????? img(x(i),y(i))=1;
??????? end
??? %???? imwrite(img,[save_mask_path,name]);
??????? label_num=double(unique(img));
??????? element(:,1)=[0;1];
??????? if (length(element(:,1))==length(label_num))
??????????? element(:,1)=label_num;
??????? end
??????? for j=1:length(label_num)
??????????? a=label_num(j);
??????????? e=length(find(img==a));
??????????? element(j,i-1)=e;
??????? end
??? end
??? num=element(:,2:end);
??? sum_num=sum(num,2);
??? median=sum(sum_num)/length(sum_num);
??? class_weighting=median./sum_num;
??? total=[element(:,1),class_weighting];
??? save('class_weight.mat','total');

最后查看class_weighting中保存的值即可，博主是2類，所有最后生成的class_weighting有兩個值，ignore_label=2，num_outout? = 2;完畢。

訓練模型需要的東西：

預訓練模型：

??? Segnet Basic model file: segnet_basic_camvid.prototxt weights: [http://mi.eng.cam.ac.uk/~agk34/resources/SegNet/segnet_basic_camvid.caffemodel]

solver.prototxt如下所示，train.prototxt就不貼了，因為有些長。

??? net: "examples/segnet/segnet_train.prototxt" ??? ??? ?# Change this to the absolute path to your model file
??? test_initialization: false
??? test_iter: 1
??? test_interval: 10000000
??? base_lr: 0.01 #0.1
??? lr_policy: "step"
??? gamma: 1.0
??? stepsize: 50 #10000000
??? display: 20
??? momentum: 0.9
??? max_iter: 50000
??? weight_decay: 0.0005
??? snapshot: 50#1000
??? snapshot_prefix: "examples/segnet/segnet_train/segnet_basic/seg" ??? ?# Change this to the absolute path to where you wish to output solver snapshots
??? solver_mode: GPU

??? #!bin/sh
??? ./build/tools/caffe train -gpu 2,3 -solver examples/segnet/segnet_solver.prototxt -weights examples/segnet/segnet_train/premodel/segnet_basic_camvid.caffemodel # This will begin training SegNet-Basic on GPU 0

上面就是訓練腳本了。完畢。

三、測試

1.生成含bn層的推理模型，腳本是：

??? #!bin/sh
??? echo "------------------generating bn statistics is begin-----------------------------"
??? python generate_bn_statistics.py examples/segnet/segnet_train.prototxt examples/segnet/segnet_train/segnet_basic/seg_iter_15700.caffemodel models/inference? # compute BN statistics for SegNet
??? echo "------------------generating bn statistics is end-----------------------------"

generate_bn_statistics.py如下：

??? #-*-coding:utf8-*-
??? #!/usr/bin/env python
??? import os
??? import numpy as np
??? from skimage.io import ImageCollection
??? from argparse import ArgumentParser
??? ?
??? ?
??? ?
??? ?
??? caffe_root = '/data/xxxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/' ?? ??? ??? ?# Change this to the absolute directoy to SegNet Caffe
??? import sys
??? sys.path.insert(0, caffe_root + 'python')
??? ?
??? import caffe
??? from caffe.proto import caffe_pb2
??? from google.protobuf import text_format
??? ?
??? ?
??? def extract_dataset(net_message):
??????? assert net_message.layer[0].type == "DenseImageData"
??????? source = net_message.layer[0].dense_image_data_param.source
??????? with open(source) as f:
??????????? data = f.read().split()
??????? ims = ImageCollection(data[::2])
??????? labs = ImageCollection(data[1::2])
??????? assert len(ims) == len(labs) > 0
??????? return ims, labs
??? ?
??? ?
??? def make_testable(train_model_path):
??????? # load the train net prototxt as a protobuf message
??????? with open(train_model_path) as f:
??????????? train_str = f.read()
??????? train_net = caffe_pb2.NetParameter()
??????? text_format.Merge(train_str, train_net)
??? ?
??????? # add the mean, var top blobs to all BN layers
??????? for layer in train_net.layer:
??????????? if layer.type == "BN" and len(layer.top) == 1:
??????????????? layer.top.append(layer.top[0] + "-mean")
??????????????? layer.top.append(layer.top[0] + "-var")
??? ?
??????? # remove the test data layer if present
??????? if train_net.layer[1].name == "data" and train_net.layer[1].include:
??????????? train_net.layer.remove(train_net.layer[1])
??????????? if train_net.layer[0].include:
??????????????? # remove the 'include {phase: TRAIN}' layer param
??????????????? train_net.layer[0].include.remove(train_net.layer[0].include[0])
??????? return train_net
??? ?
??? ?
??? def make_test_files(testable_net_path, train_weights_path, num_iterations,
??????????????????????? in_h, in_w):
??????? # load the train net prototxt as a protobuf message
??????? with open(testable_net_path) as f:
??????????? testable_str = f.read()
??????? testable_msg = caffe_pb2.NetParameter()
??????? text_format.Merge(testable_str, testable_msg)
?????? ?
??????? bn_layers = [l.name for l in testable_msg.layer if l.type == "BN"]
??????? bn_blobs = [l.top[0] for l in testable_msg.layer if l.type == "BN"]
??????? bn_means = [l.top[1] for l in testable_msg.layer if l.type == "BN"]
??????? bn_vars = [l.top[2] for l in testable_msg.layer if l.type == "BN"]
??? ?
??????? net = caffe.Net(testable_net_path, train_weights_path, caffe.TEST)
?????? ?
??????? # init our blob stores with the first forward pass
??????? res = net.forward()
??????? bn_avg_mean = {bn_mean: np.squeeze(res[bn_mean]).copy() for bn_mean in bn_means}
??????? bn_avg_var = {bn_var: np.squeeze(res[bn_var]).copy() for bn_var in bn_vars}
??? ?
??????? # iterate over the rest of the training set
??????? for i in xrange(1, num_iterations):
??????????? res = net.forward()
??????????? for bn_mean in bn_means:
??????????????? bn_avg_mean[bn_mean] += np.squeeze(res[bn_mean])
??????????? for bn_var in bn_vars:
??????????????? bn_avg_var[bn_var] += np.squeeze(res[bn_var])
??????????? print 'progress: {}/{}'.format(i, num_iterations)
??? ?
??????? # compute average means and vars
??????? for bn_mean in bn_means:
??????????? bn_avg_mean[bn_mean] /= num_iterations
??????? for bn_var in bn_vars:
??????????? bn_avg_var[bn_var] /= num_iterations
??? ?
??????? for bn_blob, bn_var in zip(bn_blobs, bn_vars):
??????????? m = np.prod(net.blobs[bn_blob].data.shape) / np.prod(bn_avg_var[bn_var].shape)
??????????? bn_avg_var[bn_var] *= (m / (m - 1))
??? ?
??????? # calculate the new scale and shift blobs for all the BN layers
??????? scale_data = {bn_layer: np.squeeze(net.params[bn_layer][0].data)
????????????????????? for bn_layer in bn_layers}
??????? shift_data = {bn_layer: np.squeeze(net.params[bn_layer][1].data)
????????????????????? for bn_layer in bn_layers}
??? ?
??????? var_eps = 1e-9
??????? new_scale_data = {}
??????? new_shift_data = {}
??????? for bn_layer, bn_mean, bn_var in zip(bn_layers, bn_means, bn_vars):
??????????? gamma = scale_data[bn_layer]
??????????? beta = shift_data[bn_layer]
??????????? Ex = bn_avg_mean[bn_mean]
??????????? Varx = bn_avg_var[bn_var]
??????????? new_gamma = gamma / np.sqrt(Varx + var_eps)
??????????? new_beta = beta - (gamma * Ex / np.sqrt(Varx + var_eps))
??? ?
??????????? new_scale_data[bn_layer] = new_gamma
??????????? new_shift_data[bn_layer] = new_beta
??????? print "New data:"
??????? print new_scale_data.keys()
??????? print new_shift_data.keys()
??? ?
??????? # assign computed new scale and shift values to net.params
??????? for bn_layer in bn_layers:
??????????? net.params[bn_layer][0].data[...] = new_scale_data[bn_layer].reshape(
??????????????? net.params[bn_layer][0].data.shape
??????????? )
??????????? net.params[bn_layer][1].data[...] = new_shift_data[bn_layer].reshape(
??????????????? net.params[bn_layer][1].data.shape
??????????? )
?????????? ?
??????? # build a test net prototxt
??????? test_msg = testable_msg
??????? # replace data layers with 'input' net param
??????? data_layers = [l for l in test_msg.layer if l.type.endswith("Data")]
??????? for data_layer in data_layers:
??????????? test_msg.layer.remove(data_layer)
??????? test_msg.input.append("data")
??????? test_msg.input_dim.append(1)
??????? test_msg.input_dim.append(3)
??????? test_msg.input_dim.append(in_h)
??????? test_msg.input_dim.append(in_w)
??????? # Set BN layers to INFERENCE so they use the new stat blobs
??????? # and remove mean, var top blobs.
??????? for l in test_msg.layer:
??????????? if l.type == "BN":
??????????????? if len(l.top) > 1:
??????????????????? dead_tops = l.top[1:]
??????????????????? for dl in dead_tops:
??????????????????????? l.top.remove(dl)
??????????????? l.bn_param.bn_mode = caffe_pb2.BNParameter.INFERENCE
??????? # replace output loss, accuracy layers with a softmax
??????? dead_outputs = [l for l in test_msg.layer if l.type in ["SoftmaxWithLoss", "Accuracy"]]
??????? out_bottom = dead_outputs[0].bottom[0]
??????? for dead in dead_outputs:
??????????? test_msg.layer.remove(dead)
??????? test_msg.layer.add(
??????????? name="prob", type="Softmax", bottom=[out_bottom], top=['prob']
??????? )
??????? return net, test_msg
??? ?
??? ?
??? def make_parser():
??????? p = ArgumentParser()
??????? p.add_argument('train_model')
??????? p.add_argument('weights')
??????? p.add_argument('out_dir')
??????? return p
??? ?
??? ?
??? if __name__ == '__main__':
??????? caffe.set_mode_gpu()
??????? p = make_parser()
??????? args = p.parse_args()
??? ?
??????? # build and save testable net
??????? if not os.path.exists(args.out_dir):
??????????? os.makedirs(args.out_dir)
??????? print "Building BN calc net..."
??????? testable_msg = make_testable(args.train_model)
??????? BN_calc_path = os.path.join(
??????????? args.out_dir, '__for_calculating_BN_stats_' + os.path.basename(args.train_model)
??????? )
??????? with open(BN_calc_path, 'w') as f:
??????????? f.write(text_format.MessageToString(testable_msg))
??? ?
??????? # use testable net to calculate BN layer stats
??????? print "Calculate BN stats..."
??????? train_ims, train_labs = extract_dataset(testable_msg)
??????? train_size = len(train_ims)
??????? minibatch_size = testable_msg.layer[0].dense_image_data_param.batch_size
??????? num_iterations = train_size // minibatch_size + train_size % minibatch_size
??????? in_h, in_w =(360, 480)?? #記得修改和自己圖片一樣的大小
??????? test_net, test_msg = make_test_files(BN_calc_path, args.weights, num_iterations,
???????????????????????????????????????????? in_h, in_w)
?????? ?
??????? # save deploy prototxt
??????? #print "Saving deployment prototext file..."
??????? #test_path = os.path.join(args.out_dir, "deploy.prototxt")
??????? #with open(test_path, 'w') as f:
??????? #??? f.write(text_format.MessageToString(test_msg))
?????? ?
??????? print "Saving test net weights..."
??????? test_net.save(os.path.join(args.out_dir, "test_weights_15750.caffemodel"))?? #記得修改迭代多少次命名
??????? print "done"

2.生成預測圖片

腳本：

??? #!bin/sh
??? echo "-------------------test segmentation is begin---------------------"
??? python test_segmentation.py --model models/inference/segmentation_inference.prototxt --weights models/inference/test_weights_15750.caffemodel --iter 26 #12250 #15750? # Test SegNet
??? echo "-------------------test segmentation is end---------------------"

test_segmentation.py

??? #-*-coding=utf8-*-
??? import numpy as np
??? import matplotlib.pyplot as plt
??? import os.path
??? import json
??? import scipy
??? import argparse
??? import math
??? import pylab
??? from sklearn.preprocessing import normalize
??? import cv2
??? caffe_root = '/data/xxxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/' ?? ??? ??? ?# Change this to the absolute directoy to SegNet Caffe
??? import sys
??? sys.path.insert(0, caffe_root + 'python')
??? ?
??? import caffe
??? ?
??? # Import arguments
??? parser = argparse.ArgumentParser()
??? parser.add_argument('--model', type=str, required=True)
??? parser.add_argument('--weights', type=str, required=True)
??? parser.add_argument('--iter', type=int, required=True)
??? args = parser.parse_args()
??? ?
??? caffe.set_mode_gpu()
??? ?
??? net = caffe.Net(args.model,
??????????????????? args.weights,
??????????????????? caffe.TEST)
??? ?
??? ?
??? for i in range(0, args.iter):
??? ?
?? ??? ?net.forward()
?? ??? ?print(i)
?? ??? ?image = net.blobs['data'].data
?? ??? ?#print(image.shape)
?? ??? ?label = net.blobs['label'].data
?? ??? ?#print(label.shape)
?? ??? ?predicted = net.blobs['prob'].data #predicted: float32
?? ??? ?
?? ??? ?# convert np.float64 to np.uint8
?? ??? ?#image = (image* 50000).astype(np.uint8)
?? ??? ?#lahel = (label * 50000).astype(np.uint8)
?? ??? ?#predicted = (predicted * 50000).astype(np.uint8)
??? ?
?? ??? ?#print(predicted.shape)
?? ??? ?image = np.squeeze(image[0,:,:,:])
?? ??? ?output = np.squeeze(predicted[0,:,:,:])
?? ??? ?ind = np.argmax(output, axis=0)
?? ??? ?cv2.imwrite(str(i%26) + "predicted.png", ind * 100)# predicted: float32, this predicated is kuoda * 100
?? ??? ?r = ind.copy()
?? ??? ?g = ind.copy()
?? ??? ?b = ind.copy()
?? ??? ?r_gt = label.copy()
?? ??? ?g_gt = label.copy()
?? ??? ?b_gt = label.copy()
?? ??? ?#print(output.shape)
?? ??? ?#print(output.dtype)
??? ??? ?#print(output)
??? #?? ?Sky = [128,128,128]
??? #?? ?Building = [128,0,0]
??? #?? ?Pole = [192,192,128]
??? #?? ?Road_marking = [255,69,0]
??? #?? ?Road = [128,64,128]
??? #?? ?Pavement = [60,40,222]
??? #?? ?Tree = [128,128,0]
??? #?? ?SignSymbol = [192,128,128]
??? #?? ?Fence = [64,64,128]
??? #?? ?Car = [64,0,128]
??? #?? ?Pedestrian = [64,64,0]
??? #?? ?Bicyclist = [0,128,192]
??? #?? ?Unlabelled = [0,0,0]
??? ?
??? #?? ?label_colours = np.array([Sky, Building, Pole, Road, Pavement, Tree, SignSymbol, Fence, Car, Pedestrian, Bicyclist, Unlabelled])
??????????? BG = [0,0,0]
??????????? M = [0,255,0]
??????????? label_colours = np.array([BG, M])
?? ??? ?for l in range(0,2):
?? ??? ??? ?r[ind==l] = label_colours[l,0]
?? ??? ??? ?g[ind==l] = label_colours[l,1]
?? ??? ??? ?b[ind==l] = label_colours[l,2]
?? ??? ??? ?r_gt[label==l] = label_colours[l,0]
?? ??? ??? ?g_gt[label==l] = label_colours[l,1]
?? ??? ??? ?b_gt[label==l] = label_colours[l,2]
??????? # we do not normalize
?? ??? ?rgb = np.zeros((ind.shape[0], ind.shape[1], 3))
?? ??? ?rgb[:,:,0] = r#/255.0
?? ??? ?rgb[:,:,1] = g#/255.0
?? ??? ?rgb[:,:,2] = b#/255.0
?? ??? ?rgb_gt = np.zeros((ind.shape[0], ind.shape[1], 3))
?? ??? ?rgb_gt[:,:,0] = r_gt#/255.0
?? ??? ?rgb_gt[:,:,1] = g_gt#/255.0
?? ??? ?rgb_gt[:,:,2] = b_gt#/255.0
??? ?
?? ??? ?image = image#/255.0
??? ?
?? ??? ?image = np.transpose(image, (1,2,0))
?? ??? ?output = np.transpose(output, (1,2,0))
?? ??? ?image = image[:,:,(2,1,0)]
??? ?
??? ?
?? ??? ?#scipy.misc.toimage(rgb, cmin=0.0, cmax=255).save(IMAGE_FILE+'_segnet.png') #保存文件
??? ?
?? ??? ?cv2.imwrite(str(i%26)+'image.png', image.astype(np.uint8))
?? ??? ?cv2.imwrite(str(i%26)+'rgb_gt.png', rgb_gt.astype(np.uint8))
?? ??? ?cv2.imwrite(str(i%26)+'rgb.png', rgb.astype(np.uint8))
??? ?
?? ??? ?
?? ??? ?#plt.figure()
?? ??? ?#plt.imshow(image,vmin=0, vmax=1)? #顯示源文件
?? ??? ?#plt.figure()
?? ??? ?#plt.imshow(rgb_gt,vmin=0, vmax=1) #給的mask圖片，如果測試的圖片沒有mask，可以隨便放個圖片列表，省的修改代碼
?? ??? ?#plt.figure()
?? ??? ?#plt.imshow(rgb,vmin=0, vmax=1) # 預測圖片
?? ??? ?#plt.show()
??? ?
??? ?
??? print 'Success!'

四、mean IOU 評價的計算

??? #!/usr/bin/python
??? ?
??? import numpy as np
??? from skimage import io
??? import cv2
??? ?
??? def pixel_accuracy(eval_segm, gt_segm):
??????? '''
??????? sum_i(n_ii) / sum_i(t_i)
??????? '''
??? ?
??????? check_size(eval_segm, gt_segm)
??? ?
??????? cl, n_cl = extract_classes(gt_segm)
??????? eval_mask, gt_mask = extract_both_masks(eval_segm, gt_segm, cl, n_cl)
??? ?
??????? sum_n_ii = 0
??????? sum_t_i? = 0
??? ?
??????? for i, c in enumerate(cl):
??????????? curr_eval_mask = eval_mask[i, :, :]
??????????? curr_gt_mask = gt_mask[i, :, :]
??? ?
??????????? sum_n_ii += np.sum(np.logical_and(curr_eval_mask, curr_gt_mask))
??????????? sum_t_i? += np.sum(curr_gt_mask)
??? ?
??????? if (sum_t_i == 0):
??????????? pixel_accuracy_ = 0
??????? else:
??????????? pixel_accuracy_ = sum_n_ii / sum_t_i
??? ?
??????? return pixel_accuracy_
??? ?
??? def mean_accuracy(eval_segm, gt_segm):
??????? '''
??????? (1/n_cl) sum_i(n_ii/t_i)
??????? '''
??? ?
??????? check_size(eval_segm, gt_segm)
??? ?
??????? cl, n_cl = extract_classes(gt_segm)
??????? eval_mask, gt_mask = extract_both_masks(eval_segm, gt_segm, cl, n_cl)
??? ?
??????? accuracy = list([0]) * n_cl
??? ?
??????? for i, c in enumerate(cl):
??????????? curr_eval_mask = eval_mask[i, :, :]
??????????? curr_gt_mask = gt_mask[i, :, :]
??? ?
??????????? n_ii = np.sum(np.logical_and(curr_eval_mask, curr_gt_mask))
??????????? t_i? = np.sum(curr_gt_mask)
??? ?
??????????? if (t_i != 0):
??????????????? accuracy[i] = n_ii / t_i
??? ?
??????? mean_accuracy_ = np.mean(accuracy)
??????? return mean_accuracy_
??? ?
??? def mean_IU(eval_segm, gt_segm):
??????? '''
??????? (1/n_cl) * sum_i(n_ii / (t_i + sum_j(n_ji) - n_ii))
??????? '''
??? ?
??????? check_size(eval_segm, gt_segm)
??? ?
??????? cl, n_cl?? = union_classes(eval_segm, gt_segm)
??????? _, n_cl_gt = extract_classes(gt_segm)
??????? eval_mask, gt_mask = extract_both_masks(eval_segm, gt_segm, cl, n_cl)
??? ?
??????? IU = list([0]) * n_cl
??? ?
??????? for i, c in enumerate(cl):
??????????? curr_eval_mask = eval_mask[i, :, :]
??????????? curr_gt_mask = gt_mask[i, :, :]
??? ?
??????????? if (np.sum(curr_eval_mask) == 0) or (np.sum(curr_gt_mask) == 0):
??????????????? continue
??? ?
??????????? n_ii = np.sum(np.logical_and(curr_eval_mask, curr_gt_mask))
??????????? t_i? = np.sum(curr_gt_mask)
??????????? n_ij = np.sum(curr_eval_mask)
??? ?
??????????? IU[i] = n_ii / (t_i + n_ij - n_ii)
??? ?
??????? mean_IU_ = np.sum(IU) / n_cl_gt
??????? return mean_IU_
??? ?
??? def frequency_weighted_IU(eval_segm, gt_segm):
??????? '''
??????? sum_k(t_k)^(-1) * sum_i((t_i*n_ii)/(t_i + sum_j(n_ji) - n_ii))
??????? '''
??? ?
??????? check_size(eval_segm, gt_segm)
??? ?
??????? cl, n_cl = union_classes(eval_segm, gt_segm)
??????? eval_mask, gt_mask = extract_both_masks(eval_segm, gt_segm, cl, n_cl)
??? ?
??????? frequency_weighted_IU_ = list([0]) * n_cl
??? ?
??????? for i, c in enumerate(cl):
??????????? curr_eval_mask = eval_mask[i, :, :]
??????????? curr_gt_mask = gt_mask[i, :, :]
??? ?
??????????? if (np.sum(curr_eval_mask) == 0) or (np.sum(curr_gt_mask) == 0):
??????????????? continue
??? ?
??????????? n_ii = np.sum(np.logical_and(curr_eval_mask, curr_gt_mask))
??????????? t_i? = np.sum(curr_gt_mask)
??????????? n_ij = np.sum(curr_eval_mask)
??? ?
??????????? frequency_weighted_IU_[i] = (t_i * n_ii) / (t_i + n_ij - n_ii)
??? ?
??????? sum_k_t_k = get_pixel_area(eval_segm)
?????? ?
??????? frequency_weighted_IU_ = np.sum(frequency_weighted_IU_) / sum_k_t_k
??????? return frequency_weighted_IU_
??? ?
??? '''
??? Auxiliary functions used during evaluation.
??? '''
??? def get_pixel_area(segm):
??????? return segm.shape[0] * segm.shape[1]
??? ?
??? def extract_both_masks(eval_segm, gt_segm, cl, n_cl):
??????? eval_mask = extract_masks(eval_segm, cl, n_cl)
??????? gt_mask?? = extract_masks(gt_segm, cl, n_cl)
??? ?
??????? return eval_mask, gt_mask
??? ?
??? def extract_classes(segm):
??????? cl = np.unique(segm)
??????? n_cl = len(cl)
??? ?
??????? return cl, n_cl
??? ?
??? def union_classes(eval_segm, gt_segm):
??????? eval_cl, _ = extract_classes(eval_segm)
??????? gt_cl, _?? = extract_classes(gt_segm)
??? ?
??????? cl = np.union1d(eval_cl, gt_cl)
??????? n_cl = len(cl)
??? ?
??????? return cl, n_cl
??? ?
??? def extract_masks(segm, cl, n_cl):
??????? h, w? = segm_size(segm)
??????? masks = np.zeros((n_cl, h, w))
??? ?
??????? for i, c in enumerate(cl):
??????????? masks[i, :, :] = segm == c
??? ?
??????? return masks
??? ?
??? def segm_size(segm):
??????? try:
??????????? height = segm.shape[0]
??????????? width? = segm.shape[1]
??????? except IndexError:
??????????? raise
??? ?
??????? return height, width
??? ?
??? def check_size(eval_segm, gt_segm):
??????? h_e, w_e = segm_size(eval_segm)
??????? h_g, w_g = segm_size(gt_segm)
??? ?
??????? if (h_e != h_g) or (w_e != w_g):
??????????? raise EvalSegErr("DiffDim: Different dimensions of matrices!")
??? ?
??? '''
??? Exceptions
??? '''
??? class EvalSegErr(Exception):
??????? def __init__(self, value):
??????????? self.value = value
??? ?
??????? def __str__(self):
??????????? return repr(self.value)
??? ###############now we do some eval.
??? # test image only 1 image
??? def eval_segm(preddir, gtdir):
?? ??? ?pred = io.imread(preddir, 1)
?? ??? ?gt = io.imread(gtdir, 1)
?? ??? ?pred = (pred ).astype(np.uint8)
?? ??? ?np.set_printoptions(threshold='nan')
?? ??? ?#print(pred[10:50,:])
?? ??? ?_, pred_th= cv2.threshold(pred, 0.0000000000000001, 1, cv2.THRESH_BINARY)
?? ??? ?#print(gt[10:50,:])
?? ??? ?gt = (gt).astype(np.uint8)
?? ??? ?_, gt_th= cv2.threshold(gt, 0.0000000000000001, 1, cv2.THRESH_BINARY)
?? ??? ?
?? ??? ?pixel_accu = pixel_accuracy(pred_th, gt_th)
?? ??? ?mean_accu = mean_accuracy(pred_th, gt_th)
?? ??? ?mean_iou = mean_IU(pred_th, gt_th)
?? ??? ?fw_iou = frequency_weighted_IU(pred_th, gt_th)
?? ??? ?print("pixel_accu is: ", pixel_accu)
?? ??? ?print("mean_accu is: ", mean_accu)
?? ??? ?print("mean_iou is: ",mean_iou)
?? ??? ?print("fw_iou is: ", fw_iou)
?? ??? ?return pixel_accu, mean_accu, mean_iou, fw_iou
??? # test batch image
??? def eval_batch(rootdir):
?? ??? ?res_sum = []
?? ??? ?pixel_accu = 0.0
?? ??? ?mean_accu = 0.0
?? ??? ?mean_iou = 0.0
?? ??? ?fw_iou = 0.0
?? ??? ?
?? ??? ?for i in range(16):
?? ??? ??? ?preddir = rootdir + str(i)+"predicted.png"
?? ??? ??? ?gtdir = rootdir + str(i) + "rgb_gt.png"
?? ??? ??? ?print("===============%d==================", i)
?? ??? ??? ?resperimage = eval_segm(preddir, gtdir)
?? ??? ??? ?res_sum.append(resperimage)
?? ??? ?# compute avg eval metrics?? ?
?? ??? ?print("==================avg eval seg=========================")
?? ??? ?len_res_sum = len(res_sum)
?? ??? ?for i in range(len_res_sum):
?? ??? ??? ?pixel_accu += res_sum[i][0]
?? ??? ??? ?mean_accu += res_sum[i][1]
?? ??? ??? ?mean_iou += res_sum[i][2]
?? ??? ??? ?fw_iou += res_sum[i][3]
?? ??? ?print("avg pixel_accu : ", pixel_accu / len_res_sum, "avg mean_accu : ", mean_accu / len_res_sum,\
?? ??? ?"avg mean_iou : ", mean_iou / len_res_sum, "avg fw_iou : ", fw_iou/len_res_sum)
?? ??? ?
??? # get the contours of huizibiao?? ?
??? def get_contour(imagedir, preddir):
?? ??? ?#np.set_printoptions(threshold='nan')
??? ?
?? ??? ?pred = io.imread(preddir, 1)
?? ??? ?print(pred.dtype)
?? ??? ?#print(pred[:,10:50])
?? ??? ?pred = (pred ).astype(np.uint8)
?? ??? ?#print(" ")
?? ??? ?image = io.imread(imagedir, 1) # because it is float64
?? ??? ?print(image.dtype)
?? ??? ?print(image.shape)
?? ??? ?#print(image[:,10:50])
?? ??? ?image = (image* 255).astype(np.uint8)?? ?
?? ??? ?#cv2.imwrite("image.png",image)
?? ??? ?_, pred_th= cv2.threshold(pred, 0.0000000000000001, 1, cv2.THRESH_BINARY)
?? ??? ?contours, _ = cv2.findContours(pred_th,cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
?? ??? ?
?? ??? ?pred_contours = image
?? ??? ?
?? ??? ?for i in range(len(contours)):
?? ??? ??? ?cv2.drawContours(pred_contours, contours[i], -1, (0, 255, 0), 1)
?? ??? ?
?? ??? ?return pred_contours
??? # batch test contours of huizibiao
??? def get_contour_batch(rootdir):
?? ??? ?for i in range(16):
?? ??? ??? ?preddir = rootdir + str(i)+"predicted.png"
?? ??? ??? ?imagedir = rootdir + str(i) + "image.png"
?? ??? ??? ?print("=================================", i)
?? ??? ??? ?cv2.imwrite(str(i)+"image_countours.png", get_contour(imagedir, preddir))
?? ??? ??? ?
??? if __name__ == "__main__":
?? ??? ?'''
?? ??? ?# test only one image.
?? ??? ?preddir = "/data/xxxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/test_result/3_iter7700/2/predicted.png"
?? ??? ?gtdir = "/data/xxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/test_result/3_iter7700/2/rgb_gt.png"
?? ??? ?eval_segm(preddir, gtdir)
?? ??? ?'''
?? ??? ?
?? ??? ?# test batch image
?? ??? ?#rootdir = "/data/xxxxxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/test_result/116/iter17w/"
?? ??? ?rootdir = "/data/xxxxxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/"
?? ??? ?#eval_batch(rootdir)
?? ??? ?
?? ??? ?#draw contours on the one? image
?? ??? ?#preddir = "/data/xxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/0predicted.png"
?? ??? ?#imagedir = "/data/xxxxx/caffe-segnet-cudnn/caffe-segnet-cudnn5/0image.png"
?? ??? ?#get_contour(imagedir, preddir)
?? ??? ?
?? ??? ?#test batch
?? ??? ?get_contour_batch(rootdir)

樓主最后的mean IOU 是95.19%。

至此完畢。
--------------------- ?
作者：努力努力再努力tq ?
來源：CSDN ?
原文：https://blog.csdn.net/u012426298/article/details/81386817 ?
版權聲明：本文為博主原創文章，轉載請附上博文鏈接！

?

總結

以上是生活随笔為你收集整理的语义分割 | segnet 制作自己的数据，如何训练，如何测试,如何评价的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。