前言

網上其實有好多unet的教程，但是大多不支持多波段(遙感圖像除了RGB波段還有紅外等其他波段)，多類別的話標簽做onehot編碼的時候類別顏色要手動輸入。針對這兩個問題，今天寫下這篇文字。

有問題歡迎留言評論，覺得不錯可以動動手指點個贊同&喜歡

如果我們的keras環境還沒有搭建好，請先移步我下面這個文字：

馨意：深度學習Win10_Keras_TensorFlow-GPU環境搭建?zhuanlan.zhihu.com

如果我們還沒有制作標簽，請先移步我下面這個文字：

?

馨意：利用Arcgis制作遙感圖像深度學習語義分割標簽?zhuanlan.zhihu.com

如果我們的圖像還沒有裁剪成深度學習樣本，請先移步我下面這個文字：

馨意：python遙感圖像裁剪成深度學習樣本_支持多波段?zhuanlan.zhihu.com

如果我們的樣本還沒有數據增強，請先移步我下面這個文字：

馨意：numpy實現深度學習遙感圖像語義分割數據增強(支持多波段)?zhuanlan.zhihu.com

目錄

讀取圖像數據

圖像預處理

Unet模型編寫

模型訓練

繪制loss/acc曲線圖

模型預測

遙感圖像大圖像預測

精度評定

按文件分列全部代碼

正文

1 讀取圖像數據

首先，我們要讀取圖像的像素矩陣，這里為了能支持多波段，我們利用GDAL讀取：

import gdal # 讀取圖像像素矩陣 # fileName 圖像文件名 def readTif(fileName):dataset = gdal.Open(fileName)width = dataset.RasterXSizeheight = dataset.RasterYSizeGdalImg_data = dataset.ReadAsArray(0, 0, width, height)return GdalImg_data

2 圖像預處理

讀取了圖像之后就要做預處理了：

• 對圖像進行歸一化，這里我們采用最大值歸一化，即除以最大值255(對于8bit數據來說)；
• 對標簽進行onehot編碼，即將平面的label的每類都單獨變成由0和1組成的一層。

# 數據預處理：圖像歸一化+標簽onehot編碼 # img 圖像數據 # label 標簽數據 # classNum 類別總數(含背景) # colorDict_GRAY 顏色字典 def dataPreprocess(img, label, classNum, colorDict_GRAY):# 歸一化img = img / 255.0for i in range(colorDict_GRAY.shape[0]):label[label == colorDict_GRAY[i][0]] = i# 將數據厚度擴展到classNum(包括背景)層new_label = np.zeros(label.shape + (classNum,))# 將平面的label的每類，都單獨變成一層for i in range(classNum):new_label[label == i,i] = 1 label = new_labelreturn (img, label)

我們發現函數里有個colorDict_GRAY參數，這個是我們的各類別的顏色字典，比如我們如果只有兩類的話，colorDict_GRAY = (0, 255)。如果類別多了我們還要手動輸入比較麻煩，這里我們采用程序自動獲取colorDict_GRAY：

# 獲取顏色字典 # labelFolder 標簽文件夾,之所以遍歷文件夾是因為一張標簽可能不包含所有類別顏色 # classNum 類別總數(含背景) def color_dict(labelFolder, classNum):colorDict = []# 獲取文件夾內的文件名ImageNameList = os.listdir(labelFolder)for i in range(len(ImageNameList)):ImagePath = labelFolder + "/" + ImageNameList[i]img = cv2.imread(ImagePath).astype(np.uint32)# 如果是灰度，轉成RGBif(len(img.shape) == 2):img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB).astype(np.uint32)# 為了提取唯一值，將RGB轉成一個數img_new = img[:,:,0] * 1000000 + img[:,:,1] * 1000 + img[:,:,2]unique = np.unique(img_new)# 將第i個像素矩陣的唯一值添加到colorDict中for j in range(unique.shape[0]):colorDict.append(unique[j])# 對目前i個像素矩陣里的唯一值再取唯一值colorDict = sorted(set(colorDict))# 若唯一值數目等于總類數(包括背景)ClassNum，停止遍歷剩余的圖像if(len(colorDict) == classNum):break# 存儲顏色的RGB字典，用于預測時的渲染結果colorDict_RGB = []for k in range(len(colorDict)):# 對沒有達到九位數字的結果進行左邊補零(eg:5,201,111->005,201,111)color = str(colorDict[k]).rjust(9, '0')# 前3位R,中3位G,后3位Bcolor_RGB = [int(color[0 : 3]), int(color[3 : 6]), int(color[6 : 9])]colorDict_RGB.append(color_RGB)# 轉為numpy格式colorDict_RGB = np.array(colorDict_RGB)# 存儲顏色的GRAY字典，用于預處理時的onehot編碼colorDict_GRAY = colorDict_RGB.reshape((colorDict_RGB.shape[0], 1 ,colorDict_RGB.shape[1])).astype(np.uint8)colorDict_GRAY = cv2.cvtColor(colorDict_GRAY, cv2.COLOR_BGR2GRAY)return colorDict_RGB, colorDict_GRAY

color_dict函數除了返回colorDict_GRAY，還會返回colorDict_RGB，用于預測時RGB渲染顯示。

我們利用keras.Model.fit_generator()函數進行訓練，所以我們需要一個訓練數據生成器，keras自帶的生成器不支持多波段，所以我們自己編寫實現：

# 訓練數據生成器 # batch_size 批大小 # train_image_path 訓練圖像路徑 # train_label_path 訓練標簽路徑 # classNum 類別總數(含背景) # colorDict_GRAY 顏色字典 # resize_shape resize大小 def trainGenerator(batch_size, train_image_path, train_label_path, classNum, colorDict_GRAY, resize_shape = None):imageList = os.listdir(train_image_path)labelList = os.listdir(train_label_path)img = readTif(train_image_path + "\\" + imageList[0])# GDAL讀數據是(BandNum,Width,Height)要轉換為->(Width,Height,BandNum)img = img.swapaxes(1, 0)img = img.swapaxes(1, 2)# 無限生成數據while(True):img_generator = np.zeros((batch_size, img.shape[0], img.shape[1], img.shape[2]), np.uint8)label_generator = np.zeros((batch_size, img.shape[0], img.shape[1]), np.uint8)if(resize_shape != None):img_generator = np.zeros((batch_size, resize_shape[0], resize_shape[1], resize_shape[2]), np.uint8)label_generator = np.zeros((batch_size, resize_shape[0], resize_shape[1]), np.uint8)# 隨機生成一個batch的起點rand = random.randint(0, len(imageList) - batch_size)for j in range(batch_size):img = readTif(train_image_path + "\\" + imageList[rand + j])img = img.swapaxes(1, 0)img = img.swapaxes(1, 2)# 改變圖像尺寸至特定尺寸(# 因為resize用的不多，我就用了OpenCV實現的，這個不支持多波段，需要的話可以用np進行resizeif(resize_shape != None):img = cv2.resize(img, (resize_shape[0], resize_shape[1]))img_generator[j] = imglabel = readTif(train_label_path + "\\" + labelList[rand + j]).astype(np.uint8)# 若為彩色，轉為灰度if(len(label.shape) == 3):label = label.swapaxes(1, 0)label = label.swapaxes(1, 2)label = cv2.cvtColor(label, cv2.COLOR_RGB2GRAY)if(resize_shape != None):label = cv2.resize(label, (resize_shape[0], resize_shape[1]))label_generator[j] = labelimg_generator, label_generator = dataPreprocess(img_generator, label_generator, classNum, colorDict_GRAY)yield (img_generator,label_generator)

3 Unet模型編寫

這里我們對Unet添加BN層和Dropout層，優化器選用Adam，損失函數為交叉熵函數。利用keras編寫實現：

from keras.models import Model from keras.layers import Input, BatchNormalization, Conv2D, MaxPooling2D, Dropout, concatenate, merge, UpSampling2D from keras.optimizers import Adamdef unet(pretrained_weights = None, input_size = (256, 256, 4), classNum = 2, learning_rate = 1e-5):inputs = Input(input_size)# 2D卷積層conv1 = BatchNormalization()(Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(inputs))conv1 = BatchNormalization()(Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv1))# 對于空間數據的最大池化pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)conv2 = BatchNormalization()(Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool1))conv2 = BatchNormalization()(Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv2))pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)conv3 = BatchNormalization()(Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool2))conv3 = BatchNormalization()(Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv3))pool3 = MaxPooling2D(pool_size=(2, 2))(conv3)conv4 = BatchNormalization()(Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool3))conv4 = BatchNormalization()(Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv4))# Dropout正規化，防止過擬合drop4 = Dropout(0.5)(conv4)pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)conv5 = BatchNormalization()(Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(pool4))conv5 = BatchNormalization()(Conv2D(1024, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv5))drop5 = Dropout(0.5)(conv5)# 上采樣之后再進行卷積，相當于轉置卷積操作up6 = Conv2D(512, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(drop5))try:merge6 = concatenate([drop4,up6],axis = 3)except:merge6 = merge([drop4,up6], mode = 'concat', concat_axis = 3)conv6 = BatchNormalization()(Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge6))conv6 = BatchNormalization()(Conv2D(512, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv6))up7 = Conv2D(256, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv6))try:merge7 = concatenate([conv3,up7],axis = 3)except:merge7 = merge([conv3,up7], mode = 'concat', concat_axis = 3)conv7 = BatchNormalization()(Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge7))conv7 = BatchNormalization()(Conv2D(256, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv7))up8 = Conv2D(128, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv7))try:merge8 = concatenate([conv2,up8],axis = 3)except:merge8 = merge([conv2,up8],mode = 'concat', concat_axis = 3)conv8 = BatchNormalization()(Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge8))conv8 = BatchNormalization()(Conv2D(128, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv8))up9 = Conv2D(64, 2, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(UpSampling2D(size = (2,2))(conv8))try:merge9 = concatenate([conv1,up9],axis = 3)except:merge9 = merge([conv1,up9],mode = 'concat', concat_axis = 3)conv9 = BatchNormalization()(Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(merge9))conv9 = BatchNormalization()(Conv2D(64, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9))conv9 = Conv2D(2, 3, activation = 'relu', padding = 'same', kernel_initializer = 'he_normal')(conv9)conv10 = Conv2D(classNum, 1, activation = 'softmax')(conv9)model = Model(inputs = inputs, outputs = conv10)# 用于配置訓練模型（優化器、目標函數、模型評估標準）model.compile(optimizer = Adam(lr = learning_rate), loss = 'categorical_crossentropy', metrics = ['accuracy'])# 如果有預訓練的權重if(pretrained_weights):model.load_weights(pretrained_weights)return model

改進Unet

4 模型訓練

至此，我們可以編寫模型訓練的代碼了：

''' 數據集相關參數 ''' # 訓練數據圖像路徑 train_image_path = "Data\\train\\image" # 訓練數據標簽路徑 train_label_path = "Data\\train\\label" # 驗證數據圖像路徑 validation_image_path = "Data\\validation\\image" # 驗證數據標簽路徑 validation_label_path = "Data\\validation\\label"''' 模型相關參數 ''' # 批大小 batch_size = 2 # 類的數目(包括背景) classNum = 2 # 模型輸入圖像大小 input_size = (512, 512, 3) # 訓練模型的迭代總輪數 epochs = 100 # 初始學習率 learning_rate = 1e-4 # 預訓練模型地址 premodel_path = None # 訓練模型保存地址 model_path = "Model\\unet_model.hdf5"# 訓練數據數目 train_num = len(os.listdir(train_image_path)) # 驗證數據數目 validation_num = len(os.listdir(validation_image_path)) # 訓練集每個epoch有多少個batch_size steps_per_epoch = train_num / batch_size # 驗證集每個epoch有多少個batch_size validation_steps = validation_num / batch_size # 標簽的顏色字典,用于onehot編碼 colorDict_RGB, colorDict_GRAY = color_dict(train_label_path, classNum)# 得到一個生成器，以batch_size的速率生成訓練數據 train_Generator = trainGenerator(batch_size,train_image_path, train_label_path,classNum ,colorDict_GRAY,input_size)# 得到一個生成器，以batch_size的速率生成驗證數據 validation_data = trainGenerator(batch_size,validation_image_path,validation_label_path,classNum,colorDict_GRAY,input_size) # 定義模型 model = unet(pretrained_weights = premodel_path, input_size = input_size, classNum = classNum, learning_rate = learning_rate) # 打印模型結構 model.summary() # 回調函數 model_checkpoint = ModelCheckpoint(model_path,monitor = 'loss',verbose = 1,# 日志顯示模式:0->安靜模式,1->進度條,2->每輪一行save_best_only = True)# 獲取當前時間 start_time = datetime.datetime.now()# 模型訓練 history = model.fit_generator(train_Generator,steps_per_epoch = steps_per_epoch,epochs = epochs,callbacks = [model_checkpoint],validation_data = validation_data,validation_steps = validation_steps)# 訓練總時間 end_time = datetime.datetime.now() log_time = "訓練總時間: " + str((end_time - start_time).seconds / 60) + "m" print(log_time) with open('TrainTime.txt','w') as f:f.write(log_time)

5 繪制loss/acc曲線圖

model.fit_generator()函數可以返回loss和acc數據，然后我們利用matplotlib繪制：

# 保存并繪制loss,acc acc = history.history['acc'] val_acc = history.history['val_acc'] loss = history.history['loss'] val_loss = history.history['val_loss'] book = xlwt.Workbook(encoding='utf-8', style_compression=0) sheet = book.add_sheet('test', cell_overwrite_ok=True) for i in range(len(acc)):sheet.write(i, 0, acc[i])sheet.write(i, 1, val_acc[i])sheet.write(i, 2, loss[i])sheet.write(i, 3, val_loss[i]) book.save(r'AccAndLoss.xls') epochs = range(1, len(acc) + 1) plt.plot(epochs, acc, 'r', label = 'Training acc') plt.plot(epochs, val_acc, 'b', label = 'Validation acc') plt.title('Training and validation accuracy') plt.legend() plt.savefig("accuracy.png",dpi = 300) plt.figure() plt.plot(epochs, loss, 'r', label = 'Training loss') plt.plot(epochs, val_loss, 'b', label = 'Validation loss') plt.title('Training and validation loss') plt.legend() plt.savefig("loss.png", dpi = 300) plt.show()

Training and validation loss

Training and validation accuracy

6 模型預測

模型預測時數據格式要和預測時保持一致，也需要利用生成器：

# 測試數據生成器 # test_iamge_path 測試數據路徑 # resize_shape resize大小 def testGenerator(test_iamge_path, resize_shape = None):imageList = os.listdir(test_iamge_path)for i in range(len(imageList)):img = readTif(test_iamge_path + "\\" + imageList[i])img = img.swapaxes(1, 0)img = img.swapaxes(1, 2)# 歸一化img = img / 255.0if(resize_shape != None):# 改變圖像尺寸至特定尺寸img = cv2.resize(img, (resize_shape[0], resize_shape[1]))# 將測試圖片擴展一個維度,與訓練時的輸入[batch_size,img.shape]保持一致img = np.reshape(img, (1, ) + img.shape)yield img

模型預測出的結果需要進行onehot解碼并渲染保存結果：

# 保存結果 # test_iamge_path 測試數據圖像路徑 # test_predict_path 測試數據圖像預測結果路徑 # model_predict 模型的預測結果 # color_dict 顏色詞典 def saveResult(test_image_path, test_predict_path, model_predict, color_dict, output_size):imageList = os.listdir(test_image_path)for i, img in enumerate(model_predict):channel_max = np.argmax(img, axis = -1)img_out = np.uint8(color_dict[channel_max.astype(np.uint8)])# 修改差值方式為最鄰近差值img_out = cv2.resize(img_out, (output_size[0], output_size[1]), interpolation = cv2.INTER_NEAREST)# 保存為無損壓縮pngcv2.imwrite(test_predict_path + "\\" + imageList[i][:-4] + ".png", img_out)

7 遙感圖像大圖像預測

如果將較大的待分類遙感影像直接輸入到網絡模型中會造成內存溢出，故一般將待分類圖像裁剪為一系列較小圖像分別輸入網絡進行預測，然后將預測結果按照裁剪順序拼接成一張最終結果圖像。詳見我下面這個博客：

馨意：遙感大圖像深度學習忽略邊緣（劃窗）預測?zhuanlan.zhihu.com

8 精度評定

我們使用精確率(Precision)、召回率(Recall)、F1分數(F1-Score)、交并比(Intersection-over-Union, IoU)、平均交并比(mean Intersection-over-Union, mIoU)、頻權交并比(Frequency Weighted Intersection-over-Union, FWIoU)等指標進行精度評定。詳見我下面這個博客：

馨意：遙感圖像語義分割常用精度指標及其python實現?zhuanlan.zhihu.com

?

9 全部代碼

Data
-----train
----------image
----------label
-----validation
----------image
----------label
-----test
----------image
----------label
----------predict
Model
-----seg_unet.py
-----unet_model
dataProcess.py
train.py
test.py
seg_metrics.py 原文：https://zhuanlan.zhihu.com/p/161925744

總結

以上是生活随笔為你收集整理的keras遥感图像Unet语义分割(支持多波段多类)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。