目錄

1、搭建模型的流程

1）步驟

?2）完整代碼——手寫(xiě)minist數(shù)據(jù)集為例（這里使用的數(shù)據(jù)集是自帶的）

2、搭建模型的四種方法

1）方法一——利用nn.Sequential（）

2）方法二——利用collections.orderDict()

3）方法三—— 先創(chuàng)建容器類(lèi)，然后使用add_module函數(shù)向里面添加新模塊

4）方法四——利用nn.function中的函數(shù)

3、VGG16搭建

4、全卷積層實(shí)現(xiàn)方法

5、保存各個(gè)子模塊的輸出特征層

---

---

1、搭建模型的流程

1）步驟

1）先清楚需要搭建模型的構(gòu)造，確定每層的通道數(shù)、步長(zhǎng)、padding、卷積核大小等

2）創(chuàng)建模型類(lèi)，將類(lèi)繼承于nn.modules();

class VGG16(nn.Module):def __init__(self,num_classes = 1000):super(VGG16,self).__init__() # 繼承父類(lèi)屬性和方法

3）根據(jù)模型前向傳播的順序，組建好每一個(gè)子模塊；一般是用容器nn.Sequential()來(lái)存放子模塊；一般是放在模型類(lèi)的__init__()函數(shù)當(dāng)中；

self.conv1 = nn.Sequential(nn.Conv2d(3,64,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)

4）添加forward方法（）——上面__init__()方法中只是創(chuàng)建了每一個(gè)子模塊，每個(gè)模塊都是獨(dú)立的，因此在forward函數(shù)中主要是根據(jù)前向傳播的順序，將各個(gè)子模塊串起來(lái)。

# 前向傳播函數(shù)def forward(self,x):# 十三個(gè)卷積層x = self.conv1(x)x = self.conv2(x)x = self.conv3(x)x = self.conv4(x)x = self.conv5(x)x = self.conv6(x)x = self.conv7(x)x = self.conv8(x)x = self.conv9(x)x = self.conv10(x)x = self.conv11(x)x = self.conv12(x)x = self.conv13(x)x = self.conv14(x)# 將圖像扁平化為一維向量,[1,512,7,7]-->1*512*7*7output = x.view(x.size(0),-1)return output

5）到第4）步模型已經(jīng)搭建好了，接下來(lái)實(shí)例化模型后，需要確定損失函數(shù)、梯度下降優(yōu)化算法、學(xué)習(xí)率更新策略等

# 指定優(yōu)化器，即梯度下降使用的梯度下降算法，一般有sgd和adam用的多 optimizer = optim.Adam(net.parameters(), lr=0.01) # 指定學(xué)習(xí)率優(yōu)化器，即學(xué)習(xí)率的變化函數(shù) ''' milestones:分段點(diǎn) 0-75：lr 75-150:gamma*lr 150-:gamma*gamma*lr ''' scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[75, 150], gamma=0.5) # 指定損失函數(shù)，這里選擇的交叉熵?fù)p失函數(shù)來(lái)計(jì)算模型的loss criterion = nn.CrossEntropyLoss() # print(net)

6）如果數(shù)據(jù)樣本很大，則需要分批次進(jìn)行訓(xùn)練，即batch

7）訓(xùn)練

8）測(cè)試

9）保存模型

?2）完整代碼——手寫(xiě)minist數(shù)據(jù)集為例（這里使用的數(shù)據(jù)集是自帶的）

from collections import OrderedDict # OrderedDict是字典的子類(lèi)，可以記住元素的添加順序 from torch import optim from torch import nn# 構(gòu)建模型 class Net(nn.Module):def __init__(self):super(Net, self).__init__()'''一個(gè)卷積模塊一般有卷積層、激活層、池化層組成一個(gè)模型一般由多個(gè)卷積模塊和多個(gè)全連接層組成'''# input shape 28,28,3self.conv1 = nn.Sequential(OrderedDict([('conv1', nn.Conv2d(1, 16, (5, 5), (1, 1), (2, 2))), # 28,28,16('ReLU1', nn.ReLU()),('pool1', nn.MaxPool2d(2)), # 14,14,16]))self.conv2 = nn.Sequential(OrderedDict([('conv2', nn.Conv2d(16, 32, (5, 5), (1, 1), (2, 2))), # 14,14,32('ReLU2', nn.ReLU()),('pool2', nn.MaxPool2d(2)) # 7,7,32]))self.linear = nn.Linear(32 * 7 * 7, 10)def forward(self, x):x = self.conv1(x)x = self.conv2(x)x = x.view(x.size(0), -1) # 將圖像扁平化為向量，即shape=【7,7,32】轉(zhuǎn)化為shape=【1,1,7*7*32】output = self.linear(x)return output# 模型實(shí)例化后，選擇優(yōu)化器和學(xué)習(xí) net = Net() # 指定優(yōu)化器，即梯度下降使用的梯度下降算法，一般有sgd和adam用的多 optimizer = optim.Adam(net.parameters(), lr=0.01) # 指定學(xué)習(xí)率優(yōu)化器，即學(xué)習(xí)率的變化函數(shù) ''' milestones:分段點(diǎn) 0-75：lr 75-150:gamma*lr 150-:gamma*gamma*lr ''' scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[75, 150], gamma=0.5) # 指定損失函數(shù)，這里選擇的交叉熵?fù)p失函數(shù)來(lái)計(jì)算模型的loss criterion = nn.CrossEntropyLoss() # print(net)# 將數(shù)據(jù)分成指定大小的批次，然后將數(shù)據(jù)分批輸入后進(jìn)行訓(xùn)練 def iterate_minibatches(inputs, targets, batch_size, shuffle=True):assert len(inputs) == len(targets)if shuffle:indices = np.arange(len(inputs))np.random.shuffle(indices)start_idx = None# 按照步長(zhǎng)為batchsize來(lái)計(jì)算得到每一個(gè)batch的起始樣本索引號(hào)for start_idx in range(0, len(inputs) - batch_size + 1, batch_size):if shuffle:excerpt = indices[start_idx:start_idx + batch_size]else:excerpt = slice(start_idx, start_idx + batch_size) # 切片# 返回的是一個(gè)生成器對(duì)象，通過(guò)for循環(huán)可以取出，并且含有next函數(shù)yield inputs[excerpt], targets[excerpt] # 根據(jù)索引分割出了一個(gè)batch，返回對(duì)應(yīng)的特征集和標(biāo)簽集，一個(gè)生成器if start_idx is not None and start_idx + batch_size < len(inputs):excerpt = indices[start_idx + batch_size:] if shuffle else slice(start_idx + batch_size, len(inputs))yield inputs[excerpt], targets[excerpt]# 手寫(xiě)訓(xùn)練過(guò)程 # 導(dǎo)入數(shù)據(jù) import numpy as np import torch from tensorflow.examples.tutorials.mnist import input_datamnist = input_data.read_data_sets("data/MNIST_data/", one_hot=False) train_data_images = [] train_data_labels = [] test_data_images = [] test_data_labels = []X_train, y_train = mnist.train.images, mnist.train.labels # 返回的 X_train 是 numpy 下的 多維數(shù)組，(55000, 784), (55000,) X_test, y_test = mnist.test.images, mnist.test.labels # (10000, 784), (10000,) X_valid, y_valid = mnist.validation.images, mnist.validation.labels # (5000, 784),(5000, )train_data_images = np.concatenate((X_train, X_valid), axis=0) # (60000, 784) train_data_labels = np.concatenate((y_train, y_valid), axis=0) # (60000,) test_data_images = X_test # (10000, 784) test_data_labels = y_test # (10000,)# 變形 train_data_images = np.reshape(train_data_images, [-1, 1, 28, 28]) # (60000, 1, 28, 28) test_data_images = np.reshape(test_data_images, [-1, 1, 28, 28]) # (10000, 1, 28, 28)# 訓(xùn)練過(guò)程 train_loss = 0 train_correct = 0 total = 0 for epoch in range(epochs=100):# iterate_minibatches（）對(duì)train_data_images, train_data_labels批量劃分，shuffle=True，表示亂序# iterate_minibatches（）返回的是一個(gè)迭代器對(duì)象for data, target in iterate_minibatches(train_data_images, train_data_labels, batch_size=8, shuffle=True):optimizer.zero_grad() # 清除梯度output = net(data)loss = criterion(output, target) # 計(jì)算誤差loss.backward() # 后向傳播optimizer.step()train_loss += loss.item() # 計(jì)算1個(gè)epoch的loss和# 將預(yù)測(cè)得分最大的類(lèi)別作為最終的預(yù)測(cè)類(lèi)別，元組pred = torch.max(output, 1) # max函數(shù)會(huì)返回兩個(gè)tensor，第一個(gè)tensor是每行的最大值；第二個(gè)tensor是每行最大值的索引。train_correct += np.sum(pred[1] == target) # 計(jì)算1個(gè)epoch的accuracy，這里是通過(guò)判斷最大預(yù)測(cè)得分的類(lèi)別索引和真實(shí)標(biāo)簽的索引是否一致，是則表示預(yù)測(cè)成功total += target.size(0)Train_Accuracy = train_correct / total Train_Loss = train_loss# 測(cè)試過(guò)程 test_loss = 0 test_correct = 0 total = 0 for data, target in iterate_minibatches(test_data_images, test_data_labels, batch_size=8, shuffle=True):output = net(data)loss = criterion(output, target)test_loss += loss.item()pred = torch.max(output, 1)test_correct += np.sum(pred[1] == target)total += target.size(0)Test_Accuracy = test_correct / total Test_Loss = test_loss# 保存模型 model_out_path = "model.pth" torch.save(net, model_out_path) print("Checkpoint saved to {}".format(model_out_path))

2、搭建模型的四種方法

1）方法一——利用nn.Sequential（）

import torch.nn as nnclass Net(nn.Module):def __init__(self):super(Net, self).__init__()self.conv1 = nn.Sequential( # input shape (1, 28, 28)nn.Conv2d(1, 16, 5, 1, 2), # output shape (16, 28, 28)nn.ReLU(),nn.MaxPool2d(2), # output shape (16, 14, 14))self.conv2 = nn.Sequential(nn.Conv2d(16, 32, 5, 1, 2), # output shape (32, 14, 14)nn.ReLU(),nn.MaxPool2d(2), # output shape (32, 7, 7))self.linear = nn.Linear(32 * 7 * 7, 10)def forward(self, x):x = self.conv1(x)x = self.conv2(x)x = x.view(x.size(0), -1)output = self.linear(x)return output

2）方法二——利用collections.orderDict()

import torch.nn as nn from collections import OrderedDict # OrderedDict是字典的子類(lèi)，可以記住元素的添加順序class Net(nn.Module):def __init__(self):super(Net, self).__init__()self.conv1 = nn.Sequential(OrderedDict([('conv1', nn.Conv2d(1, 16, 5, 1, 2)),('ReLU1', nn.ReLU()),('pool1', nn.MaxPool2d(2)),]))self.conv2 = nn.Sequential(OrderedDict([('conv2', nn.Conv2d(16, 32, 5, 1, 2)),('ReLU2', nn.ReLU()),('pool2', nn.MaxPool2d(2)),]))self.linear = nn.Linear(32 * 7 * 7, 10)def forward(self, x):x = self.conv1(x)x = self.conv2(x)x = x.view(x.size(0), -1)output = self.linear(x)return output

3）方法三—— 先創(chuàng)建容器類(lèi)，然后使用add_module函數(shù)向里面添加新模塊

import torch.nn as nnclass Net(nn.Module):def __init__(self):super(Net, self).__init__()self.conv1 = nn.Sequential()self.conv1.add_module('conv1', nn.Conv2d(1, 16, 5, 1, 2))self.conv1.add_module('ReLU1', nn.ReLU())self.conv1.add_module('pool1', nn.MaxPool2d(2))self.conv2 = nn.Sequential()self.conv2.add_module('conv2', nn.Conv2d(16, 32, 5, 1, 2))self.conv2.add_module('ReLU2', nn.ReLU())self.conv2.add_module('pool2', nn.MaxPool2d(2))self.linear = nn.Linear(32 * 7 * 7, 10)def forward(self, x):x = self.conv1(x)x = self.conv2(x)x = x.view(x.size(0), -1)output = self.linear(x)return output

4）方法四——利用nn.function中的函數(shù)

import torch.nn as nn import torch.nn.functional as Fclass Net(nn.Module):def __init__(self):super(Net, self).__init__()self.conv1 = nn.Conv2d(1, 16, 5, 1, 2)self.conv2 = nn.Conv2d(16, 32, 5, 1, 2)self.linear = nn.Linear(32 * 7 * 7, 10)def forward(self, x):x = F.max_pool2d(F.relu(self.conv1(x)), 2)x = F.max_pool2d(F.relu(self.conv2(x)), 2)output = self.linear(x)return output

3、VGG16搭建

from torch import nnclass VGG16(nn.Module):def __init__(self,num_classes = 1000):super(VGG16,self).__init__() # 繼承父類(lèi)屬性和方法# 根據(jù)前向傳播的順序，搭建各個(gè)子網(wǎng)絡(luò)模塊## 十三個(gè)卷積層，每個(gè)卷積模塊都有卷積層、激活層和池化層，用nn.Sequential()這個(gè)容器將各個(gè)模塊存放起來(lái)# [1,3,224,224]self.conv1 = nn.Sequential(nn.Conv2d(3,64,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,64,224,224]self.conv2 = nn.Sequential(nn.Conv2d(64,64,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,64,112,112]self.conv3 = nn.Sequential(nn.Conv2d(64,128,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,128,112,112]self.conv4 = nn.Sequential(nn.Conv2d(128,128,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,128,56,56]self.conv5 = nn.Sequential(nn.Conv2d(128,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,256,56,56]self.conv6 = nn.Sequential(nn.Conv2d(256,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,256,56,56]self.conv7 = nn.Sequential(nn.Conv2d(256,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,256,28,28]self.conv8 = nn.Sequential(nn.Conv2d(256,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True))# [1,512,28,28]self.conv9 = nn.Sequential(nn.Conv2d(512,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True))# [1,512,28,28]self.conv10 = nn.Sequential(nn.Conv2d(512,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True),nn.MaxPool2d((2,2),(2,2)))# [1,512,14,14]self.conv11 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),)# [1,512,14,14]self.conv12 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),)# [1,512,14,14]-->[1,512,7,7]self.conv13 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),nn.MaxPool2d((2, 2), (2, 2)))# 三個(gè)全連接層，每個(gè)全連接層之間存在激活層和dropout層self.classfier = nn.Sequential(# [1*512*7*7]nn.Linear(1*512*7*7,4096),nn.ReLU(True),nn.Dropout(),# 4096nn.Linear(4096,4096),nn.ReLU(True),nn.Dropout(),# 4096-->1000nn.Linear(4096,num_classes))# 前向傳播函數(shù)def forward(self,x):# 十三個(gè)卷積層x = self.conv1(x)x = self.conv2(x)x = self.conv3(x)x = self.conv4(x)x = self.conv5(x)x = self.conv6(x)x = self.conv7(x)x = self.conv8(x)x = self.conv9(x)x = self.conv10(x)x = self.conv11(x)x = self.conv12(x)x = self.conv13(x)# 將圖像扁平化為一維向量,[1,512,7,7]-->1*512*7*7x = x.view(x.size(0),-1)# 三個(gè)全連接層output = self.classfier(x)return output## 測(cè)試 import torch vgg16 = VGG16(21) print(vgg16)input_ = torch.randn(1,3,224,224) output = vgg16(input_) print(output.shape) print(output)

4、全卷積層實(shí)現(xiàn)方法

核心思想：其實(shí)就是將全連接層用卷積層去替換了，一般需要經(jīng)過(guò)精心的設(shè)計(jì)，使得最后輸出的是【1,1，channels】的shape。這里以vgg16為例，vgg16最后的特征圖大小為【1,512,7,7】，若要變?yōu)?,1大小的特征圖，則可以使用7,7的卷積核進(jìn)行卷積，然后利用num_classes個(gè)卷積核去進(jìn)行卷積，最后就得到了特征圖【1，num_classes，1,1】，在輸出前使用激活函數(shù)得到分類(lèi)得分。

# 全卷積層self.conv14 = nn.Sequential(nn.Conv2d(512,num_classes,(7,7),(1,1)),nn.ReLU(inplace=True)) from torch import nnclass VGG16(nn.Module):def __init__(self,num_classes = 1000):super(VGG16,self).__init__() # 繼承父類(lèi)屬性和方法# 根據(jù)前向傳播的順序，搭建各個(gè)子網(wǎng)絡(luò)模塊## 十三個(gè)卷積層，每個(gè)卷積模塊都有卷積層、激活層和池化層，用nn.Sequential()這個(gè)容器將各個(gè)模塊存放起來(lái)# [1,3,224,224]self.conv1 = nn.Sequential(nn.Conv2d(3,64,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,64,224,224]self.conv2 = nn.Sequential(nn.Conv2d(64,64,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,64,112,112]self.conv3 = nn.Sequential(nn.Conv2d(64,128,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,128,112,112]self.conv4 = nn.Sequential(nn.Conv2d(128,128,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,128,56,56]self.conv5 = nn.Sequential(nn.Conv2d(128,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,256,56,56]self.conv6 = nn.Sequential(nn.Conv2d(256,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算)# [1,256,56,56]self.conv7 = nn.Sequential(nn.Conv2d(256,256,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True), # inplace = True表示是否進(jìn)行覆蓋計(jì)算nn.MaxPool2d((2,2),(2,2)))# [1,256,28,28]self.conv8 = nn.Sequential(nn.Conv2d(256,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True))# [1,512,28,28]self.conv9 = nn.Sequential(nn.Conv2d(512,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True))# [1,512,28,28]self.conv10 = nn.Sequential(nn.Conv2d(512,512,(3,3),(1,1),(1,1)),nn.ReLU(inplace=True),nn.MaxPool2d((2,2),(2,2)))# [1,512,14,14]self.conv11 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),)# [1,512,14,14]self.conv12 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),)# [1,512,14,14]-->[1,512,7,7]self.conv13 = nn.Sequential(nn.Conv2d(512, 512, (3, 3), (1, 1), (1, 1)),nn.ReLU(inplace=True),nn.MaxPool2d((2, 2), (2, 2)))# 全卷積層self.conv14 = nn.Sequential(nn.Conv2d(512,num_classes,(7,7),(1,1)),nn.ReLU(inplace=True))# 前向傳播函數(shù)def forward(self,x):# 十三個(gè)卷積層x = self.conv1(x)x = self.conv2(x)x = self.conv3(x)x = self.conv4(x)x = self.conv5(x)x = self.conv6(x)x = self.conv7(x)x = self.conv8(x)x = self.conv9(x)x = self.conv10(x)x = self.conv11(x)x = self.conv12(x)x = self.conv13(x)x = self.conv14(x)# 將圖像扁平化為一維向量,[1,512,7,7]-->1*512*7*7output = x.view(x.size(0),-1)return output## 測(cè)試 import torch vgg16 = VGG16(21) print(vgg16)input_ = torch.randn(1,3,224,224) output = vgg16(input_) print(output.shape) print(output)

5、保存各個(gè)子模塊的輸出特征層

?在forward函數(shù)中，將需要保存的特征層的輸出保存在列表中即可，這里以ssd中的為例，其中feathers就是將需要的幾個(gè)特征圖保存了起來(lái)，便于后續(xù)進(jìn)行特征圖訓(xùn)練，實(shí)現(xiàn)多尺度的訓(xùn)練。

def forward(self, x):features = []for i in range(23):x = self.vgg[i](x)s = self.l2_norm(x) # Conv4_3 L2 normalizationfeatures.append(s)# apply vgg up to fc7for i in range(23, len(self.vgg)):x = self.vgg[i](x)features.append(x)for k, v in enumerate(self.extras):x = F.relu(v(x), inplace=True)if k % 2 == 1:features.append(x)return tuple(features)

總結(jié)

以上是生活随笔為你收集整理的【深度学习】——pytorch搭建模型及相关模型的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。