網(wǎng)絡(luò)中的網(wǎng)絡(luò)（NiN）

LeNet、AlexNet和VGG在設(shè)計(jì)上的共同之處是：先以由卷積層構(gòu)成的模塊充分抽取空間特征，再以由全連接層構(gòu)成的模塊來(lái)輸出分類結(jié)果。其中，AlexNet和VGG對(duì)LeNet的改進(jìn)主要在于如何對(duì)這兩個(gè)模塊加寬（增加通道數(shù)）和加深。

網(wǎng)絡(luò)中的網(wǎng)絡(luò)（NiN）提出了另外一個(gè)思路，即串聯(lián)多個(gè)由卷積層和“全連接”層構(gòu)成的小網(wǎng)絡(luò)來(lái)構(gòu)建一個(gè)深層網(wǎng)絡(luò)。

NiN塊

卷積層的輸入和輸出通常是四維數(shù)組（樣本，通道，高，寬），而全連接層的輸入和輸出則通常是二維數(shù)組（樣本，特征）。如果想在全連接層后再接上卷積層，則需要將全連接層的輸出變換為四維。NiN使用$1\times 1$卷積層來(lái)替代全連接層，從而使空間信息能夠自然傳遞到后面的層中去。

NiN塊是NiN中的基礎(chǔ)塊。它由一個(gè)卷積層加兩個(gè)充當(dāng)全連接層的$1\times 1$卷積層串聯(lián)而成。其中第一個(gè)卷積層的超參數(shù)可以自行設(shè)置，而第二和第三個(gè)卷積層的超參數(shù)一般是固定的。

import time import torch from torch import nn, optimdevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')def nin_block(in_channels, out_channels, kernel_size, stride, padding):blk = nn.Sequential(nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding),nn.ReLU(),nn.Conv2d(out_channels, out_channels, kernel_size=1),nn.ReLU(),nn.Conv2d(out_channels, out_channels, kernel_size=1),nn.ReLU())return blk

NiN模型

NiN使用卷積窗口形狀分別為$11\times 11$、$5\times 5$和$3\times 3$的卷積層，每個(gè)NiN塊后接一個(gè)步幅為2、窗口形狀為$3\times 3$的最大池化層。

NiN去掉了AlexNet最后的3個(gè)全連接層，取而代之使用了輸出通道數(shù)等于標(biāo)簽類別數(shù)的NiN塊，然后使用全局平均池化層對(duì)每個(gè)通道中所有元素求平均并直接用于分類。(這里的全局平均池化層即窗口形狀等于輸入空間維形狀的平均池化層).

NiN的這個(gè)設(shè)計(jì)的好處是可以顯著減小模型參數(shù)尺寸，從而緩解過(guò)擬合。然而，該設(shè)計(jì)有時(shí)會(huì)造成獲得有效模型的訓(xùn)練時(shí)間的增加。

import torch.nn.functional as F class GlobalAvgPool2d(nn.Module):# 全局平均池化層可通過(guò)將池化窗口形狀設(shè)置成輸入的高和寬實(shí)現(xiàn)def __init__(self):super(GlobalAvgPool2d, self).__init__()def forward(self, x):return F.avg_pool2d(x, kernel_size=x.size()[2:])class FlattenLayer(torch.nn.Module):def __init__(self):super(FlattenLayer, self).__init__()def forward(self, x): # x shape: (batch, *, *, ...)return x.view(x.shape[0], -1)net = nn.Sequential(nin_block(1, 96, kernel_size=11, stride=4, padding=0),nn.MaxPool2d(kernel_size=3, stride=2),nin_block(96, 256, kernel_size=5, stride=1, padding=2),nn.MaxPool2d(kernel_size=3, stride=2),nin_block(256, 384, kernel_size=3, stride=1, padding=1),nn.MaxPool2d(kernel_size=3, stride=2), nn.Dropout(0.5),# 標(biāo)簽類別數(shù)是10nin_block(384, 10, kernel_size=3, stride=1, padding=1),GlobalAvgPool2d(), # 將四維的輸出轉(zhuǎn)成二維的輸出，其形狀為(批量大小, 10)FlattenLayer())

簡(jiǎn)單說(shuō)網(wǎng)絡(luò)結(jié)構(gòu)就是：
$$\begin{gathered} \begin{array}{c}卷積\\ (96個(gè)11?11的核)\\ (步長(zhǎng)為4)\\ (padding為0)\end{array}\to \begin{array}{c}卷積\\ (96個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \begin{array}{c}卷積\\ (96個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \\ . \\ \begin{array}{c}降采樣(最大池化)\\ (3?3的核，步長(zhǎng)2)\end{array}\to \\ . \\ \begin{array}{c}卷積\\ (256個(gè)5?5的核)\\ (步長(zhǎng)為1)\\ (padding為2)\end{array}\to \begin{array}{c}卷積\\ (256個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \begin{array}{c}卷積\\ (256個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \\ . \\ \begin{array}{c}降采樣(最大池化)\\ (3?3的核，步長(zhǎng)2)\end{array}\to \\ . \\ \begin{array}{c}卷積\\ (384個(gè)3?3的核)\\ (步長(zhǎng)為1)\\ (padding為1)\end{array}\to \begin{array}{c}卷積\\ (384個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \begin{array}{c}卷積\\ (384個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \\ . \\ \begin{array}{c}降采樣(最大池化)\\ (3?3的核，步長(zhǎng)2)\end{array}\to \\ . \\ \begin{array}{c}卷積\\ (10個(gè)3?3的核)\\ (步長(zhǎng)為1)\\ (padding為1)\end{array}\to \begin{array}{c}卷積\\ (10個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \begin{array}{c}卷積\\ (10個(gè)1?1的核)\\ (步長(zhǎng)為1)\end{array}\to \\ . \\ \begin{array}{c}降采樣(平均池化)\\ (x.size的核，步長(zhǎng)1)\end{array} \end{gathered}$$

構(gòu)建一個(gè)數(shù)據(jù)樣本來(lái)查看每一層的輸出形狀。

X = torch.rand(1, 1, 224, 224) for name, blk in net.named_children(): X = blk(X)print(name, 'output shape: ', X.shape) 0 output shape: torch.Size([1, 96, 54, 54]) 1 output shape: torch.Size([1, 96, 26, 26]) 2 output shape: torch.Size([1, 256, 26, 26]) 3 output shape: torch.Size([1, 256, 12, 12]) 4 output shape: torch.Size([1, 384, 12, 12]) 5 output shape: torch.Size([1, 384, 5, 5]) 6 output shape: torch.Size([1, 384, 5, 5]) 7 output shape: torch.Size([1, 10, 5, 5]) 8 output shape: torch.Size([1, 10, 1, 1]) 9 output shape: torch.Size([1, 10])

訓(xùn)練：

def train(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs):net = net.to(device)print("training on ", device)loss = torch.nn.CrossEntropyLoss()for epoch in range(num_epochs):train_l_sum, train_acc_sum, n, batch_count, start = 0.0, 0.0, 0, 0, time.time()for X, y in train_iter:X = X.to(device)y = y.to(device)y_hat = net(X)l = loss(y_hat, y)optimizer.zero_grad()l.backward()optimizer.step()train_l_sum += l.cpu().item()train_acc_sum += (y_hat.argmax(dim=1) == y).sum().cpu().item()n += y.shape[0]batch_count += 1test_acc = evaluate_accuracy(test_iter, net)print('epoch %d, loss %.4f, train acc %.3f, test acc %.3f, time %.1f sec'% (epoch + 1, train_l_sum / batch_count, train_acc_sum / n, test_acc, time.time() - start)) batch_size = 128 # 如出現(xiàn)“out of memory”的報(bào)錯(cuò)信息，可減小batch_size或resize train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)lr, num_epochs = 0.002, 5 optimizer = torch.optim.Adam(net.parameters(), lr=lr) train(net, train_iter, test_iter, batch_size, optimizer, device, num_epochs)

總結(jié)

以上是生活随笔為你收集整理的（pytorch-深度学习系列）网络中的网络（NiN）的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。