【神经网络】(17) EfficientNet 代码复现，网络解析，附Tensorflow完整代码
生活随笔
收集整理的這篇文章主要介紹了
【神经网络】(17) EfficientNet 代码复现，网络解析，附Tensorflow完整代码
小編覺得挺不錯的,現在分享給大家,幫大家做個參考.
各位同學好，今天和大家分享一下如何使用 Tensorflow 復現 EfficientNet 卷積神經網絡模型。
EfficientNet 的網絡結構和 MobileNetV3 比較相似，建議大家在學習 EfficientNet 之前先學習一下 MobileNetV3?
MobileNetV3：https://blog.csdn.net/dgvv4/article/details/123476899
EfficientNet-B7在imagenet上準確率達到了當年最高的84.3%，與之前準確率最高的GPipe相比，參數量僅為其1/8.4，推理速度提高了6.1倍。
1. 引言
（1）根據以往的經驗，增加網絡的深度能得到更加豐富、復雜的特征，并且能夠很好的應用到其他任務中。但網絡的深度過深會面臨梯度消失，訓練困難的問題。
（2）增加網絡的寬度能夠獲得更高細粒度的特征，并且也更容易訓練，但對于寬度很大而深度很淺的網絡往往很難學習到更深層次的特征。
（3）增加輸入網絡的圖像分辨率能夠潛在的獲得更高細粒度的特征模板，但對于非常高的輸入分辨率，準確率的增益也會減小。并且大分辨率的圖像會增加計算量。
論文中就研究了如果同時增加網絡的寬度、深度、分辨率，那會有什么樣的效果。如下圖所示，紅色曲線就是同時增加網絡的深度、寬度和分辨率，網絡效果明顯提高。
2. 網絡核心模塊
網絡的核心模塊大體上和MobileNetV3相似，這里再簡單復習一下
2.1?深度可分離卷積
MobileNetV1 中主要使用了深度可分離卷積模塊，大大減少了參數量和計算量。
普通卷積是一個卷積核處理所有的通道，輸入特征圖有多少個通道，卷積核就有幾個通道，一個卷積核生成一張特征圖。
深度可分離卷積 可理解為 深度卷積 + 逐點卷積
深度卷積只處理長寬方向的空間信息；逐點卷積只處理跨通道方向的信息。能大大減少參數量，提高計算效率
深度卷積： 一個卷積核只處理一個通道，即每個卷積核只處理自己對應的通道。輸入特征圖有多少個通道就有多少個卷積核。將每個卷積核處理后的特征圖堆疊在一起。輸入和輸出特征圖的通道數相同。
由于只處理長寬方向的信息會導致丟失跨通道信息，為了將跨通道的信息補充回來，需要進行逐點卷積。
逐點卷積： 是使用1x1卷積對跨通道維度處理，有多少個1x1卷積核就會生成多少個特征圖。
2.2 逆轉殘差模塊
逆轉殘差模塊流程如下。輸入圖像，先使用1x1卷積提升通道數；然后在高維空間下使用深度卷積；再使用1x1卷積下降通道數，降維時采用線性激活函數(y=x)。當步長等于1且輸入和輸出特征圖的shape相同時，使用殘差連接輸入和輸出；當步長=2（下采樣階段）直接輸出降維后的特征圖。
對比 ResNet 的殘差結構。輸入圖像，先使用1x1卷積下降通道數；然后在低維空間下使用標準卷積，再使用1x1卷積上升通道數，激活函數都是ReLU函數。當步長等于1且輸入和輸出特征圖的shape相同時，使用殘差連接輸入和輸出；當步長=2（下采樣階段）直接輸出降維后的特征圖。
2.3 SE注意力機制
（1）先將特征圖進行全局平均池化，特征圖有多少個通道，那么池化結果（一維向量）就有多少個元素，[h, w, c]==>[None, c]。
（2）然后經過兩個全連接層得到輸出向量。在EfficientNet中，第一個全連接層降維，輸出通道數等于該逆轉殘差模塊的輸入圖像的通道數的1/4；第二個全連接層升維，輸出通道數等于全局平均池化前的特征圖的通道數。
（3）全連接層的輸出向量可理解為，向量的每個元素是對每張特征圖進行分析得出的權重關系。比較重要的特征圖就會賦予更大的權重，即該特征圖對應的向量元素的值較大。反之，不太重要的特征圖對應的權重值較小。
（4）經過兩個全連接層得到一個由channel個元素組成的向量，每個元素是針對每個通道的權重，將權重和原特征圖的像素值對應相乘，得到新的特征圖數據
以下圖為例，特征圖經過兩個全連接層之后，比較重要的特征圖對應的向量元素的值就較大。將得到的權重和對應特征圖中的所有元素相乘，得到新的輸出特征圖
2.4 總體流程
基本模塊(stride=1)：圖像輸入，先經過1x1卷積上升通道數；然后在高緯空間下使用深度卷積；再經過SE注意力機制優化特征圖數據；再經過1x1卷積下降通道數（使用線性激活函數）；若此時輸入特征圖的shape和輸出特征圖的shape相同，那么對1x1卷積降維后的特征圖加一個Dropout層，防止過擬合；最后殘差連接輸入和輸出
下采樣模塊(stride=2)：大致流程和基本模塊相同，不采用Dropout層和殘差連接，1x1卷積降維后直接輸出特征圖。
3. 代碼復現
3.1 網絡架構圖
以 EfficientNet-B0 為例，網絡結構如下圖所示。
3.2 EfficientNet 系列網絡參數
（1）width_coefficient 代表通道維度上的倍率因子。比如，在EfficientNet-B0中Stage1的3x3卷積層使用的卷積核個數是32個，那么EfficientNet-B6中Stage1的3x3卷積層使用卷積核個數是 32*1.8=57.6，取整到離57.6最近的8的倍數，即56
（2）depth_coefficient 代表深度維度上的倍率因子。比如，在EfficientNet-B0中Stage7的layers=4，即該模塊重復4次。那么在EfficientNet-B6中Stage7的layers=4*2.6=10.4，向上取整為11。
（3）dropout_rate 代表Dropout的隨機殺死神經元的概率
'''
Model | input_size | width_coefficient | depth_coefficient | dropout_rate
-------------------------------------------------------------------------------------------
EfficientNetB0 | 224x224 | 1.0 | 1.0 | 0.2
-------------------------------------------------------------------------------------------
EfficientNetB1 | 240x240 | 1.0 | 1.1 | 0.2
-------------------------------------------------------------------------------------------
EfficientNetB2 | 260x260 | 1.1 | 1.2 | 0.3
-------------------------------------------------------------------------------------------
EfficientNetB3 | 300x300 | 1.2 | 1.4 | 0.3
-------------------------------------------------------------------------------------------
EfficientNetB4 | 380x380 | 1.4 | 1.8 | 0.4
-------------------------------------------------------------------------------------------
EfficientNetB5 | 456x456 | 1.6 | 2.2 | 0.4
-------------------------------------------------------------------------------------------
EfficientNetB6 | 528x528 | 1.8 | 2.6 | 0.5
-------------------------------------------------------------------------------------------
EfficientNetB7 | 600x600 | 2.0 | 3.1 | 0.5
'''3.3 網絡核心模塊代碼
（1）標準卷積塊
一個標準卷積塊由 普通卷積+批標準化+激活函數 組成
#（1）激活函數
def swish(x):# swish激活函數x = x*tf.nn.sigmoid(x)return x#（2）標準卷積
def conv_block(input_tensor, filters, kernel_size, stride, activation=True):# 普通卷積+標準化+激活x = layers.Conv2D(filters = filters, # 輸出特征圖個數 kernel_size = kernel_size, # 卷積核sizestrides = stride, # 步長=2，size長寬減半use_bias = False)(input_tensor) # 有BN層就不要偏置x = layers.BatchNormalization()(x) # 批標準化if activation: # 判斷是否需要使用激活函數x = swish(x) # 激活函數return x（2）SE注意力機制
為了減少計算量，SE注意力機制中的全連接層可以換成1*1卷積層。這里要注意，第一個卷積層降維的通道數，是MBConv模塊的輸入特征圖通道數的1/4，也就是在逆轉殘差模塊中1*1卷積升維之前的特征圖通道數的1/4
#（3）SE注意力機制
def squeeze_excitation(input_tensor, inputs_channel):squeeze = inputs_channel / 4 # 通道數下降為輸入該MBConv的特征圖的1/4excitation = input_tensor.shape[-1] # 通道數上升為深度卷積的輸出特征圖個數# 全局平均池化 [h,w,c]==>[None,c]x = layers.GlobalAveragePooling2D()(input_tensor)# [None,c]==>[1,1,c]x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)# 1*1卷積降維，通道數變為輸入MBblock模塊的圖像的通道數的1/4x = layers.Conv2D(filters = squeeze, kernel_size = (1,1), strides = 1,padding = 'same')(x)x = swish(x) # swish激活函數# 1*1卷積升維，通道數變為深度卷積的輸出特征圖個數x = layers.Conv2D(filters = excitation, kernel_size = (1,1),strides = 1,padding = 'same')(x)x = tf.nn.sigmoid(x) # sigmoid激活函數# 將深度卷積的輸入特征圖的每個通道和SE得到的針對每個通道的權重相乘x = layers.multiply([input_tensor, x])return x（3）逆轉殘差模塊
以基本模塊為例(stride=1)。如果需要提升特征圖的通道數，那么先經過1x1卷積上升通道數；然后在高緯空間下使用深度卷積；再經過SE注意力機制優化特征圖數據；再經過1x1卷積下降通道數（使用線性激活函數，y=x）；若此時輸入特征圖的shape和輸出特征圖的shape相同，那么對1x1卷積降維后的特征圖加一個Dropout層，防止過擬合；最后殘差連接輸入和輸出。
如第2.4小節所示。
#（4）逆轉殘差模塊
def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):'''expansion代表第一個1*1卷積上升的通道數是輸入圖像通道數的expansion倍out_channel代表MBConv模塊輸出通道數個數，即第二個1*1卷積的卷積核個數dropout_rate代表dropout層隨機殺死神經元的概率'''# 殘差邊residual = x# 輸入的特征圖的通道數in_channel = x.shape[-1]# ① 若expansion==1，1*1卷積升維就不用執行if expansion != 1:# 調用自定義的1*1標準卷積x = conv_block(x, filters=in_channel*expansion, # 通道數上升expansion倍kernel_size=(1,1), stride=1, activation=True)# ② 深度卷積x = layers.DepthwiseConv2D(kernel_size = kernel_size,strides = stride, # 步長=2下采樣padding = 'same', # 下采樣時，特征圖長寬減半use_bias = False)(x) # 有BN層就不用偏置x = layers.BatchNormalization()(x) # 批標準化 x = swish(x) # swish激活# ③ SE注意力機制，傳入深度卷積輸出的tensor，和輸入至MBConv模塊的特征圖通道數x = squeeze_excitation(x, inputs_channel=in_channel)# ④ 1*1卷積上升通道數，使用線性激活，即卷積+BNx = conv_block(input_tensor = x, filters = out_channel, # 1*1卷積輸出通道數就是MBConv模塊輸出通道數 kernel_size=(1,1), stride=1,activation = False)# ⑤ 只有使用殘差連接，并且dropout_rate>0時才會使用Dropout層if stride == 1 and residual.shape == x.shape:# 判斷dropout_rate是否大于0if dropout_rate > 0:x = layers.Dropout(rate = dropout_rate)(x)# 殘差連接輸入和輸出x = layers.Add()([residual, x])return x# 如果步長=2，直接輸出1*1降維的結果return x3.4 完整代碼
以EfficientNet-B0為例，展示代碼，如果需要使用其他EfficientNet系列的網絡，只需要在主函數中（第9步）修改參數即可。
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import Model, layers
import math#（1）激活函數
def swish(x):# swish激活函數x = x*tf.nn.sigmoid(x)return x#（2）標準卷積
def conv_block(input_tensor, filters, kernel_size, stride, activation=True):# 普通卷積+標準化+激活x = layers.Conv2D(filters = filters, # 輸出特征圖個數 kernel_size = kernel_size, # 卷積核sizestrides = stride, # 步長=2，size長寬減半use_bias = False)(input_tensor) # 有BN層就不要偏置x = layers.BatchNormalization()(x) # 批標準化if activation: # 判斷是否需要使用激活函數x = swish(x) # 激活函數return x#（3）SE注意力機制
def squeeze_excitation(input_tensor, inputs_channel):squeeze = inputs_channel / 4 # 通道數下降為輸入該MBConv的特征圖的1/4excitation = input_tensor.shape[-1] # 通道數上升為深度卷積的輸出特征圖個數# 全局平均池化 [h,w,c]==>[None,c]x = layers.GlobalAveragePooling2D()(input_tensor)# [None,c]==>[1,1,c]x = layers.Reshape(target_shape=(1, 1, x.shape[-1]))(x)# 1*1卷積降維，通道數變為輸入MBblock模塊的圖像的通道數的1/4x = layers.Conv2D(filters = squeeze, kernel_size = (1,1), strides = 1,padding = 'same')(x)x = swish(x) # swish激活函數# 1*1卷積升維，通道數變為深度卷積的輸出特征圖個數x = layers.Conv2D(filters = excitation, kernel_size = (1,1),strides = 1,padding = 'same')(x)x = tf.nn.sigmoid(x) # sigmoid激活函數# 將深度卷積的輸入特征圖的每個通道和SE得到的針對每個通道的權重相乘x = layers.multiply([input_tensor, x])return x#（4）逆轉殘差模塊
def MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate):'''expansion代表第一個1*1卷積上升的通道數是輸入圖像通道數的expansion倍out_channel代表MBConv模塊輸出通道數個數，即第二個1*1卷積的卷積核個數dropout_rate代表dropout層隨機殺死神經元的概率'''# 殘差邊residual = x# 輸入的特征圖的通道數in_channel = x.shape[-1]# ① 若expansion==1，1*1卷積升維就不用執行if expansion != 1:# 調用自定義的1*1標準卷積x = conv_block(x, filters=in_channel*expansion, # 通道數上升expansion倍kernel_size=(1,1), stride=1, activation=True)# ② 深度卷積x = layers.DepthwiseConv2D(kernel_size = kernel_size,strides = stride, # 步長=2下采樣padding = 'same', # 下采樣時，特征圖長寬減半use_bias = False)(x) # 有BN層就不用偏置x = layers.BatchNormalization()(x) # 批標準化 x = swish(x) # swish激活# ③ SE注意力機制，傳入深度卷積輸出的tensor，和輸入至MBConv模塊的特征圖通道數x = squeeze_excitation(x, inputs_channel=in_channel)# ④ 1*1卷積上升通道數，使用線性激活，即卷積+BNx = conv_block(input_tensor = x, filters = out_channel, # 1*1卷積輸出通道數就是MBConv模塊輸出通道數 kernel_size=(1,1), stride=1,activation = False)# ⑤ 只有使用殘差連接，并且dropout_rate>0時才會使用Dropout層if stride == 1 and residual.shape == x.shape:# 判斷dropout_rate是否大于0if dropout_rate > 0:x = layers.Dropout(rate = dropout_rate)(x)# 殘差連接輸入和輸出x = layers.Add()([residual, x])return x# 如果步長=2，直接輸出1*1降維的結果return x#（5）一個stage模塊是由多個MBConv模塊組成
def stage(x, n, out_channel, expansion, kernel_size, stride, dropout_rate):# 重復執行MBConv模塊n次for _ in range(n):# 逆殘差模塊x = MBConv(x, expansion, out_channel, kernel_size, stride, dropout_rate)return x # 返回每個stage的輸出特征圖#（6）通道數乘維度因子后，取8的倍數
def round_filters(filters, width_coefficient, divisor=8):filters = filters * width_coefficient # 通道數乘寬度因子# 新的通道數是距離遠通道數最近的8的倍數new_filters = max(divisor, int(filters + divisor/2) // divisor * divisor)# if new_filters < 0.9 * filters:new_filters += filtersreturn new_filters#（7）深度乘上深度因子后，向上取整
def round_repeats(repeats, depth_coefficient):# 求得每一個卷積模塊重復執行的次數repeats = int(math.ceil(repeats * depth_coefficient)) #向上取整后小數部分=0，int()舍棄小數部分return repeats#（8）主干模型結構
def efficientnet(input_shape, classes, width_coefficient, depth_coefficient, dropout_rate):'''width_coefficient，通道維度上的倍率因子。與卷積核個數相乘，取整到離它最近的8的倍數depth_coefficient，深度維度上的倍率因子。和模塊重復次數相乘，向上取整dropout_rate，dropout層殺死神經元的概率'''# 構建輸入層inputs = keras.Input(shape=input_shape)# 標準卷積 [224,224,3]==>[112,112,32]x = conv_block(inputs, filters=round_filters(32, width_coefficient), # 維度因子改變卷積核個數kernel_size=(3,3), stride=2)# [112,112,32]==>[112,112,16]x = stage(x, n=round_repeats(1, depth_coefficient), expansion=1, out_channel=round_filters(16, width_coefficient),kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)# [112,112,16]==>[56,56,24]x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(24, width_coefficient),expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)# [56,56,24]==>[28,28,40]x = stage(x, n=round_repeats(2, depth_coefficient), out_channel=round_filters(40, width_coefficient),expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)# [28,28,40]==>[14,14,80]x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(80, width_coefficient),expansion=6, kernel_size=(3,3), stride=2, dropout_rate=dropout_rate)# [14,14,80]==>[14,14,112]x = stage(x, n=round_repeats(3, depth_coefficient), out_channel=round_filters(112, width_coefficient),expansion=6, kernel_size=(5,5), stride=1, dropout_rate=dropout_rate)# [14,14,112]==>[7,7,192]x = stage(x, n=round_repeats(4, depth_coefficient), out_channel=round_filters(192, width_coefficient),expansion=6, kernel_size=(5,5), stride=2, dropout_rate=dropout_rate)# [7,7,192]==>[7,7,320]x = stage(x, n=round_repeats(1, depth_coefficient), out_channel=round_filters(320, width_coefficient),expansion=6, kernel_size=(3,3), stride=1, dropout_rate=dropout_rate)# [7,7,320]==>[7,7,1280]x = layers.Conv2D(filters=1280, kernel_size=(1*1), strides=1,padding='same', use_bias=False)(x)x = layers.BatchNormalization()(x)x = swish(x)# [7,7,1280]==>[None,1280]x = layers.GlobalAveragePooling2D()(x)# [None,1280]==>[None,1000]x = layers.Dropout(rate=dropout_rate)(x) # 隨機殺死神經元防止過擬合logits = layers.Dense(classes)(x) # 訓練時再使用softmax# 構建模型model = Model(inputs, logits)return model#（9）接收網絡模型
if __name__ == '__main__':# 以efficientnetB0為例，輸入參數model = efficientnet(input_shape=[224,224,3], classes=1000, # 輸入圖象size，分類數width_coefficient=1.0, depth_coefficient=1.0, dropout_rate=0.2)model.summary() # 參看網絡模型結構3.5 查看網絡架構
使用model.summary()查看網絡架構，EfficientNet-B0有五百多萬參數
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, 111, 111, 32) 864 input_1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 111, 111, 32) 128 conv2d[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
tf.math.multiply (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization[0][0]tf.math.sigmoid[0][0]
__________________________________________________________________________________________________
depthwise_conv2d (DepthwiseConv (None, 111, 111, 32) 288 tf.math.multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 111, 111, 32) 128 depthwise_conv2d[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
tf.math.multiply_1 (TFOpLambda) (None, 111, 111, 32) 0 batch_normalization_1[0][0]tf.math.sigmoid_1[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 32) 0 tf.math.multiply_1[0][0]
__________________________________________________________________________________________________
reshape (Reshape) (None, 1, 1, 32) 0 global_average_pooling2d[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, 1, 1, 8) 264 reshape[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_2 (TFOpLambda) (None, 1, 1, 8) 0 conv2d_1[0][0]
__________________________________________________________________________________________________
tf.math.multiply_2 (TFOpLambda) (None, 1, 1, 8) 0 conv2d_1[0][0]tf.math.sigmoid_2[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, 1, 1, 32) 288 tf.math.multiply_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_3 (TFOpLambda) (None, 1, 1, 32) 0 conv2d_2[0][0]
__________________________________________________________________________________________________
multiply (Multiply) (None, 111, 111, 32) 0 tf.math.multiply_1[0][0]tf.math.sigmoid_3[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, 111, 111, 16) 512 multiply[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 111, 111, 16) 64 conv2d_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, 111, 111, 96) 1536 batch_normalization_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 111, 111, 96) 384 conv2d_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_4 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
tf.math.multiply_3 (TFOpLambda) (None, 111, 111, 96) 0 batch_normalization_3[0][0]tf.math.sigmoid_4[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_1 (DepthwiseCo (None, 56, 56, 96) 864 tf.math.multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 56, 56, 96) 384 depthwise_conv2d_1[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_5 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
tf.math.multiply_4 (TFOpLambda) (None, 56, 56, 96) 0 batch_normalization_4[0][0]tf.math.sigmoid_5[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_1 (Glo (None, 96) 0 tf.math.multiply_4[0][0]
__________________________________________________________________________________________________
reshape_1 (Reshape) (None, 1, 1, 96) 0 global_average_pooling2d_1[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, 1, 1, 4) 388 reshape_1[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_6 (TFOpLambda) (None, 1, 1, 4) 0 conv2d_5[0][0]
__________________________________________________________________________________________________
tf.math.multiply_5 (TFOpLambda) (None, 1, 1, 4) 0 conv2d_5[0][0]tf.math.sigmoid_6[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, 1, 1, 96) 480 tf.math.multiply_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_7 (TFOpLambda) (None, 1, 1, 96) 0 conv2d_6[0][0]
__________________________________________________________________________________________________
multiply_1 (Multiply) (None, 56, 56, 96) 0 tf.math.multiply_4[0][0]tf.math.sigmoid_7[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, 56, 56, 24) 2304 multiply_1[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 56, 56, 24) 96 conv2d_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, 56, 56, 144) 3456 batch_normalization_5[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 56, 56, 144) 576 conv2d_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_8 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
tf.math.multiply_6 (TFOpLambda) (None, 56, 56, 144) 0 batch_normalization_6[0][0]tf.math.sigmoid_8[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_2 (DepthwiseCo (None, 28, 28, 144) 1296 tf.math.multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 28, 28, 144) 576 depthwise_conv2d_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
tf.math.multiply_7 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_7[0][0]tf.math.sigmoid_9[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 144) 0 tf.math.multiply_7[0][0]
__________________________________________________________________________________________________
reshape_2 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_2[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, 1, 1, 6) 870 reshape_2[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_10 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_9[0][0]
__________________________________________________________________________________________________
tf.math.multiply_8 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_9[0][0]tf.math.sigmoid_10[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_11 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_10[0][0]
__________________________________________________________________________________________________
multiply_2 (Multiply) (None, 28, 28, 144) 0 tf.math.multiply_7[0][0]tf.math.sigmoid_11[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, 28, 28, 24) 3456 multiply_2[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 28, 28, 24) 96 conv2d_11[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, 28, 28, 144) 3456 batch_normalization_8[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, 28, 28, 144) 576 conv2d_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_12 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
tf.math.multiply_9 (TFOpLambda) (None, 28, 28, 144) 0 batch_normalization_9[0][0]tf.math.sigmoid_12[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_3 (DepthwiseCo (None, 14, 14, 144) 3600 tf.math.multiply_9[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, 14, 14, 144) 576 depthwise_conv2d_3[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_13 (TFOpLambda) (None, 14, 14, 144) 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
tf.math.multiply_10 (TFOpLambda (None, 14, 14, 144) 0 batch_normalization_10[0][0]tf.math.sigmoid_13[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_3 (Glo (None, 144) 0 tf.math.multiply_10[0][0]
__________________________________________________________________________________________________
reshape_3 (Reshape) (None, 1, 1, 144) 0 global_average_pooling2d_3[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, 1, 1, 6) 870 reshape_3[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_14 (TFOpLambda) (None, 1, 1, 6) 0 conv2d_13[0][0]
__________________________________________________________________________________________________
tf.math.multiply_11 (TFOpLambda (None, 1, 1, 6) 0 conv2d_13[0][0]tf.math.sigmoid_14[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, 1, 1, 144) 1008 tf.math.multiply_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_15 (TFOpLambda) (None, 1, 1, 144) 0 conv2d_14[0][0]
__________________________________________________________________________________________________
multiply_3 (Multiply) (None, 14, 14, 144) 0 tf.math.multiply_10[0][0]tf.math.sigmoid_15[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, 14, 14, 40) 5760 multiply_3[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, 14, 14, 40) 160 conv2d_15[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, 14, 14, 240) 9600 batch_normalization_11[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, 14, 14, 240) 960 conv2d_16[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_16 (TFOpLambda) (None, 14, 14, 240) 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
tf.math.multiply_12 (TFOpLambda (None, 14, 14, 240) 0 batch_normalization_12[0][0]tf.math.sigmoid_16[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_4 (DepthwiseCo (None, 7, 7, 240) 6000 tf.math.multiply_12[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, 7, 7, 240) 960 depthwise_conv2d_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_17 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
tf.math.multiply_13 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_13[0][0]tf.math.sigmoid_17[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_4 (Glo (None, 240) 0 tf.math.multiply_13[0][0]
__________________________________________________________________________________________________
reshape_4 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_4[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, 1, 1, 10) 2410 reshape_4[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_18 (TFOpLambda) (None, 1, 1, 10) 0 conv2d_17[0][0]
__________________________________________________________________________________________________
tf.math.multiply_14 (TFOpLambda (None, 1, 1, 10) 0 conv2d_17[0][0]tf.math.sigmoid_18[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_19 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_18[0][0]
__________________________________________________________________________________________________
multiply_4 (Multiply) (None, 7, 7, 240) 0 tf.math.multiply_13[0][0]tf.math.sigmoid_19[0][0]
__________________________________________________________________________________________________
conv2d_19 (Conv2D) (None, 7, 7, 40) 9600 multiply_4[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, 7, 7, 40) 160 conv2d_19[0][0]
__________________________________________________________________________________________________
conv2d_20 (Conv2D) (None, 7, 7, 240) 9600 batch_normalization_14[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, 7, 7, 240) 960 conv2d_20[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_20 (TFOpLambda) (None, 7, 7, 240) 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
tf.math.multiply_15 (TFOpLambda (None, 7, 7, 240) 0 batch_normalization_15[0][0]tf.math.sigmoid_20[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_5 (DepthwiseCo (None, 4, 4, 240) 2160 tf.math.multiply_15[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, 4, 4, 240) 960 depthwise_conv2d_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_21 (TFOpLambda) (None, 4, 4, 240) 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
tf.math.multiply_16 (TFOpLambda (None, 4, 4, 240) 0 batch_normalization_16[0][0]tf.math.sigmoid_21[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_5 (Glo (None, 240) 0 tf.math.multiply_16[0][0]
__________________________________________________________________________________________________
reshape_5 (Reshape) (None, 1, 1, 240) 0 global_average_pooling2d_5[0][0]
__________________________________________________________________________________________________
conv2d_21 (Conv2D) (None, 1, 1, 10) 2410 reshape_5[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_22 (TFOpLambda) (None, 1, 1, 10) 0 conv2d_21[0][0]
__________________________________________________________________________________________________
tf.math.multiply_17 (TFOpLambda (None, 1, 1, 10) 0 conv2d_21[0][0]tf.math.sigmoid_22[0][0]
__________________________________________________________________________________________________
conv2d_22 (Conv2D) (None, 1, 1, 240) 2640 tf.math.multiply_17[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_23 (TFOpLambda) (None, 1, 1, 240) 0 conv2d_22[0][0]
__________________________________________________________________________________________________
multiply_5 (Multiply) (None, 4, 4, 240) 0 tf.math.multiply_16[0][0]tf.math.sigmoid_23[0][0]
__________________________________________________________________________________________________
conv2d_23 (Conv2D) (None, 4, 4, 80) 19200 multiply_5[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, 4, 4, 80) 320 conv2d_23[0][0]
__________________________________________________________________________________________________
conv2d_24 (Conv2D) (None, 4, 4, 480) 38400 batch_normalization_17[0][0]
__________________________________________________________________________________________________
batch_normalization_18 (BatchNo (None, 4, 4, 480) 1920 conv2d_24[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_24 (TFOpLambda) (None, 4, 4, 480) 0 batch_normalization_18[0][0]
__________________________________________________________________________________________________
tf.math.multiply_18 (TFOpLambda (None, 4, 4, 480) 0 batch_normalization_18[0][0]tf.math.sigmoid_24[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_6 (DepthwiseCo (None, 2, 2, 480) 4320 tf.math.multiply_18[0][0]
__________________________________________________________________________________________________
batch_normalization_19 (BatchNo (None, 2, 2, 480) 1920 depthwise_conv2d_6[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_25 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_19[0][0]
__________________________________________________________________________________________________
tf.math.multiply_19 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_19[0][0]tf.math.sigmoid_25[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_6 (Glo (None, 480) 0 tf.math.multiply_19[0][0]
__________________________________________________________________________________________________
reshape_6 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_6[0][0]
__________________________________________________________________________________________________
conv2d_25 (Conv2D) (None, 1, 1, 20) 9620 reshape_6[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_26 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_25[0][0]
__________________________________________________________________________________________________
tf.math.multiply_20 (TFOpLambda (None, 1, 1, 20) 0 conv2d_25[0][0]tf.math.sigmoid_26[0][0]
__________________________________________________________________________________________________
conv2d_26 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_20[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_27 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_26[0][0]
__________________________________________________________________________________________________
multiply_6 (Multiply) (None, 2, 2, 480) 0 tf.math.multiply_19[0][0]tf.math.sigmoid_27[0][0]
__________________________________________________________________________________________________
conv2d_27 (Conv2D) (None, 2, 2, 80) 38400 multiply_6[0][0]
__________________________________________________________________________________________________
batch_normalization_20 (BatchNo (None, 2, 2, 80) 320 conv2d_27[0][0]
__________________________________________________________________________________________________
conv2d_28 (Conv2D) (None, 2, 2, 480) 38400 batch_normalization_20[0][0]
__________________________________________________________________________________________________
batch_normalization_21 (BatchNo (None, 2, 2, 480) 1920 conv2d_28[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_28 (TFOpLambda) (None, 2, 2, 480) 0 batch_normalization_21[0][0]
__________________________________________________________________________________________________
tf.math.multiply_21 (TFOpLambda (None, 2, 2, 480) 0 batch_normalization_21[0][0]tf.math.sigmoid_28[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_7 (DepthwiseCo (None, 1, 1, 480) 4320 tf.math.multiply_21[0][0]
__________________________________________________________________________________________________
batch_normalization_22 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_7[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_29 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_22[0][0]
__________________________________________________________________________________________________
tf.math.multiply_22 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_22[0][0]tf.math.sigmoid_29[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_7 (Glo (None, 480) 0 tf.math.multiply_22[0][0]
__________________________________________________________________________________________________
reshape_7 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_7[0][0]
__________________________________________________________________________________________________
conv2d_29 (Conv2D) (None, 1, 1, 20) 9620 reshape_7[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_30 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_29[0][0]
__________________________________________________________________________________________________
tf.math.multiply_23 (TFOpLambda (None, 1, 1, 20) 0 conv2d_29[0][0]tf.math.sigmoid_30[0][0]
__________________________________________________________________________________________________
conv2d_30 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_23[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_31 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_30[0][0]
__________________________________________________________________________________________________
multiply_7 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_22[0][0]tf.math.sigmoid_31[0][0]
__________________________________________________________________________________________________
conv2d_31 (Conv2D) (None, 1, 1, 80) 38400 multiply_7[0][0]
__________________________________________________________________________________________________
batch_normalization_23 (BatchNo (None, 1, 1, 80) 320 conv2d_31[0][0]
__________________________________________________________________________________________________
conv2d_32 (Conv2D) (None, 1, 1, 480) 38400 batch_normalization_23[0][0]
__________________________________________________________________________________________________
batch_normalization_24 (BatchNo (None, 1, 1, 480) 1920 conv2d_32[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_32 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_24[0][0]
__________________________________________________________________________________________________
tf.math.multiply_24 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_24[0][0]tf.math.sigmoid_32[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_8 (DepthwiseCo (None, 1, 1, 480) 12000 tf.math.multiply_24[0][0]
__________________________________________________________________________________________________
batch_normalization_25 (BatchNo (None, 1, 1, 480) 1920 depthwise_conv2d_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_33 (TFOpLambda) (None, 1, 1, 480) 0 batch_normalization_25[0][0]
__________________________________________________________________________________________________
tf.math.multiply_25 (TFOpLambda (None, 1, 1, 480) 0 batch_normalization_25[0][0]tf.math.sigmoid_33[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_8 (Glo (None, 480) 0 tf.math.multiply_25[0][0]
__________________________________________________________________________________________________
reshape_8 (Reshape) (None, 1, 1, 480) 0 global_average_pooling2d_8[0][0]
__________________________________________________________________________________________________
conv2d_33 (Conv2D) (None, 1, 1, 20) 9620 reshape_8[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_34 (TFOpLambda) (None, 1, 1, 20) 0 conv2d_33[0][0]
__________________________________________________________________________________________________
tf.math.multiply_26 (TFOpLambda (None, 1, 1, 20) 0 conv2d_33[0][0]tf.math.sigmoid_34[0][0]
__________________________________________________________________________________________________
conv2d_34 (Conv2D) (None, 1, 1, 480) 10080 tf.math.multiply_26[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_35 (TFOpLambda) (None, 1, 1, 480) 0 conv2d_34[0][0]
__________________________________________________________________________________________________
multiply_8 (Multiply) (None, 1, 1, 480) 0 tf.math.multiply_25[0][0]tf.math.sigmoid_35[0][0]
__________________________________________________________________________________________________
conv2d_35 (Conv2D) (None, 1, 1, 112) 53760 multiply_8[0][0]
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 1, 1, 112) 448 conv2d_35[0][0]
__________________________________________________________________________________________________
conv2d_36 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_26[0][0]
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 1, 1, 672) 2688 conv2d_36[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_36 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_27[0][0]
__________________________________________________________________________________________________
tf.math.multiply_27 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_27[0][0]tf.math.sigmoid_36[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_9 (DepthwiseCo (None, 1, 1, 672) 16800 tf.math.multiply_27[0][0]
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_9[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_37 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_28[0][0]
__________________________________________________________________________________________________
tf.math.multiply_28 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_28[0][0]tf.math.sigmoid_37[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_9 (Glo (None, 672) 0 tf.math.multiply_28[0][0]
__________________________________________________________________________________________________
reshape_9 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_9[0][0]
__________________________________________________________________________________________________
conv2d_37 (Conv2D) (None, 1, 1, 28) 18844 reshape_9[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_38 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_37[0][0]
__________________________________________________________________________________________________
tf.math.multiply_29 (TFOpLambda (None, 1, 1, 28) 0 conv2d_37[0][0]tf.math.sigmoid_38[0][0]
__________________________________________________________________________________________________
conv2d_38 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_29[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_39 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_38[0][0]
__________________________________________________________________________________________________
multiply_9 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_28[0][0]tf.math.sigmoid_39[0][0]
__________________________________________________________________________________________________
conv2d_39 (Conv2D) (None, 1, 1, 112) 75264 multiply_9[0][0]
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 1, 1, 112) 448 conv2d_39[0][0]
__________________________________________________________________________________________________
conv2d_40 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_29[0][0]
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 1, 1, 672) 2688 conv2d_40[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_40 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_30[0][0]
__________________________________________________________________________________________________
tf.math.multiply_30 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_30[0][0]tf.math.sigmoid_40[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_10 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_30[0][0]
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_10[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_41 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_31[0][0]
__________________________________________________________________________________________________
tf.math.multiply_31 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_31[0][0]tf.math.sigmoid_41[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_10 (Gl (None, 672) 0 tf.math.multiply_31[0][0]
__________________________________________________________________________________________________
reshape_10 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_10[0][0]
__________________________________________________________________________________________________
conv2d_41 (Conv2D) (None, 1, 1, 28) 18844 reshape_10[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_42 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_41[0][0]
__________________________________________________________________________________________________
tf.math.multiply_32 (TFOpLambda (None, 1, 1, 28) 0 conv2d_41[0][0]tf.math.sigmoid_42[0][0]
__________________________________________________________________________________________________
conv2d_42 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_32[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_43 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_42[0][0]
__________________________________________________________________________________________________
multiply_10 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_31[0][0]tf.math.sigmoid_43[0][0]
__________________________________________________________________________________________________
conv2d_43 (Conv2D) (None, 1, 1, 112) 75264 multiply_10[0][0]
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 1, 1, 112) 448 conv2d_43[0][0]
__________________________________________________________________________________________________
conv2d_44 (Conv2D) (None, 1, 1, 672) 75264 batch_normalization_32[0][0]
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 1, 1, 672) 2688 conv2d_44[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_44 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_33[0][0]
__________________________________________________________________________________________________
tf.math.multiply_33 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_33[0][0]tf.math.sigmoid_44[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_11 (DepthwiseC (None, 1, 1, 672) 16800 tf.math.multiply_33[0][0]
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 1, 1, 672) 2688 depthwise_conv2d_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_45 (TFOpLambda) (None, 1, 1, 672) 0 batch_normalization_34[0][0]
__________________________________________________________________________________________________
tf.math.multiply_34 (TFOpLambda (None, 1, 1, 672) 0 batch_normalization_34[0][0]tf.math.sigmoid_45[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_11 (Gl (None, 672) 0 tf.math.multiply_34[0][0]
__________________________________________________________________________________________________
reshape_11 (Reshape) (None, 1, 1, 672) 0 global_average_pooling2d_11[0][0]
__________________________________________________________________________________________________
conv2d_45 (Conv2D) (None, 1, 1, 28) 18844 reshape_11[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_46 (TFOpLambda) (None, 1, 1, 28) 0 conv2d_45[0][0]
__________________________________________________________________________________________________
tf.math.multiply_35 (TFOpLambda (None, 1, 1, 28) 0 conv2d_45[0][0]tf.math.sigmoid_46[0][0]
__________________________________________________________________________________________________
conv2d_46 (Conv2D) (None, 1, 1, 672) 19488 tf.math.multiply_35[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_47 (TFOpLambda) (None, 1, 1, 672) 0 conv2d_46[0][0]
__________________________________________________________________________________________________
multiply_11 (Multiply) (None, 1, 1, 672) 0 tf.math.multiply_34[0][0]tf.math.sigmoid_47[0][0]
__________________________________________________________________________________________________
conv2d_47 (Conv2D) (None, 1, 1, 192) 129024 multiply_11[0][0]
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 1, 1, 192) 768 conv2d_47[0][0]
__________________________________________________________________________________________________
conv2d_48 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_35[0][0]
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 1, 1, 1152) 4608 conv2d_48[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_48 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_36[0][0]
__________________________________________________________________________________________________
tf.math.multiply_36 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_36[0][0]tf.math.sigmoid_48[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_12 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_36[0][0]
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_49 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_37[0][0]
__________________________________________________________________________________________________
tf.math.multiply_37 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_37[0][0]tf.math.sigmoid_49[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_12 (Gl (None, 1152) 0 tf.math.multiply_37[0][0]
__________________________________________________________________________________________________
reshape_12 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_12[0][0]
__________________________________________________________________________________________________
conv2d_49 (Conv2D) (None, 1, 1, 48) 55344 reshape_12[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_50 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_49[0][0]
__________________________________________________________________________________________________
tf.math.multiply_38 (TFOpLambda (None, 1, 1, 48) 0 conv2d_49[0][0]tf.math.sigmoid_50[0][0]
__________________________________________________________________________________________________
conv2d_50 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_38[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_51 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_50[0][0]
__________________________________________________________________________________________________
multiply_12 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_37[0][0]tf.math.sigmoid_51[0][0]
__________________________________________________________________________________________________
conv2d_51 (Conv2D) (None, 1, 1, 192) 221184 multiply_12[0][0]
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 1, 1, 192) 768 conv2d_51[0][0]
__________________________________________________________________________________________________
conv2d_52 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_38[0][0]
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 1, 1, 1152) 4608 conv2d_52[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_52 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_39[0][0]
__________________________________________________________________________________________________
tf.math.multiply_39 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_39[0][0]tf.math.sigmoid_52[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_13 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_39[0][0]
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_13[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_53 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_40[0][0]
__________________________________________________________________________________________________
tf.math.multiply_40 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_40[0][0]tf.math.sigmoid_53[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_13 (Gl (None, 1152) 0 tf.math.multiply_40[0][0]
__________________________________________________________________________________________________
reshape_13 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_13[0][0]
__________________________________________________________________________________________________
conv2d_53 (Conv2D) (None, 1, 1, 48) 55344 reshape_13[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_54 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_53[0][0]
__________________________________________________________________________________________________
tf.math.multiply_41 (TFOpLambda (None, 1, 1, 48) 0 conv2d_53[0][0]tf.math.sigmoid_54[0][0]
__________________________________________________________________________________________________
conv2d_54 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_41[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_55 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_54[0][0]
__________________________________________________________________________________________________
multiply_13 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_40[0][0]tf.math.sigmoid_55[0][0]
__________________________________________________________________________________________________
conv2d_55 (Conv2D) (None, 1, 1, 192) 221184 multiply_13[0][0]
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 1, 1, 192) 768 conv2d_55[0][0]
__________________________________________________________________________________________________
conv2d_56 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_41[0][0]
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 1, 1, 1152) 4608 conv2d_56[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_56 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_42[0][0]
__________________________________________________________________________________________________
tf.math.multiply_42 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_42[0][0]tf.math.sigmoid_56[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_14 (DepthwiseC (None, 1, 1, 1152) 28800 tf.math.multiply_42[0][0]
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_57 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_43[0][0]
__________________________________________________________________________________________________
tf.math.multiply_43 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_43[0][0]tf.math.sigmoid_57[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_14 (Gl (None, 1152) 0 tf.math.multiply_43[0][0]
__________________________________________________________________________________________________
reshape_14 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_14[0][0]
__________________________________________________________________________________________________
conv2d_57 (Conv2D) (None, 1, 1, 48) 55344 reshape_14[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_58 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_57[0][0]
__________________________________________________________________________________________________
tf.math.multiply_44 (TFOpLambda (None, 1, 1, 48) 0 conv2d_57[0][0]tf.math.sigmoid_58[0][0]
__________________________________________________________________________________________________
conv2d_58 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_44[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_59 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_58[0][0]
__________________________________________________________________________________________________
multiply_14 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_43[0][0]tf.math.sigmoid_59[0][0]
__________________________________________________________________________________________________
conv2d_59 (Conv2D) (None, 1, 1, 192) 221184 multiply_14[0][0]
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 1, 1, 192) 768 conv2d_59[0][0]
__________________________________________________________________________________________________
conv2d_60 (Conv2D) (None, 1, 1, 1152) 221184 batch_normalization_44[0][0]
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 1, 1, 1152) 4608 conv2d_60[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_60 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_45[0][0]
__________________________________________________________________________________________________
tf.math.multiply_45 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_45[0][0]tf.math.sigmoid_60[0][0]
__________________________________________________________________________________________________
depthwise_conv2d_15 (DepthwiseC (None, 1, 1, 1152) 10368 tf.math.multiply_45[0][0]
__________________________________________________________________________________________________
batch_normalization_46 (BatchNo (None, 1, 1, 1152) 4608 depthwise_conv2d_15[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_61 (TFOpLambda) (None, 1, 1, 1152) 0 batch_normalization_46[0][0]
__________________________________________________________________________________________________
tf.math.multiply_46 (TFOpLambda (None, 1, 1, 1152) 0 batch_normalization_46[0][0]tf.math.sigmoid_61[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_15 (Gl (None, 1152) 0 tf.math.multiply_46[0][0]
__________________________________________________________________________________________________
reshape_15 (Reshape) (None, 1, 1, 1152) 0 global_average_pooling2d_15[0][0]
__________________________________________________________________________________________________
conv2d_61 (Conv2D) (None, 1, 1, 48) 55344 reshape_15[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_62 (TFOpLambda) (None, 1, 1, 48) 0 conv2d_61[0][0]
__________________________________________________________________________________________________
tf.math.multiply_47 (TFOpLambda (None, 1, 1, 48) 0 conv2d_61[0][0]tf.math.sigmoid_62[0][0]
__________________________________________________________________________________________________
conv2d_62 (Conv2D) (None, 1, 1, 1152) 56448 tf.math.multiply_47[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_63 (TFOpLambda) (None, 1, 1, 1152) 0 conv2d_62[0][0]
__________________________________________________________________________________________________
multiply_15 (Multiply) (None, 1, 1, 1152) 0 tf.math.multiply_46[0][0]tf.math.sigmoid_63[0][0]
__________________________________________________________________________________________________
conv2d_63 (Conv2D) (None, 1, 1, 320) 368640 multiply_15[0][0]
__________________________________________________________________________________________________
batch_normalization_47 (BatchNo (None, 1, 1, 320) 1280 conv2d_63[0][0]
__________________________________________________________________________________________________
conv2d_64 (Conv2D) (None, 1, 1, 1280) 409600 batch_normalization_47[0][0]
__________________________________________________________________________________________________
batch_normalization_48 (BatchNo (None, 1, 1, 1280) 5120 conv2d_64[0][0]
__________________________________________________________________________________________________
tf.math.sigmoid_64 (TFOpLambda) (None, 1, 1, 1280) 0 batch_normalization_48[0][0]
__________________________________________________________________________________________________
tf.math.multiply_48 (TFOpLambda (None, 1, 1, 1280) 0 batch_normalization_48[0][0]tf.math.sigmoid_64[0][0]
__________________________________________________________________________________________________
global_average_pooling2d_16 (Gl (None, 1280) 0 tf.math.multiply_48[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 1280) 0 global_average_pooling2d_16[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 1000) 1281000 dropout[0][0]
==================================================================================================
Total params: 5,330,564
Trainable params: 5,288,548
Non-trainable params: 42,016
__________________________________________________________________________________________________總結
以上是生活随笔為你收集整理的【神经网络】(17) EfficientNet 代码复现，网络解析，附Tensorflow完整代码的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 【深度学习理论】(3) 激活函数
- 下一篇： 【深度学习】(8) CNN中的通道注意力