InceptionV4
目录
在下面的结构图中，每一个inception模块中都有一个1∗1的没有激活层的卷积层，用来扩展通道数，从而补偿因为inception模块导致的维度约间。其中Inception-ResNet-V1的结果与Inception v3相当；Inception-ResNet-V1与Inception v4结果差不多，不过实际过程中Inception v4会明显慢于Inception-ResNet-v2，这也许是因为层数太多了。且在Inception-ResNet结构中，只在传统层的上面使用BN层，而不在合并层上使用BN，虽然处处使用BN是有好处，不过更希望能够将一个完整的组件放入单独的GPU中。因为具有大量激活单元的层会占用过多的显存，所以希望这些地方丢弃BN，从而总体增加Inception模块的数量。使得不需要去解决计算资源和模块什么的权衡问题。
1. inception v4
**图1.1 inception v4 网络结构图**
**图1.2 图1.1的stem和Inception-A部分结构图**
**图1.3 图1.1的Reduction-A和Inception-B部分结构图**
**图1.4 图1.1的Reduction-B和Inception-C部分结构图**
2. Inception-resnet-v1 & Inception-resnet-v2
**图2.1 Inception-resnet-v1 & Inception-resnet-v2的结构图**
2.1 Inception-resnet-v1的组成模块
**图2.1.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.1.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.1.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
2.2 Inception-resnet-v2的组成模块
**图2.2.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.2.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.2.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
3. 模型训练
在上述的Inception V4，Inception-Resnet-V1，Inception-ResNet-v2这三个模型中都用到了Reduction-A，他们各自的具体参数如下：
图3.1 不同模型下Reduction-A的模型超参数
作者们在训练的过程中发现，如果通道数超过1000，那么Inception-resnet等网络都会开始变得不稳定，并且过早的就“死掉了”，即在迭代几万次之后，平均池化的前面一层就会生成很多的0值。作者们通过调低学习率，增加BN都没有任何改善。
不过他们发现如果在将残差汇入之前，对残差进行缩小，可以让模型稳定训练，值通常选择[0,1.0.3]，如图3.2
**图3.2 对inception-resnet模块进行最后输出值的等比例缩小**
同样的在ResNet-v1中，何凯明等人也在cifar-10中发现了模型的不稳定现象：即在特别深的网络基础上去训cifar-10，需要先以0.01的学习率去训练，然后在以0.1的学习率训练。
不过这里的作者们认为如果通道数特别多的话，即使以特别低的学习率（0.00001）训练也无法让模型收敛，如果之后再用大学习率，那么就会轻松的破坏掉之前的成果。然而简单的缩小残差的输出值有助于学习的稳定，即使进行了简单的缩小，那么对最终结果也造成不了多大的损失，反而有助于稳定训练。
- 在inception-resnet-v1与inception v3的对比中，inception-resnet-v1虽然训练速度更快，不过最后结果有那么一丢丢的差于inception v3；
- 而在inception-resnet-v2与inception v4的对比中，inception-resnet-v2的训练速度更块，而且结果比inception v4也更好一点。所以最后胜出的就是inception-resnet-v2。
**图3.3 不同模型的结果对比**
4. 代码
4.1 Inception-V4
from keras.layers import Input
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras import backend as K
from keras.models import Model
from keras.utils import plot_model
CONV_BLOCK_COUNT = 0 # 用来命名计数卷积编号
INCEPTION_A_COUNT = 0
INCEPTION_B_COUNT = 0
INCEPTION_C_COUNT = 0
def conv_block(x, nb_filters, nb_row, nb_col, strides=(1, 1), padding='same', use_bias=False):
global CONV_BLOCK_COUNT
CONV_BLOCK_COUNT += 1
with K.name_scope('conv_block_'+str(CONV_BLOCK_COUNT)):
x = Conv2D(filters=nb_filters,
kernel_size=(nb_row, nb_col),
strides=strides,
padding=padding,
use_bias=use_bias)(x)
x = BatchNormalization(axis=-1, momentum=0.9997, scale=False)(x)
x = Activation("relu")(x)
return x
def stem(x_input):
with K.name_scope('stem'):
x = conv_block(x_input, 32, 3, 3, strides=(2, 2), padding='valid')
x = conv_block(x, 32, 3, 3, padding='valid')
x = conv_block(x, 64, 3, 3)
x1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
x2 = conv_block(x, 96, 3, 3, strides=(2, 2), padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 64, 1, 1)
x1 = conv_block(x1, 96, 3, 3, padding='valid')
x2 = conv_block(x, 64, 1, 1)
x2 = conv_block(x2, 64, 7, 1)
x2 = conv_block(x2, 64, 1, 7)
x2 = conv_block(x2, 96, 3, 3, padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 192, 3, 3, strides=(2, 2), padding='valid')
x2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
merged_vector = concatenate([x1, x2], axis=-1)
return merged_vector
def inception_A(x_input):
"""35*35 卷积块"""
global INCEPTION_A_COUNT
INCEPTION_A_COUNT += 1
with K.name_scope('inception_A' + str(INCEPTION_A_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input) # 35 * 35 * 192
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 96, 1, 1) # 35 * 35 * 96
conv1x1 = conv_block(x_input, 96, 1, 1) # 35 * 35 * 96
conv1x1_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv1x1_3x3 = conv_block(conv1x1_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x1_3x3, conv3x3_3x3], axis=-1) # 35 * 35 * 384
return merged_vector
def inception_B(x_input):
"""17*17 卷积块"""
global INCEPTION_B_COUNT
INCEPTION_B_COUNT += 1
with K.name_scope('inception_B' + str(INCEPTION_B_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 128, 1, 1)
conv1x1 = conv_block(x_input, 384, 1, 1)
conv1x7_1x7 = conv_block(x_input, 192, 1, 1)
conv1x7_1x7 = conv_block(conv1x7_1x7, 224, 1, 7)
conv1x7_1x7 = conv_block(conv1x7_1x7, 256, 1, 7)
conv2_1x7_7x1 = conv_block(x_input, 192, 1, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 192, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 7, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 256, 7, 1)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x7_1x7, conv2_1x7_7x1], axis=-1)
return merged_vector
def inception_C(x_input):
"""8*8 卷积块"""
global INCEPTION_C_COUNT
INCEPTION_C_COUNT += 1
with K.name_scope('Inception_C' + str(INCEPTION_C_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 256, 1, 1)
conv1x1 = conv_block(x_input, 256, 1, 1)
# 用 1x3 和 3x1 替代 3x3
conv3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv3x3_1 = conv_block(conv3x3_1x1, 256, 1, 3)
conv3x3_2 = conv_block(conv3x3_1x1, 256, 3, 1)
conv2_3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 448, 1, 3)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 512, 3, 1)
conv2_3x3_1x1_1 = conv_block(conv2_3x3_1x1, 256, 3, 1)
conv2_3x3_1x1_2 = conv_block(conv2_3x3_1x1, 256, 1, 3)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv3x3_1, conv3x3_2, conv2_3x3_1x1_1, conv2_3x3_1x1_2], axis=-1)
return merged_vector
def reduction_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('Reduction_A'):
"""Architecture of a 35 * 35 to 17 * 17 Reduction_A block."""
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, n, 3, 3, strides=(2, 2), padding='valid')
conv2_3x3 = conv_block(x_input, k, 1, 1)
conv2_3x3 = conv_block(conv2_3x3, l, 3, 3)
conv2_3x3 = conv_block(conv2_3x3, m, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv2_3x3], axis=-1)
return merged_vector
def reduction_B(x_input):
"""Architecture of a 17 * 17 to 8 * 8 Reduction_B block."""
with K.name_scope('Reduction_B'):
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, 192, 1, 1)
conv3x3 = conv_block(conv3x3, 192, 3, 3, strides=(2, 2), padding='valid')
conv1x7_7x1_3x3 = conv_block(x_input, 256, 1, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 256, 1, 7)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 7, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv1x7_7x1_3x3], axis=-1)
return merged_vector
def inception_v4_backbone(nb_classes=1000, load_weights=True):
x_input = Input(shape=(299, 299, 3))
# Stem
x = stem(x_input) # 35 x 35 x 384
# 4 x Inception_A
for i in range(4):
x = inception_A(x) # 35 x 35 x 384
# Reduction_A
x = reduction_A(x, k=192, l=224, m=256, n=384) # 17 x 17 x 1024
# 7 x Inception_B
for i in range(7):
x = inception_B(x) # 17 x 17 x1024
# Reduction_B
x = reduction_B(x) # 8 x 8 x 1536
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x) # 1536
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x) # 1536
# 全连接层
x = Dense(units=nb_classes, activation='softmax')(x)
model = Model(inputs=x_input, outputs=x, name='Inception-V4')
return model
if __name__ == '__main__':
inception_v4 = inception_v4_backbone()
plot_model(inception_v4, 'inception_v4.png', show_shapes=True)
4.2 inception_resnet_v1
from keras.layers import Input
from keras.layers.merge import concatenate, add
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras import backend as K
from keras.utils import plot_model
RESNET_V1_A_COUNT = 0
RESNET_V1_B_COUNT = 0
RESNET_V1_C_COUNT = 0
def resnet_v1_stem(x_input):
with K.name_scope('Stem'):
x = Conv2D(filters=32, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='valid')(x_input)
x = Conv2D(32, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=2, padding='valid')(x)
x = Conv2D(80, (1, 1), activation='relu', padding='same')(x)
x = Conv2D(192, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(x)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_A(x_input, scale_residual=True):
""" 35x35 卷积核"""
global RESNET_V1_A_COUNT
RESNET_V1_A_COUNT += 1
with K.name_scope('resnet_v1_A' + str(RESNET_V1_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar2)
ar3 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
merged_vector = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(256, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual: # 是否缩小
ar = Lambda(lambda x: 0.1*x)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_B(x_input, scale_residual=True):
""" 17x17 卷积核"""
global RESNET_V1_B_COUNT
RESNET_V1_B_COUNT += 1
with K.name_scope('resnet_v1_B' + str(RESNET_V1_B_COUNT)):
br1 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 7), activation='relu', padding='same')(br2)
br2 = Conv2D(128, (7, 1), activation='relu', padding='same')(br2)
merged_vector = concatenate([br1, br2], axis=-1)
br = Conv2D(896, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual:
br = Lambda(lambda x: 0.1*x)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_C(x_input, scale_residual=True):
global RESNET_V1_C_COUNT
RESNET_V1_C_COUNT += 1
with K.name_scope('resnet_v1_C' + str(RESNET_V1_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 3), activation='relu', padding='same')(cr2)
cr2 = Conv2D(192, (3, 1), activation='relu', padding='same')(cr2)
merged_vector = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(1792, (1, 1), activation='relu', padding='same')(merged_vector)
if scale_residual:
cr = Lambda(lambda x: 0.1*x)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def reduction_resnet_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('reduction_resnet_A'):
ra1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
ra2 = Conv2D(n, (3, 3), activation='relu', strides=(2, 2), padding='valid')(x_input)
ra3 = Conv2D(k, (1, 1), activation='relu', padding='same')(x_input)
ra3 = Conv2D(l, (3, 3), activation='relu', padding='same')(ra3)
ra3 = Conv2D(m, (3, 3), activation='relu', strides=(2, 2), padding='valid')(ra3)
merged_vector = concatenate([ra1, ra2, ra3], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def reduction_resnet_B(x_input):
with K.name_scope('reduction_resnet_B'):
rb1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2),padding='valid')(x_input)
rb2 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb2 = Conv2D(384, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb2)
rb3 = Conv2D(256, (1, 1),activation='relu', padding='same')(x_input)
rb3 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb3)
rb4 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb4 = Conv2D(256, (3, 3), activation='relu', padding='same')(rb4)
rb4 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb4)
merged_vector = concatenate([rb1, rb2, rb3, rb4], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def inception_resnet_v1_backbone(nb_classes=1000, scale=True):
x_input = Input(shape=(299, 299, 3))
# stem
x = resnet_v1_stem(x_input)
# 5 x inception_resnet_v1_A
for i in range(5):
x = inception_resnet_v1_A(x, scale_residual=False)
# reduction_resnet_A
x = reduction_resnet_A(x, k=192, l=192, m=256, n=384)
# 10 x inception_resnet_v1_B
for i in range(10):
x = inception_resnet_v1_B(x, scale_residual=True)
# Reduction B
x = reduction_resnet_B(x)
# 5 x Inception C
for i in range(5):
x = inception_resnet_v1_C(x, scale_residual=True)
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x)
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x)
x = Dense(units=nb_classes, activation='softmax')(x)
return Model(inputs=x_input, outputs=x, name='Inception-Resnet-v1')
if __name__ == '__main__':
inception_resnet_v1_model = inception_resnet_v1_backbone()
plot_model(inception_resnet_v1_model, to_file='inception_resnet_v1.png', show_shapes=True)
4.3 inception_resnet_v2
from keras.layers import Input, add
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from inception_resnet_v1 import reduction_resnet_A
from keras.utils import plot_model
import keras.backend as K
RESNET_V2_A_COUNT = 0
RESNET_V2_B_COUNT = 0
RESNET_V2_C_COUNT = 0
def resnet_v2_stem(x_input):
'''The stem of the pure Inception-v4 and Inception-ResNet-v2 networks. This is input part of those networks.'''
# Input shape is 299 * 299 * 3 (Tensorflow dimension ordering)
with K.name_scope("stem"):
x = Conv2D(32, (3, 3), activation="relu", strides=(2, 2))(x_input) # 149 * 149 * 32
x = Conv2D(32, (3, 3), activation="relu")(x) # 147 * 147 * 32
x = Conv2D(64, (3, 3), activation="relu", padding="same")(x) # 147 * 147 * 64
x1 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x2 = Conv2D(96, (3, 3), activation="relu", strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 73 * 73 * 160
x1 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x1 = Conv2D(96, (3, 3), activation="relu")(x1)
x2 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x2 = Conv2D(64, (7, 1), activation="relu", padding="same")(x2)
x2 = Conv2D(64, (1, 7), activation="relu", padding="same")(x2)
x2 = Conv2D(96, (3, 3), activation="relu", padding="valid")(x2)
x = concatenate([x1, x2], axis=-1) # 71 * 71 * 192
x1 = Conv2D(192, (3, 3), activation="relu", strides=(2, 2))(x)
x2 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 35 * 35 * 384
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_A(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_A block which is a 35 * 35 grid module.'''
global RESNET_V2_A_COUNT
RESNET_V2_A_COUNT += 1
with K.name_scope('inception_resnet_v2_A' + str(RESNET_V2_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (3, 3), activation="relu", padding="same")(ar2)
ar3 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar3 = Conv2D(48, (3, 3), activation="relu", padding="same")(ar3)
ar3 = Conv2D(64, (3, 3), activation="relu", padding="same")(ar3)
merged = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(384, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: ar = Lambda(lambda a: a * 0.1)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_B(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_B block which is a 17 * 17 grid module.'''
global RESNET_V2_B_COUNT
RESNET_V2_B_COUNT += 1
with K.name_scope('inception_resnet_v2_B' + str(RESNET_V2_B_COUNT)):
br1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(128, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(160, (1, 7), activation="relu", padding="same")(br2)
br2 = Conv2D(192, (7, 1), activation="relu", padding="same")(br2)
merged = concatenate([br1, br2], axis=-1)
br = Conv2D(1152, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: br = Lambda(lambda b: b * 0.1)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_C(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_C block which is a 8 * 8 grid module.'''
global RESNET_V2_C_COUNT
RESNET_V2_C_COUNT += 1
with K.name_scope('inception_resnet_v2_C' + str(RESNET_V2_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(224, (1, 3), activation="relu", padding="same")(cr2)
cr2 = Conv2D(256, (3, 1), activation="relu", padding="same")(cr2)
merged = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(2144, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: cr = Lambda(lambda c: c * 0.1)(cr)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def reduction_resnet_v2_B(x_input):
'''Architecture of a 17 * 17 to 8 * 8 Reduction_ResNet_B block.'''
with K.name_scope('reduction_resnet_v2_B'):
rbr1 = MaxPooling2D((3, 3), strides=(2, 2), padding="valid")(x_input)
rbr2 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr2 = Conv2D(384, (3, 3), activation="relu", strides=(2, 2))(rbr2)
rbr3 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr3 = Conv2D(288, (3, 3), activation="relu", strides=(2, 2))(rbr3)
rbr4 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr4 = Conv2D(288, (3, 3), activation="relu", padding="same")(rbr4)
rbr4 = Conv2D(320, (3, 3), activation="relu", strides=(2, 2))(rbr4)
merged = concatenate([rbr1, rbr2, rbr3, rbr4], axis=-1)
rbr = BatchNormalization(axis=-1)(merged)
rbr = Activation("relu")(rbr)
return rbr
def inception_resnet_v2(nb_classes=1001, scale=True):
'''Creates the Inception_ResNet_v1 network.'''
init = Input((299, 299, 3)) # Channels last, as using Tensorflow backend with Tensorflow image dimension ordering
# Input shape is 299 * 299 * 3
x = resnet_v2_stem(init) # Output: 35 * 35 * 256
# 5 x Inception A
for i in range(5):
x = inception_resnet_v2_A(x, scale_residual=scale)
# Output: 35 * 35 * 256
# Reduction A
x = reduction_resnet_A(x, k=256, l=256, m=384, n=384) # Output: 17 * 17 * 896
# 10 x Inception B
for i in range(10):
x = inception_resnet_v2_B(x, scale_residual=scale)
# Output: 17 * 17 * 896
# Reduction B
x = reduction_resnet_v2_B(x) # Output: 8 * 8 * 1792
# 5 x Inception C
for i in range(5):
x = inception_resnet_v2_C(x, scale_residual=scale)
# Output: 8 * 8 * 1792
# Average Pooling
x = AveragePooling2D((8, 8))(x) # Output: 1792
# Dropout
x = Dropout(0.2)(x) # Keep dropout 0.2 as mentioned in the paper
x = Flatten()(x) # Output: 1792
# Output layer
output = Dense(units=nb_classes, activation="softmax")(x) # Output: 10000
model = Model(init, output, name="Inception-ResNet-v2")
return model
if __name__ == "__main__":
inception_resnet_v2_model = inception_resnet_v2()
plot_model(inception_resnet_v2_model, to_file='inception_resnet_v2.png', show_shapes=True)
InceptionV4的更多相关文章
- 【Network Architecture】Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning（转）
文章来源: https://www.cnblogs.com/shouhuxianjian/p/7786760.html Feature Extractor[Inception v4] 0. 背景 随着 ...
- Tensorflow 训练inceptionV4 并移植
安装brazel (请使用最新版的brazel 和最新版的tensorflow ,版本不匹配会出错!!!) 下载bazel-0.23 https://pan.baidu.com/s/1X ...
- (转) Awesome - Most Cited Deep Learning Papers
转自:https://github.com/terryum/awesome-deep-learning-papers Awesome - Most Cited Deep Learning Papers ...
- awesome-very-deep-learning
awesome-very-deep-learning is a curated list for papers and code about implementing and training ver ...
- 学习笔记TF032:实现Google Inception Net
Google Inception Net,ILSVRC 2014比赛第一名.控制计算量.参数量,分类性能非常好.V1,top-5错误率6.67%,22层,15亿次浮点运算,500万参数(AlexNet ...
- 读论文系列：Deep transfer learning person re-identification
读论文系列:Deep transfer learning person re-identification arxiv 2016 by Mengyue Geng, Yaowei Wang, Tao X ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现经典CNN网络GoogLeNet
前面讲了LeNet.AlexNet和Vgg,这周来讲讲GoogLeNet.GoogLeNet是由google的Christian Szegedy等人在2014年的论文<Going Deeper ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现GoogLeNet InceptionV2/V3/V4
上一篇文章我们引出了GoogLeNet InceptionV1的网络结构,这篇文章中我们会详细讲到Inception V2/V3/V4的发展历程以及它们的网络结构和亮点. GoogLeNet Ince ...
- 吐血整理：人工智能PDF中文教材资源包2.73G基本包含全部学习资料-人工智能学习书单
吐血整理:人工智能PDF中文教材资源包2.73G基本包含全部学习资料 人工智能学习书单(关注微信公众号:aibbtcom获取更多资源) 文末附百度网盘下载地址 人工神经网络与盲信号处理 人工神经网络与 ...
随机推荐
- Andrew Ng - 深度学习工程师 - Part 1. 神经网络和深度学习（Week 4. 深层神经网络）
=================第2周 神经网络基础=============== ===4.1 深层神经网络=== Although for any given problem it migh ...
- C++ 公有继承、保护继承和私有继承的对比
在c++的继承控制中,有三种不同的控制权限,分别是public.protected和private.定义派生类时,若不显示加上这三个关键字,就会使用默认的方式,用struct定义的类是默认public ...
- 490. The Maze
原题链接:https://leetcode.com/articles/the-maze/ 这道题目是需要冲会员才能使用的,然而我个穷逼现在还是失业状态根本冲不起...以后如果把免费题目都刷完了的话,再 ...
- 关于JavaScript函数
object.defineProperty()函数 再学习这个函数之前,我们先创建一个object对象 var person = {} person.name = "junlebao&quo ...
- Apache POI 操作Excel（3）-- Excel基础
Excel基本组成 首先在生成Excel前,我们需要了解Excel文件的组织形式.一个Excel文件称为一个workbook,一个workerbook至少包含一个表单(sheet),一个表单有多个行( ...
- Hive 报错SemanticException Error in parsing
以下sql执行时报错SemanticException Error in parsing select clr.id,clr.customer_id,clr,contract_code,clr.cor ...
- 修改CentOS7登录欢迎界面信息
vi /etc/issue 添加自己喜欢的内容,保存即可. 特殊字符的含义: \d 本地端时间的日期: \l 显示第几个终端机接口: \m 显示硬件的等级 (i386/i486/i586/i686.. ...
- Write a program that prints its input one word per line.
#include <stdio.h> #define State '\n' void main() { int Juge=;/*only one space*/ int c=; while ...
- CSS3 nth-child的使用，详解css中nth的作用，以及nth-child的兼容写法
:nth-child是css3的一个比较常用的选择器.它用于匹配属于其父元素中的子元素,不论元素的类型. 它的参数可以是数字.关键词或公式.下面讲介绍它的使用方法, nth-child的使用 html ...
- HDU3686 Traffic Real Time Query System 题解
题目 City C is really a nightmare of all drivers for its traffic jams. To solve the traffic problem, t ...