InceptionV4
目录
在下面的结构图中，每一个inception模块中都有一个1∗1的没有激活层的卷积层，用来扩展通道数，从而补偿因为inception模块导致的维度约间。其中Inception-ResNet-V1的结果与Inception v3相当；Inception-ResNet-V1与Inception v4结果差不多，不过实际过程中Inception v4会明显慢于Inception-ResNet-v2，这也许是因为层数太多了。且在Inception-ResNet结构中，只在传统层的上面使用BN层，而不在合并层上使用BN，虽然处处使用BN是有好处，不过更希望能够将一个完整的组件放入单独的GPU中。因为具有大量激活单元的层会占用过多的显存，所以希望这些地方丢弃BN，从而总体增加Inception模块的数量。使得不需要去解决计算资源和模块什么的权衡问题。
1. inception v4
**图1.1 inception v4 网络结构图**
**图1.2 图1.1的stem和Inception-A部分结构图**
**图1.3 图1.1的Reduction-A和Inception-B部分结构图**
**图1.4 图1.1的Reduction-B和Inception-C部分结构图**
2. Inception-resnet-v1 & Inception-resnet-v2
**图2.1 Inception-resnet-v1 & Inception-resnet-v2的结构图**
2.1 Inception-resnet-v1的组成模块
**图2.1.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.1.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.1.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
2.2 Inception-resnet-v2的组成模块
**图2.2.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.2.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.2.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
3. 模型训练
在上述的Inception V4，Inception-Resnet-V1，Inception-ResNet-v2这三个模型中都用到了Reduction-A，他们各自的具体参数如下：
图3.1 不同模型下Reduction-A的模型超参数
作者们在训练的过程中发现，如果通道数超过1000，那么Inception-resnet等网络都会开始变得不稳定，并且过早的就“死掉了”，即在迭代几万次之后，平均池化的前面一层就会生成很多的0值。作者们通过调低学习率，增加BN都没有任何改善。
不过他们发现如果在将残差汇入之前，对残差进行缩小，可以让模型稳定训练，值通常选择[0,1.0.3]，如图3.2
**图3.2 对inception-resnet模块进行最后输出值的等比例缩小**
同样的在ResNet-v1中，何凯明等人也在cifar-10中发现了模型的不稳定现象：即在特别深的网络基础上去训cifar-10，需要先以0.01的学习率去训练，然后在以0.1的学习率训练。
不过这里的作者们认为如果通道数特别多的话，即使以特别低的学习率（0.00001）训练也无法让模型收敛，如果之后再用大学习率，那么就会轻松的破坏掉之前的成果。然而简单的缩小残差的输出值有助于学习的稳定，即使进行了简单的缩小，那么对最终结果也造成不了多大的损失，反而有助于稳定训练。
- 在inception-resnet-v1与inception v3的对比中，inception-resnet-v1虽然训练速度更快，不过最后结果有那么一丢丢的差于inception v3；
- 而在inception-resnet-v2与inception v4的对比中，inception-resnet-v2的训练速度更块，而且结果比inception v4也更好一点。所以最后胜出的就是inception-resnet-v2。
**图3.3 不同模型的结果对比**
4. 代码
4.1 Inception-V4
from keras.layers import Input
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras import backend as K
from keras.models import Model
from keras.utils import plot_model
CONV_BLOCK_COUNT = 0 # 用来命名计数卷积编号
INCEPTION_A_COUNT = 0
INCEPTION_B_COUNT = 0
INCEPTION_C_COUNT = 0
def conv_block(x, nb_filters, nb_row, nb_col, strides=(1, 1), padding='same', use_bias=False):
global CONV_BLOCK_COUNT
CONV_BLOCK_COUNT += 1
with K.name_scope('conv_block_'+str(CONV_BLOCK_COUNT)):
x = Conv2D(filters=nb_filters,
kernel_size=(nb_row, nb_col),
strides=strides,
padding=padding,
use_bias=use_bias)(x)
x = BatchNormalization(axis=-1, momentum=0.9997, scale=False)(x)
x = Activation("relu")(x)
return x
def stem(x_input):
with K.name_scope('stem'):
x = conv_block(x_input, 32, 3, 3, strides=(2, 2), padding='valid')
x = conv_block(x, 32, 3, 3, padding='valid')
x = conv_block(x, 64, 3, 3)
x1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
x2 = conv_block(x, 96, 3, 3, strides=(2, 2), padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 64, 1, 1)
x1 = conv_block(x1, 96, 3, 3, padding='valid')
x2 = conv_block(x, 64, 1, 1)
x2 = conv_block(x2, 64, 7, 1)
x2 = conv_block(x2, 64, 1, 7)
x2 = conv_block(x2, 96, 3, 3, padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 192, 3, 3, strides=(2, 2), padding='valid')
x2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
merged_vector = concatenate([x1, x2], axis=-1)
return merged_vector
def inception_A(x_input):
"""35*35 卷积块"""
global INCEPTION_A_COUNT
INCEPTION_A_COUNT += 1
with K.name_scope('inception_A' + str(INCEPTION_A_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input) # 35 * 35 * 192
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 96, 1, 1) # 35 * 35 * 96
conv1x1 = conv_block(x_input, 96, 1, 1) # 35 * 35 * 96
conv1x1_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv1x1_3x3 = conv_block(conv1x1_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x1_3x3, conv3x3_3x3], axis=-1) # 35 * 35 * 384
return merged_vector
def inception_B(x_input):
"""17*17 卷积块"""
global INCEPTION_B_COUNT
INCEPTION_B_COUNT += 1
with K.name_scope('inception_B' + str(INCEPTION_B_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 128, 1, 1)
conv1x1 = conv_block(x_input, 384, 1, 1)
conv1x7_1x7 = conv_block(x_input, 192, 1, 1)
conv1x7_1x7 = conv_block(conv1x7_1x7, 224, 1, 7)
conv1x7_1x7 = conv_block(conv1x7_1x7, 256, 1, 7)
conv2_1x7_7x1 = conv_block(x_input, 192, 1, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 192, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 7, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 256, 7, 1)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x7_1x7, conv2_1x7_7x1], axis=-1)
return merged_vector
def inception_C(x_input):
"""8*8 卷积块"""
global INCEPTION_C_COUNT
INCEPTION_C_COUNT += 1
with K.name_scope('Inception_C' + str(INCEPTION_C_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 256, 1, 1)
conv1x1 = conv_block(x_input, 256, 1, 1)
# 用 1x3 和 3x1 替代 3x3
conv3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv3x3_1 = conv_block(conv3x3_1x1, 256, 1, 3)
conv3x3_2 = conv_block(conv3x3_1x1, 256, 3, 1)
conv2_3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 448, 1, 3)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 512, 3, 1)
conv2_3x3_1x1_1 = conv_block(conv2_3x3_1x1, 256, 3, 1)
conv2_3x3_1x1_2 = conv_block(conv2_3x3_1x1, 256, 1, 3)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv3x3_1, conv3x3_2, conv2_3x3_1x1_1, conv2_3x3_1x1_2], axis=-1)
return merged_vector
def reduction_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('Reduction_A'):
"""Architecture of a 35 * 35 to 17 * 17 Reduction_A block."""
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, n, 3, 3, strides=(2, 2), padding='valid')
conv2_3x3 = conv_block(x_input, k, 1, 1)
conv2_3x3 = conv_block(conv2_3x3, l, 3, 3)
conv2_3x3 = conv_block(conv2_3x3, m, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv2_3x3], axis=-1)
return merged_vector
def reduction_B(x_input):
"""Architecture of a 17 * 17 to 8 * 8 Reduction_B block."""
with K.name_scope('Reduction_B'):
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, 192, 1, 1)
conv3x3 = conv_block(conv3x3, 192, 3, 3, strides=(2, 2), padding='valid')
conv1x7_7x1_3x3 = conv_block(x_input, 256, 1, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 256, 1, 7)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 7, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv1x7_7x1_3x3], axis=-1)
return merged_vector
def inception_v4_backbone(nb_classes=1000, load_weights=True):
x_input = Input(shape=(299, 299, 3))
# Stem
x = stem(x_input) # 35 x 35 x 384
# 4 x Inception_A
for i in range(4):
x = inception_A(x) # 35 x 35 x 384
# Reduction_A
x = reduction_A(x, k=192, l=224, m=256, n=384) # 17 x 17 x 1024
# 7 x Inception_B
for i in range(7):
x = inception_B(x) # 17 x 17 x1024
# Reduction_B
x = reduction_B(x) # 8 x 8 x 1536
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x) # 1536
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x) # 1536
# 全连接层
x = Dense(units=nb_classes, activation='softmax')(x)
model = Model(inputs=x_input, outputs=x, name='Inception-V4')
return model
if __name__ == '__main__':
inception_v4 = inception_v4_backbone()
plot_model(inception_v4, 'inception_v4.png', show_shapes=True)
4.2 inception_resnet_v1
from keras.layers import Input
from keras.layers.merge import concatenate, add
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras import backend as K
from keras.utils import plot_model
RESNET_V1_A_COUNT = 0
RESNET_V1_B_COUNT = 0
RESNET_V1_C_COUNT = 0
def resnet_v1_stem(x_input):
with K.name_scope('Stem'):
x = Conv2D(filters=32, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='valid')(x_input)
x = Conv2D(32, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=2, padding='valid')(x)
x = Conv2D(80, (1, 1), activation='relu', padding='same')(x)
x = Conv2D(192, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(x)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_A(x_input, scale_residual=True):
""" 35x35 卷积核"""
global RESNET_V1_A_COUNT
RESNET_V1_A_COUNT += 1
with K.name_scope('resnet_v1_A' + str(RESNET_V1_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar2)
ar3 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
merged_vector = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(256, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual: # 是否缩小
ar = Lambda(lambda x: 0.1*x)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_B(x_input, scale_residual=True):
""" 17x17 卷积核"""
global RESNET_V1_B_COUNT
RESNET_V1_B_COUNT += 1
with K.name_scope('resnet_v1_B' + str(RESNET_V1_B_COUNT)):
br1 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 7), activation='relu', padding='same')(br2)
br2 = Conv2D(128, (7, 1), activation='relu', padding='same')(br2)
merged_vector = concatenate([br1, br2], axis=-1)
br = Conv2D(896, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual:
br = Lambda(lambda x: 0.1*x)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_C(x_input, scale_residual=True):
global RESNET_V1_C_COUNT
RESNET_V1_C_COUNT += 1
with K.name_scope('resnet_v1_C' + str(RESNET_V1_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 3), activation='relu', padding='same')(cr2)
cr2 = Conv2D(192, (3, 1), activation='relu', padding='same')(cr2)
merged_vector = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(1792, (1, 1), activation='relu', padding='same')(merged_vector)
if scale_residual:
cr = Lambda(lambda x: 0.1*x)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def reduction_resnet_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('reduction_resnet_A'):
ra1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
ra2 = Conv2D(n, (3, 3), activation='relu', strides=(2, 2), padding='valid')(x_input)
ra3 = Conv2D(k, (1, 1), activation='relu', padding='same')(x_input)
ra3 = Conv2D(l, (3, 3), activation='relu', padding='same')(ra3)
ra3 = Conv2D(m, (3, 3), activation='relu', strides=(2, 2), padding='valid')(ra3)
merged_vector = concatenate([ra1, ra2, ra3], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def reduction_resnet_B(x_input):
with K.name_scope('reduction_resnet_B'):
rb1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2),padding='valid')(x_input)
rb2 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb2 = Conv2D(384, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb2)
rb3 = Conv2D(256, (1, 1),activation='relu', padding='same')(x_input)
rb3 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb3)
rb4 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb4 = Conv2D(256, (3, 3), activation='relu', padding='same')(rb4)
rb4 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb4)
merged_vector = concatenate([rb1, rb2, rb3, rb4], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def inception_resnet_v1_backbone(nb_classes=1000, scale=True):
x_input = Input(shape=(299, 299, 3))
# stem
x = resnet_v1_stem(x_input)
# 5 x inception_resnet_v1_A
for i in range(5):
x = inception_resnet_v1_A(x, scale_residual=False)
# reduction_resnet_A
x = reduction_resnet_A(x, k=192, l=192, m=256, n=384)
# 10 x inception_resnet_v1_B
for i in range(10):
x = inception_resnet_v1_B(x, scale_residual=True)
# Reduction B
x = reduction_resnet_B(x)
# 5 x Inception C
for i in range(5):
x = inception_resnet_v1_C(x, scale_residual=True)
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x)
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x)
x = Dense(units=nb_classes, activation='softmax')(x)
return Model(inputs=x_input, outputs=x, name='Inception-Resnet-v1')
if __name__ == '__main__':
inception_resnet_v1_model = inception_resnet_v1_backbone()
plot_model(inception_resnet_v1_model, to_file='inception_resnet_v1.png', show_shapes=True)
4.3 inception_resnet_v2
from keras.layers import Input, add
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from inception_resnet_v1 import reduction_resnet_A
from keras.utils import plot_model
import keras.backend as K
RESNET_V2_A_COUNT = 0
RESNET_V2_B_COUNT = 0
RESNET_V2_C_COUNT = 0
def resnet_v2_stem(x_input):
'''The stem of the pure Inception-v4 and Inception-ResNet-v2 networks. This is input part of those networks.'''
# Input shape is 299 * 299 * 3 (Tensorflow dimension ordering)
with K.name_scope("stem"):
x = Conv2D(32, (3, 3), activation="relu", strides=(2, 2))(x_input) # 149 * 149 * 32
x = Conv2D(32, (3, 3), activation="relu")(x) # 147 * 147 * 32
x = Conv2D(64, (3, 3), activation="relu", padding="same")(x) # 147 * 147 * 64
x1 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x2 = Conv2D(96, (3, 3), activation="relu", strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 73 * 73 * 160
x1 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x1 = Conv2D(96, (3, 3), activation="relu")(x1)
x2 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x2 = Conv2D(64, (7, 1), activation="relu", padding="same")(x2)
x2 = Conv2D(64, (1, 7), activation="relu", padding="same")(x2)
x2 = Conv2D(96, (3, 3), activation="relu", padding="valid")(x2)
x = concatenate([x1, x2], axis=-1) # 71 * 71 * 192
x1 = Conv2D(192, (3, 3), activation="relu", strides=(2, 2))(x)
x2 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 35 * 35 * 384
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_A(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_A block which is a 35 * 35 grid module.'''
global RESNET_V2_A_COUNT
RESNET_V2_A_COUNT += 1
with K.name_scope('inception_resnet_v2_A' + str(RESNET_V2_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (3, 3), activation="relu", padding="same")(ar2)
ar3 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar3 = Conv2D(48, (3, 3), activation="relu", padding="same")(ar3)
ar3 = Conv2D(64, (3, 3), activation="relu", padding="same")(ar3)
merged = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(384, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: ar = Lambda(lambda a: a * 0.1)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_B(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_B block which is a 17 * 17 grid module.'''
global RESNET_V2_B_COUNT
RESNET_V2_B_COUNT += 1
with K.name_scope('inception_resnet_v2_B' + str(RESNET_V2_B_COUNT)):
br1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(128, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(160, (1, 7), activation="relu", padding="same")(br2)
br2 = Conv2D(192, (7, 1), activation="relu", padding="same")(br2)
merged = concatenate([br1, br2], axis=-1)
br = Conv2D(1152, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: br = Lambda(lambda b: b * 0.1)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_C(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_C block which is a 8 * 8 grid module.'''
global RESNET_V2_C_COUNT
RESNET_V2_C_COUNT += 1
with K.name_scope('inception_resnet_v2_C' + str(RESNET_V2_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(224, (1, 3), activation="relu", padding="same")(cr2)
cr2 = Conv2D(256, (3, 1), activation="relu", padding="same")(cr2)
merged = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(2144, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: cr = Lambda(lambda c: c * 0.1)(cr)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def reduction_resnet_v2_B(x_input):
'''Architecture of a 17 * 17 to 8 * 8 Reduction_ResNet_B block.'''
with K.name_scope('reduction_resnet_v2_B'):
rbr1 = MaxPooling2D((3, 3), strides=(2, 2), padding="valid")(x_input)
rbr2 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr2 = Conv2D(384, (3, 3), activation="relu", strides=(2, 2))(rbr2)
rbr3 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr3 = Conv2D(288, (3, 3), activation="relu", strides=(2, 2))(rbr3)
rbr4 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr4 = Conv2D(288, (3, 3), activation="relu", padding="same")(rbr4)
rbr4 = Conv2D(320, (3, 3), activation="relu", strides=(2, 2))(rbr4)
merged = concatenate([rbr1, rbr2, rbr3, rbr4], axis=-1)
rbr = BatchNormalization(axis=-1)(merged)
rbr = Activation("relu")(rbr)
return rbr
def inception_resnet_v2(nb_classes=1001, scale=True):
'''Creates the Inception_ResNet_v1 network.'''
init = Input((299, 299, 3)) # Channels last, as using Tensorflow backend with Tensorflow image dimension ordering
# Input shape is 299 * 299 * 3
x = resnet_v2_stem(init) # Output: 35 * 35 * 256
# 5 x Inception A
for i in range(5):
x = inception_resnet_v2_A(x, scale_residual=scale)
# Output: 35 * 35 * 256
# Reduction A
x = reduction_resnet_A(x, k=256, l=256, m=384, n=384) # Output: 17 * 17 * 896
# 10 x Inception B
for i in range(10):
x = inception_resnet_v2_B(x, scale_residual=scale)
# Output: 17 * 17 * 896
# Reduction B
x = reduction_resnet_v2_B(x) # Output: 8 * 8 * 1792
# 5 x Inception C
for i in range(5):
x = inception_resnet_v2_C(x, scale_residual=scale)
# Output: 8 * 8 * 1792
# Average Pooling
x = AveragePooling2D((8, 8))(x) # Output: 1792
# Dropout
x = Dropout(0.2)(x) # Keep dropout 0.2 as mentioned in the paper
x = Flatten()(x) # Output: 1792
# Output layer
output = Dense(units=nb_classes, activation="softmax")(x) # Output: 10000
model = Model(init, output, name="Inception-ResNet-v2")
return model
if __name__ == "__main__":
inception_resnet_v2_model = inception_resnet_v2()
plot_model(inception_resnet_v2_model, to_file='inception_resnet_v2.png', show_shapes=True)
InceptionV4的更多相关文章
- 【Network Architecture】Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning（转）
文章来源: https://www.cnblogs.com/shouhuxianjian/p/7786760.html Feature Extractor[Inception v4] 0. 背景 随着 ...
- Tensorflow 训练inceptionV4 并移植
安装brazel (请使用最新版的brazel 和最新版的tensorflow ,版本不匹配会出错!!!) 下载bazel-0.23 https://pan.baidu.com/s/1X ...
- (转) Awesome - Most Cited Deep Learning Papers
转自:https://github.com/terryum/awesome-deep-learning-papers Awesome - Most Cited Deep Learning Papers ...
- awesome-very-deep-learning
awesome-very-deep-learning is a curated list for papers and code about implementing and training ver ...
- 学习笔记TF032:实现Google Inception Net
Google Inception Net,ILSVRC 2014比赛第一名.控制计算量.参数量,分类性能非常好.V1,top-5错误率6.67%,22层,15亿次浮点运算,500万参数(AlexNet ...
- 读论文系列：Deep transfer learning person re-identification
读论文系列:Deep transfer learning person re-identification arxiv 2016 by Mengyue Geng, Yaowei Wang, Tao X ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现经典CNN网络GoogLeNet
前面讲了LeNet.AlexNet和Vgg,这周来讲讲GoogLeNet.GoogLeNet是由google的Christian Szegedy等人在2014年的论文<Going Deeper ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现GoogLeNet InceptionV2/V3/V4
上一篇文章我们引出了GoogLeNet InceptionV1的网络结构,这篇文章中我们会详细讲到Inception V2/V3/V4的发展历程以及它们的网络结构和亮点. GoogLeNet Ince ...
- 吐血整理：人工智能PDF中文教材资源包2.73G基本包含全部学习资料-人工智能学习书单
吐血整理:人工智能PDF中文教材资源包2.73G基本包含全部学习资料 人工智能学习书单(关注微信公众号:aibbtcom获取更多资源) 文末附百度网盘下载地址 人工神经网络与盲信号处理 人工神经网络与 ...
随机推荐
- 基于web网站项目的性能测试结果分析
业务背景: 最近公司研发了一款对并发要求比较高的web项目,需要对其压力测试,模拟线上可能存在的问题,这个过程中遇到一些很多问题,这里重新梳理一下思路,希望能给遇到同样问题的小伙伴提供一个参考. 工具 ...
- 13.DRF-版本
Django rest framework源码分析(4)----版本 版本 新建一个工程Myproject和一个app名为api (1)api/models.py from django.db imp ...
- web安全中的session攻击
运行着个简单的demo后,打开login.jsp,使用firebug或chrome会发现,即使没有登录,我们也会有一个JSESSIONID,这是由服务器端在会话开始是通过set-cookie来设置的匿 ...
- JavaWeb网上图书商城完整项目--day02-27.查询所有分类功能之Servlet和Service层
我们在上面实现了数据库层的代码,现在我们来实现业务层和Servlet层的代码:业务层的代码如下: package com.weiyuan.goods.category.service; import ...
- 【Spring注解驱动开发】关于BeanPostProcessor后置处理器，你了解多少？
写在前面 有些小伙伴问我,学习Spring是不是不用学习到这么细节的程度啊?感觉这些细节的部分在实际工作中使用不到啊,我到底需不需要学习到这么细节的程度呢?我的答案是:有必要学习到这么细节的程度,而且 ...
- Java工具类—包装类
Java工具类--包装类 我们都知道,JDK 其实给我们提供了很多很多 Java 开发者已经写好的现成的类,他们其实都可以理解成工具类,比如我们常见的集合类,日期相关的类,数学相关的类等等,有了这些工 ...
- 洛谷P1220关路灯【区间dp】
题目描述 某一村庄在一条路线上安装了 \(n\) 盏路灯,每盏灯的功率有大有小(即同一段时间内消耗的电量有多有少).老张就住在这条路中间某一路灯旁,他有一项工作就是每天早上天亮时一盏一盏地关掉这些路灯 ...
- 二.4vue展示用户数据及用户组操作以及给用户组添加额外字段
一.用户列表 1.新建(1)views/users/index.vue: <template> <div class="user-list-container"& ...
- navicat连接vagrant中的数据库
- web单页应用是什么？它的好处与坏处有哪些（如何解决这些缺点）
web单页应用是什么? Web单页应用就是指只有一个Web页面作为入口的应用,在浏览器中运行期间不会重新加载页面.也就是说浏览器一开始会加载它必需的thml.css和Js,之后所有的交互操作都在一个页 ...