InceptionV4
目录
在下面的结构图中，每一个inception模块中都有一个1∗1的没有激活层的卷积层，用来扩展通道数，从而补偿因为inception模块导致的维度约间。其中Inception-ResNet-V1的结果与Inception v3相当；Inception-ResNet-V1与Inception v4结果差不多，不过实际过程中Inception v4会明显慢于Inception-ResNet-v2，这也许是因为层数太多了。且在Inception-ResNet结构中，只在传统层的上面使用BN层，而不在合并层上使用BN，虽然处处使用BN是有好处，不过更希望能够将一个完整的组件放入单独的GPU中。因为具有大量激活单元的层会占用过多的显存，所以希望这些地方丢弃BN，从而总体增加Inception模块的数量。使得不需要去解决计算资源和模块什么的权衡问题。
1. inception v4
**图1.1 inception v4 网络结构图**
**图1.2 图1.1的stem和Inception-A部分结构图**
**图1.3 图1.1的Reduction-A和Inception-B部分结构图**
**图1.4 图1.1的Reduction-B和Inception-C部分结构图**
2. Inception-resnet-v1 & Inception-resnet-v2
**图2.1 Inception-resnet-v1 & Inception-resnet-v2的结构图**
2.1 Inception-resnet-v1的组成模块
**图2.1.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.1.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.1.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
2.2 Inception-resnet-v2的组成模块
**图2.2.1 图2.1的stem和Inception-ResNet-A部分结构图**
**图2.2.2 图2.1的Reduction-A和Inception-ResNet-B部分结构图**
**图2.2.3 图2.1的Reduction-B和Inception-ResNet-C部分结构图**
3. 模型训练
在上述的Inception V4，Inception-Resnet-V1，Inception-ResNet-v2这三个模型中都用到了Reduction-A，他们各自的具体参数如下：
图3.1 不同模型下Reduction-A的模型超参数
作者们在训练的过程中发现，如果通道数超过1000，那么Inception-resnet等网络都会开始变得不稳定，并且过早的就“死掉了”，即在迭代几万次之后，平均池化的前面一层就会生成很多的0值。作者们通过调低学习率，增加BN都没有任何改善。
不过他们发现如果在将残差汇入之前，对残差进行缩小，可以让模型稳定训练，值通常选择[0,1.0.3]，如图3.2
**图3.2 对inception-resnet模块进行最后输出值的等比例缩小**
同样的在ResNet-v1中，何凯明等人也在cifar-10中发现了模型的不稳定现象：即在特别深的网络基础上去训cifar-10，需要先以0.01的学习率去训练，然后在以0.1的学习率训练。
不过这里的作者们认为如果通道数特别多的话，即使以特别低的学习率（0.00001）训练也无法让模型收敛，如果之后再用大学习率，那么就会轻松的破坏掉之前的成果。然而简单的缩小残差的输出值有助于学习的稳定，即使进行了简单的缩小，那么对最终结果也造成不了多大的损失，反而有助于稳定训练。
- 在inception-resnet-v1与inception v3的对比中，inception-resnet-v1虽然训练速度更快，不过最后结果有那么一丢丢的差于inception v3；
- 而在inception-resnet-v2与inception v4的对比中，inception-resnet-v2的训练速度更块，而且结果比inception v4也更好一点。所以最后胜出的就是inception-resnet-v2。
**图3.3 不同模型的结果对比**
4. 代码
4.1 Inception-V4
from keras.layers import Input
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras import backend as K
from keras.models import Model
from keras.utils import plot_model
CONV_BLOCK_COUNT = 0 # 用来命名计数卷积编号
INCEPTION_A_COUNT = 0
INCEPTION_B_COUNT = 0
INCEPTION_C_COUNT = 0
def conv_block(x, nb_filters, nb_row, nb_col, strides=(1, 1), padding='same', use_bias=False):
global CONV_BLOCK_COUNT
CONV_BLOCK_COUNT += 1
with K.name_scope('conv_block_'+str(CONV_BLOCK_COUNT)):
x = Conv2D(filters=nb_filters,
kernel_size=(nb_row, nb_col),
strides=strides,
padding=padding,
use_bias=use_bias)(x)
x = BatchNormalization(axis=-1, momentum=0.9997, scale=False)(x)
x = Activation("relu")(x)
return x
def stem(x_input):
with K.name_scope('stem'):
x = conv_block(x_input, 32, 3, 3, strides=(2, 2), padding='valid')
x = conv_block(x, 32, 3, 3, padding='valid')
x = conv_block(x, 64, 3, 3)
x1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
x2 = conv_block(x, 96, 3, 3, strides=(2, 2), padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 64, 1, 1)
x1 = conv_block(x1, 96, 3, 3, padding='valid')
x2 = conv_block(x, 64, 1, 1)
x2 = conv_block(x2, 64, 7, 1)
x2 = conv_block(x2, 64, 1, 7)
x2 = conv_block(x2, 96, 3, 3, padding='valid')
x = concatenate([x1, x2], axis=-1)
x1 = conv_block(x, 192, 3, 3, strides=(2, 2), padding='valid')
x2 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x)
merged_vector = concatenate([x1, x2], axis=-1)
return merged_vector
def inception_A(x_input):
"""35*35 卷积块"""
global INCEPTION_A_COUNT
INCEPTION_A_COUNT += 1
with K.name_scope('inception_A' + str(INCEPTION_A_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input) # 35 * 35 * 192
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 96, 1, 1) # 35 * 35 * 96
conv1x1 = conv_block(x_input, 96, 1, 1) # 35 * 35 * 96
conv1x1_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv1x1_3x3 = conv_block(conv1x1_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(x_input, 64, 1, 1) # 35 * 35 * 64
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
conv3x3_3x3 = conv_block(conv3x3_3x3, 96, 3, 3) # 35 * 35 * 96
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x1_3x3, conv3x3_3x3], axis=-1) # 35 * 35 * 384
return merged_vector
def inception_B(x_input):
"""17*17 卷积块"""
global INCEPTION_B_COUNT
INCEPTION_B_COUNT += 1
with K.name_scope('inception_B' + str(INCEPTION_B_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 128, 1, 1)
conv1x1 = conv_block(x_input, 384, 1, 1)
conv1x7_1x7 = conv_block(x_input, 192, 1, 1)
conv1x7_1x7 = conv_block(conv1x7_1x7, 224, 1, 7)
conv1x7_1x7 = conv_block(conv1x7_1x7, 256, 1, 7)
conv2_1x7_7x1 = conv_block(x_input, 192, 1, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 192, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 7, 1)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 224, 1, 7)
conv2_1x7_7x1 = conv_block(conv2_1x7_7x1, 256, 7, 1)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv1x7_1x7, conv2_1x7_7x1], axis=-1)
return merged_vector
def inception_C(x_input):
"""8*8 卷积块"""
global INCEPTION_C_COUNT
INCEPTION_C_COUNT += 1
with K.name_scope('Inception_C' + str(INCEPTION_C_COUNT)):
averagepooling_conv1x1 = AveragePooling2D(pool_size=(3, 3), strides=(1, 1), padding='same')(x_input)
averagepooling_conv1x1 = conv_block(averagepooling_conv1x1, 256, 1, 1)
conv1x1 = conv_block(x_input, 256, 1, 1)
# 用 1x3 和 3x1 替代 3x3
conv3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv3x3_1 = conv_block(conv3x3_1x1, 256, 1, 3)
conv3x3_2 = conv_block(conv3x3_1x1, 256, 3, 1)
conv2_3x3_1x1 = conv_block(x_input, 384, 1, 1)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 448, 1, 3)
conv2_3x3_1x1 = conv_block(conv2_3x3_1x1, 512, 3, 1)
conv2_3x3_1x1_1 = conv_block(conv2_3x3_1x1, 256, 3, 1)
conv2_3x3_1x1_2 = conv_block(conv2_3x3_1x1, 256, 1, 3)
merged_vector = concatenate([averagepooling_conv1x1, conv1x1, conv3x3_1, conv3x3_2, conv2_3x3_1x1_1, conv2_3x3_1x1_2], axis=-1)
return merged_vector
def reduction_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('Reduction_A'):
"""Architecture of a 35 * 35 to 17 * 17 Reduction_A block."""
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, n, 3, 3, strides=(2, 2), padding='valid')
conv2_3x3 = conv_block(x_input, k, 1, 1)
conv2_3x3 = conv_block(conv2_3x3, l, 3, 3)
conv2_3x3 = conv_block(conv2_3x3, m, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv2_3x3], axis=-1)
return merged_vector
def reduction_B(x_input):
"""Architecture of a 17 * 17 to 8 * 8 Reduction_B block."""
with K.name_scope('Reduction_B'):
maxpool = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
conv3x3 = conv_block(x_input, 192, 1, 1)
conv3x3 = conv_block(conv3x3, 192, 3, 3, strides=(2, 2), padding='valid')
conv1x7_7x1_3x3 = conv_block(x_input, 256, 1, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 256, 1, 7)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 7, 1)
conv1x7_7x1_3x3 = conv_block(conv1x7_7x1_3x3, 320, 3, 3, strides=(2, 2), padding='valid')
merged_vector = concatenate([maxpool, conv3x3, conv1x7_7x1_3x3], axis=-1)
return merged_vector
def inception_v4_backbone(nb_classes=1000, load_weights=True):
x_input = Input(shape=(299, 299, 3))
# Stem
x = stem(x_input) # 35 x 35 x 384
# 4 x Inception_A
for i in range(4):
x = inception_A(x) # 35 x 35 x 384
# Reduction_A
x = reduction_A(x, k=192, l=224, m=256, n=384) # 17 x 17 x 1024
# 7 x Inception_B
for i in range(7):
x = inception_B(x) # 17 x 17 x1024
# Reduction_B
x = reduction_B(x) # 8 x 8 x 1536
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x) # 1536
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x) # 1536
# 全连接层
x = Dense(units=nb_classes, activation='softmax')(x)
model = Model(inputs=x_input, outputs=x, name='Inception-V4')
return model
if __name__ == '__main__':
inception_v4 = inception_v4_backbone()
plot_model(inception_v4, 'inception_v4.png', show_shapes=True)
4.2 inception_resnet_v1
from keras.layers import Input
from keras.layers.merge import concatenate, add
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from keras import backend as K
from keras.utils import plot_model
RESNET_V1_A_COUNT = 0
RESNET_V1_B_COUNT = 0
RESNET_V1_C_COUNT = 0
def resnet_v1_stem(x_input):
with K.name_scope('Stem'):
x = Conv2D(filters=32, kernel_size=(3, 3), strides=(2, 2), activation='relu', padding='valid')(x_input)
x = Conv2D(32, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(64, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=2, padding='valid')(x)
x = Conv2D(80, (1, 1), activation='relu', padding='same')(x)
x = Conv2D(192, (3, 3), activation='relu', padding='valid')(x)
x = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(x)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_A(x_input, scale_residual=True):
""" 35x35 卷积核"""
global RESNET_V1_A_COUNT
RESNET_V1_A_COUNT += 1
with K.name_scope('resnet_v1_A' + str(RESNET_V1_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar2 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar2)
ar3 = Conv2D(32, (1, 1), activation='relu', padding='same')(x_input)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
ar3 = Conv2D(32, (3, 3), activation='relu', padding='same')(ar3)
merged_vector = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(256, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual: # 是否缩小
ar = Lambda(lambda x: 0.1*x)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_B(x_input, scale_residual=True):
""" 17x17 卷积核"""
global RESNET_V1_B_COUNT
RESNET_V1_B_COUNT += 1
with K.name_scope('resnet_v1_B' + str(RESNET_V1_B_COUNT)):
br1 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 1), activation='relu', padding='same')(x_input)
br2 = Conv2D(128, (1, 7), activation='relu', padding='same')(br2)
br2 = Conv2D(128, (7, 1), activation='relu', padding='same')(br2)
merged_vector = concatenate([br1, br2], axis=-1)
br = Conv2D(896, (1, 1), activation='linear', padding='same')(merged_vector)
if scale_residual:
br = Lambda(lambda x: 0.1*x)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def inception_resnet_v1_C(x_input, scale_residual=True):
global RESNET_V1_C_COUNT
RESNET_V1_C_COUNT += 1
with K.name_scope('resnet_v1_C' + str(RESNET_V1_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 1), activation='relu', padding='same')(x_input)
cr2 = Conv2D(192, (1, 3), activation='relu', padding='same')(cr2)
cr2 = Conv2D(192, (3, 1), activation='relu', padding='same')(cr2)
merged_vector = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(1792, (1, 1), activation='relu', padding='same')(merged_vector)
if scale_residual:
cr = Lambda(lambda x: 0.1*x)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
return x
def reduction_resnet_A(x_input, k=192, l=224, m=256, n=384):
with K.name_scope('reduction_resnet_A'):
ra1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), padding='valid')(x_input)
ra2 = Conv2D(n, (3, 3), activation='relu', strides=(2, 2), padding='valid')(x_input)
ra3 = Conv2D(k, (1, 1), activation='relu', padding='same')(x_input)
ra3 = Conv2D(l, (3, 3), activation='relu', padding='same')(ra3)
ra3 = Conv2D(m, (3, 3), activation='relu', strides=(2, 2), padding='valid')(ra3)
merged_vector = concatenate([ra1, ra2, ra3], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def reduction_resnet_B(x_input):
with K.name_scope('reduction_resnet_B'):
rb1 = MaxPooling2D(pool_size=(3, 3), strides=(2, 2),padding='valid')(x_input)
rb2 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb2 = Conv2D(384, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb2)
rb3 = Conv2D(256, (1, 1),activation='relu', padding='same')(x_input)
rb3 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb3)
rb4 = Conv2D(256, (1, 1), activation='relu', padding='same')(x_input)
rb4 = Conv2D(256, (3, 3), activation='relu', padding='same')(rb4)
rb4 = Conv2D(256, (3, 3), strides=(2, 2), activation='relu', padding='valid')(rb4)
merged_vector = concatenate([rb1, rb2, rb3, rb4], axis=-1)
x = BatchNormalization(axis=-1)(merged_vector)
x = Activation('relu')(x)
return x
def inception_resnet_v1_backbone(nb_classes=1000, scale=True):
x_input = Input(shape=(299, 299, 3))
# stem
x = resnet_v1_stem(x_input)
# 5 x inception_resnet_v1_A
for i in range(5):
x = inception_resnet_v1_A(x, scale_residual=False)
# reduction_resnet_A
x = reduction_resnet_A(x, k=192, l=192, m=256, n=384)
# 10 x inception_resnet_v1_B
for i in range(10):
x = inception_resnet_v1_B(x, scale_residual=True)
# Reduction B
x = reduction_resnet_B(x)
# 5 x Inception C
for i in range(5):
x = inception_resnet_v1_C(x, scale_residual=True)
# Average Pooling
x = AveragePooling2D(pool_size=(8, 8))(x)
# dropout
x = Dropout(0.2)(x)
x = Flatten()(x)
x = Dense(units=nb_classes, activation='softmax')(x)
return Model(inputs=x_input, outputs=x, name='Inception-Resnet-v1')
if __name__ == '__main__':
inception_resnet_v1_model = inception_resnet_v1_backbone()
plot_model(inception_resnet_v1_model, to_file='inception_resnet_v1.png', show_shapes=True)
4.3 inception_resnet_v2
from keras.layers import Input, add
from keras.layers.merge import concatenate
from keras.layers import Dense, Dropout, Lambda, Flatten, Activation, Conv2D
from keras.layers.convolutional import MaxPooling2D, AveragePooling2D
from keras.layers.normalization import BatchNormalization
from keras.models import Model
from inception_resnet_v1 import reduction_resnet_A
from keras.utils import plot_model
import keras.backend as K
RESNET_V2_A_COUNT = 0
RESNET_V2_B_COUNT = 0
RESNET_V2_C_COUNT = 0
def resnet_v2_stem(x_input):
'''The stem of the pure Inception-v4 and Inception-ResNet-v2 networks. This is input part of those networks.'''
# Input shape is 299 * 299 * 3 (Tensorflow dimension ordering)
with K.name_scope("stem"):
x = Conv2D(32, (3, 3), activation="relu", strides=(2, 2))(x_input) # 149 * 149 * 32
x = Conv2D(32, (3, 3), activation="relu")(x) # 147 * 147 * 32
x = Conv2D(64, (3, 3), activation="relu", padding="same")(x) # 147 * 147 * 64
x1 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x2 = Conv2D(96, (3, 3), activation="relu", strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 73 * 73 * 160
x1 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x1 = Conv2D(96, (3, 3), activation="relu")(x1)
x2 = Conv2D(64, (1, 1), activation="relu", padding="same")(x)
x2 = Conv2D(64, (7, 1), activation="relu", padding="same")(x2)
x2 = Conv2D(64, (1, 7), activation="relu", padding="same")(x2)
x2 = Conv2D(96, (3, 3), activation="relu", padding="valid")(x2)
x = concatenate([x1, x2], axis=-1) # 71 * 71 * 192
x1 = Conv2D(192, (3, 3), activation="relu", strides=(2, 2))(x)
x2 = MaxPooling2D((3, 3), strides=(2, 2))(x)
x = concatenate([x1, x2], axis=-1) # 35 * 35 * 384
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_A(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_A block which is a 35 * 35 grid module.'''
global RESNET_V2_A_COUNT
RESNET_V2_A_COUNT += 1
with K.name_scope('inception_resnet_v2_A' + str(RESNET_V2_A_COUNT)):
ar1 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar2 = Conv2D(32, (3, 3), activation="relu", padding="same")(ar2)
ar3 = Conv2D(32, (1, 1), activation="relu", padding="same")(x_input)
ar3 = Conv2D(48, (3, 3), activation="relu", padding="same")(ar3)
ar3 = Conv2D(64, (3, 3), activation="relu", padding="same")(ar3)
merged = concatenate([ar1, ar2, ar3], axis=-1)
ar = Conv2D(384, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: ar = Lambda(lambda a: a * 0.1)(ar)
x = add([x_input, ar])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_B(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_B block which is a 17 * 17 grid module.'''
global RESNET_V2_B_COUNT
RESNET_V2_B_COUNT += 1
with K.name_scope('inception_resnet_v2_B' + str(RESNET_V2_B_COUNT)):
br1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(128, (1, 1), activation="relu", padding="same")(x_input)
br2 = Conv2D(160, (1, 7), activation="relu", padding="same")(br2)
br2 = Conv2D(192, (7, 1), activation="relu", padding="same")(br2)
merged = concatenate([br1, br2], axis=-1)
br = Conv2D(1152, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: br = Lambda(lambda b: b * 0.1)(br)
x = add([x_input, br])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def inception_resnet_v2_C(x_input, scale_residual=True):
'''Architecture of Inception_ResNet_C block which is a 8 * 8 grid module.'''
global RESNET_V2_C_COUNT
RESNET_V2_C_COUNT += 1
with K.name_scope('inception_resnet_v2_C' + str(RESNET_V2_C_COUNT)):
cr1 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(192, (1, 1), activation="relu", padding="same")(x_input)
cr2 = Conv2D(224, (1, 3), activation="relu", padding="same")(cr2)
cr2 = Conv2D(256, (3, 1), activation="relu", padding="same")(cr2)
merged = concatenate([cr1, cr2], axis=-1)
cr = Conv2D(2144, (1, 1), activation="linear", padding="same")(merged)
if scale_residual: cr = Lambda(lambda c: c * 0.1)(cr)
x = add([x_input, cr])
x = BatchNormalization(axis=-1)(x)
x = Activation("relu")(x)
return x
def reduction_resnet_v2_B(x_input):
'''Architecture of a 17 * 17 to 8 * 8 Reduction_ResNet_B block.'''
with K.name_scope('reduction_resnet_v2_B'):
rbr1 = MaxPooling2D((3, 3), strides=(2, 2), padding="valid")(x_input)
rbr2 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr2 = Conv2D(384, (3, 3), activation="relu", strides=(2, 2))(rbr2)
rbr3 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr3 = Conv2D(288, (3, 3), activation="relu", strides=(2, 2))(rbr3)
rbr4 = Conv2D(256, (1, 1), activation="relu", padding="same")(x_input)
rbr4 = Conv2D(288, (3, 3), activation="relu", padding="same")(rbr4)
rbr4 = Conv2D(320, (3, 3), activation="relu", strides=(2, 2))(rbr4)
merged = concatenate([rbr1, rbr2, rbr3, rbr4], axis=-1)
rbr = BatchNormalization(axis=-1)(merged)
rbr = Activation("relu")(rbr)
return rbr
def inception_resnet_v2(nb_classes=1001, scale=True):
'''Creates the Inception_ResNet_v1 network.'''
init = Input((299, 299, 3)) # Channels last, as using Tensorflow backend with Tensorflow image dimension ordering
# Input shape is 299 * 299 * 3
x = resnet_v2_stem(init) # Output: 35 * 35 * 256
# 5 x Inception A
for i in range(5):
x = inception_resnet_v2_A(x, scale_residual=scale)
# Output: 35 * 35 * 256
# Reduction A
x = reduction_resnet_A(x, k=256, l=256, m=384, n=384) # Output: 17 * 17 * 896
# 10 x Inception B
for i in range(10):
x = inception_resnet_v2_B(x, scale_residual=scale)
# Output: 17 * 17 * 896
# Reduction B
x = reduction_resnet_v2_B(x) # Output: 8 * 8 * 1792
# 5 x Inception C
for i in range(5):
x = inception_resnet_v2_C(x, scale_residual=scale)
# Output: 8 * 8 * 1792
# Average Pooling
x = AveragePooling2D((8, 8))(x) # Output: 1792
# Dropout
x = Dropout(0.2)(x) # Keep dropout 0.2 as mentioned in the paper
x = Flatten()(x) # Output: 1792
# Output layer
output = Dense(units=nb_classes, activation="softmax")(x) # Output: 10000
model = Model(init, output, name="Inception-ResNet-v2")
return model
if __name__ == "__main__":
inception_resnet_v2_model = inception_resnet_v2()
plot_model(inception_resnet_v2_model, to_file='inception_resnet_v2.png', show_shapes=True)
InceptionV4的更多相关文章
- 【Network Architecture】Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning（转）
文章来源: https://www.cnblogs.com/shouhuxianjian/p/7786760.html Feature Extractor[Inception v4] 0. 背景 随着 ...
- Tensorflow 训练inceptionV4 并移植
安装brazel (请使用最新版的brazel 和最新版的tensorflow ,版本不匹配会出错!!!) 下载bazel-0.23 https://pan.baidu.com/s/1X ...
- (转) Awesome - Most Cited Deep Learning Papers
转自:https://github.com/terryum/awesome-deep-learning-papers Awesome - Most Cited Deep Learning Papers ...
- awesome-very-deep-learning
awesome-very-deep-learning is a curated list for papers and code about implementing and training ver ...
- 学习笔记TF032:实现Google Inception Net
Google Inception Net,ILSVRC 2014比赛第一名.控制计算量.参数量,分类性能非常好.V1,top-5错误率6.67%,22层,15亿次浮点运算,500万参数(AlexNet ...
- 读论文系列：Deep transfer learning person re-identification
读论文系列:Deep transfer learning person re-identification arxiv 2016 by Mengyue Geng, Yaowei Wang, Tao X ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现经典CNN网络GoogLeNet
前面讲了LeNet.AlexNet和Vgg,这周来讲讲GoogLeNet.GoogLeNet是由google的Christian Szegedy等人在2014年的论文<Going Deeper ...
- 【深度学习系列】用PaddlePaddle和Tensorflow实现GoogLeNet InceptionV2/V3/V4
上一篇文章我们引出了GoogLeNet InceptionV1的网络结构,这篇文章中我们会详细讲到Inception V2/V3/V4的发展历程以及它们的网络结构和亮点. GoogLeNet Ince ...
- 吐血整理：人工智能PDF中文教材资源包2.73G基本包含全部学习资料-人工智能学习书单
吐血整理:人工智能PDF中文教材资源包2.73G基本包含全部学习资料 人工智能学习书单(关注微信公众号:aibbtcom获取更多资源) 文末附百度网盘下载地址 人工神经网络与盲信号处理 人工神经网络与 ...
随机推荐
- python中那些鲜为人知的功能特性
经常逛GitHub的可能关注一个牛叉的项目,叫 What the f*ck Python! 这个项目列出了几乎所有python中那些鲜为人知的功能特性,有些功能第一次遇见时,你会冒出 what the ...
- <前端 js 实现 代码雨 >
前端 js 实现 代码雨: <!DOCTYPE html> <html lang="en"> <head> <meta charset= ...
- 性能测试之JVM的故障分析工具VisualVM
VisualVM 是随JDK一同发布的jvm诊断工具,通过插件可以扩展很多功能,插件扩展也是其精华所在. 提供了一个可视界面,用于在Java应用程序在Java虚拟机上运行时查看有关Java应用程序的详 ...
- Redis面试专题
Redis面试专题 1. 什么是redis? Redis 是一个基于内存的高性能key-value数据库. (有空再补充,有理解错误或不足欢迎指正) 2. Reids的特点 Redis本质上是一个Ke ...
- Linux 初始化系统 SystemV Upstart
System V 特点 缺点: 启动时间长,init是串行启动,只有前一个进程启动完,才会启动下一个进程 启动脚本复杂,init只是执行启动脚本,不管其他事情,脚本需要自己处理各种情况,这往往使得脚本 ...
- 九、深度优先 && 广度优先
原文地址 一.什么是"搜索"算法? 算法是作用于具体数据结构之上的,深度优先搜索算法和广度优先搜索算法都是基于"图"这种数据结构的. 因为图这种数据结构的表达能 ...
- JavaScript图形实例：窗花图案
1．窗花基本框线 设定曲线的坐标方程为: n=25; r=100; x=r/n*cos(5*θ)+r*cos(θ); y=r/n*sin(5*θ)+r*sin(θ); (0≤θ≤2π ...
- Python数据分析帮你清晰的了解整理员工们的工作效率和整体满意度
前言 本文的文字及图片来源于网络,仅供学习.交流使用,不具有任何商业用途,版权归原作者所有,如有问题请及时联系我们以作处理. 项目背景 2018年,被称为互联网的寒冬之年.无论大小公司,纷纷走上了裁员 ...
- Redis为何是单线程的
以前一直有个误区,以为:高性能服务器 一定是 多线程来实现的 原因很简单因为误区二导致的: 多线程 一定比 单线程 效率高.其实不然. redis 核心就是 如果我的数据全都在内存里,我单线程的去操作 ...
- 我是如何用 CSS 绘制各种形状的
自适应的椭圆 1.自适应的椭圆 实现方式是通过border-radius这个属性:border-radius它可以单独指定水平和垂直半径.用 / 分隔这两个值.并且该属性的值不仅可以接受长度值,还能接 ...