TensorFlow Frontend前端

TensorFlow前端有助于将TensorFlow模型导入TVM。

Supported versions:

1.12 and below

Tested models:

Inception (V1/V2/V3/V4)
Resnet (All)
Mobilenet (V1/V2 All)
Vgg (16/19)
BERT (Base/3-layer)

Preparing a Model for Inference准备推理模型

Remove Unneeded Nodes删除不需要的节点

导出过程将删除许多不需要进行推理的节点，但不幸的是会留下一些剩余的节点。应该手动删除的节点：

Dropout, including Dropout and DropoutWrapper
Assert

Convert None Dimensions to Constants将无尺寸Dimensions转换为常数

TVM对动态张量形状的支持最少。None应将尺寸替换为常量。例如，模型可以接受带有shape的输入(None,20)。这应转换为的形状(1,20)。应该相应地修改模型，以确保这些形状在整个图形中都匹配。

Export

TensorFlow前端需要冻结的protobuf（.pb）或保存的模型作为输入。不支持检查点（.ckpt）。TensorFlow前端所需的graphdef，可以从活动会话中提取，可以使用TFParser帮助器类提取。

应该导出该模型并进行许多转换，以准备模型进行推理。设置`add_shapes=True`也很重要，因为这会将每个节点的输出形状嵌入到图形中。这是一个给定会话将模型导出为protobuf的函数：

import tensorflow as tf

from tensorflow.tools.graph_transforms import TransformGraph

def export_pb(session):

with tf.gfile.GFile("myexportedmodel.pb", "wb") as f:

inputs = ["myinput1", "myinput2"] # replace with your input names

outputs = ["myoutput1"] # replace with your output names

graph_def = session.graph.as_graph_def(add_shapes=True)

graph_def = tf.graph.util.convert_variables_to_constants(session, graph_def, outputs)

graph_def = TransformGraph(

graph_def,

inputs,

outputs,

[

"remove_nodes(op=Identity, op=CheckNumerics, op=StopGradient)",

"sort_by_execution_order", # sort by execution order after each transform to ensure correct node ordering

"remove_attribute(attribute_name=_XlaSeparateCompiledGradients)",

"remove_attribute(attribute_name=_XlaCompile)",

"remove_attribute(attribute_name=_XlaScope)",

"sort_by_execution_order",

"remove_device",

"sort_by_execution_order",

"fold_batch_norms",

"sort_by_execution_order",

"fold_old_batch_norms",

"sort_by_execution_order"

]

)

f.write(graph_def.SerializeToString())

Another method is to export and freeze the graph.

Import the Model

Explicit Shape:

确保可以在整个图形中知道形状，将`shape`参数传递给`from_tensorflow`。该词典将输入名称映射到输入形状。

Data Layout

大多数TensorFlow模型以NHWC布局发布。NCHW布局通常提供更好的性能，尤其是在GPU上。该TensorFlow前端可以通过传递参数自动转换模型的数据布局`layout='NCHW'`到`from_tensorflow`。

Best Practices

使用静态张量形状代替动态形状（删除`None`尺寸）。
`TensorArray`目前尚不支持使用静态RNN代替动态RNN。

Supported Ops

Abs
Add
AddN
All
Any
ArgMax
ArgMin
AvgPool
BatchMatMul
BatchMatMulV2
BatchNormWithGlobalNormalization
BatchToSpaceND
BiasAdd
BroadcastTo
Cast
Ceil
CheckNumerics
ClipByValue
Concat
ConcatV2
Conv2D
Cos
Tan
CropAndResize
DecodeJpeg
DepthwiseConv2dNative
DepthToSpace
Dilation2D
Equal
Elu
Enter
Erf
Exit
Exp
ExpandDims
Fill
Floor
FloorDiv
FloorMod
FusedBatchNorm
FusedBatchNormV2
Gather
GatherNd
GatherV2
Greater
GreaterEqual
Identity
IsFinite
IsInf
IsNan
LeakyRelu
LeftShift
Less
LessEqual
Log
Log1p
LoopCond
LogicalAnd
LogicalOr
LogicalNot
LogSoftmax
LRN
LSTMBlockCell
MatMul
Max
MaxPool
Maximum
Mean
Merge
Min
Minimum
MirrorPad
Mod
Mul
Neg
NextIteration
NotEqual
OneHot
Pack
Pad
PadV2
Pow
Prod
Range
Rank
RealDiv
Relu
Relu6
Reshape
ResizeBilinear
ResizeBicubic
ResizeNearestNeighbor
ReverseV2
RightShift
Round
Rsqrt
Select
Selu
Shape
Sigmoid
Sign
Sin
Size
Slice
Softmax
Softplus
SpaceToBatchND
SpaceToDepth,
Split
SplitV
Sqrt
Square
SquareDifference
Squeeze
StridedSlice
Sub
Sum
Switch
Tanh
TensorArrayV3
TensorArrayScatterV3
TensorArrayGatherV3
TensorArraySizeV3
TensorArrayWriteV3
TensorArrayReadV3
TensorArraySplitV3
TensorArrayConcatV3
Tile
TopKV2
Transpose
TruncateMod
Unpack
UnravelIndex
Where
ZerosLike