自定义Decoder继承ByteToMessageDecoder实现解码的小案例

ByteToMessageDecoder是一种ChannelInboundHandler，可以称为解码器，负责将byte字节流(ByteBuf)转换成一种Message，Message是应用可以自己定义的一种Java对象。

例如应用中使用protobuf协议，则可以将byte转换为Protobuf对象。然后交给后面的Handler来处理。

使用示例, 下面这段代码先将收到的数据按照换行符分割成一段一段的，然后将byte转换成String, 再将String转换成int, 然后把int加一后写回。

代码:

ServerBootstrap bootstrap = new ServerBootstrap();
EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
try {
    bootstrap.channel(NioServerSocketChannel.class)
            .handler(new LoggingHandler(LogLevel.DEBUG))
            .group(bossGroup, workerGroup)
            .childHandler(new ChannelInitializer<SocketChannel>() {
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LineBasedFrameDecoder(1024))
                            .addLast(new ByteToStringDecoder())
                            .addLast(new StringToIntegerDecoder())
                            .addLast(new IntegerToByteEncoder())
                            .addLast(new IntegerIncHandler());
                }
            });
    ChannelFuture bind = bootstrap.bind(8092);
    bind.sync();
    bind.channel().closeFuture().sync();
} finally {
    bossGroup.shutdownGracefully().sync();
    workerGroup.shutdownGracefully().sync();
}

这里的ChannelPipeline的组织结构是

1. ByteToStringDecoder ==> 将byte转换成String的Decoder
2. StringToIntegerDecoder ==>String转换成Integer对象的Decoder
3. IntegerToByteEncoder ==>Integer转换成byte的Encoder
4. IntegerIncHandler ==> 将接受到的int加一后返回

下面来逐一分析

ByteToStringMessageDecoder继承于ByteToMessageDecoder，并实现了ByteToMessageDecoder的

decode(ChannelHandlerContext ctx, ByteBuf in, java.util.List out)方法。</java.lang.object>decode方法实现中要求将ByteBuf中的数据进行解码然后将解码后的对象增加到list中:

public class ByteToStringDecoder extends ByteToMessageDecoder {
    protected void decode(ChannelHandlerContext channelHandlerContext, ByteBuf byteBuf, List<Object> list) throws Exception {
        byte[] data = new byte[byteBuf.readableBytes()];
        byteBuf.readBytes(data);
        list.add(new String(data, StandardCharsets.UTF_8));
    }
}

ByteToStringMessageDecoder继承于ByteToMessageDecoder，并实现了ByteToMessageDecoder的

decode(ChannelHandlerContext ctx, ByteBuf in, java.util.List out)方法。

decode方法实现中要求将ByteBuf中的数据进行解码然后将解码后的对象增加到list中

ByteToMessageDecoder

ByteToMessageDecoder继承了ChannelInboundHandlerAdapter所以是一个处理Inbound事件的Handler。

其内部保存一个Cumulator用于保存待解码的ByteBuf，然后不断调用子类需要实现的抽象方法decode去取出byte数据转换处理。

/**
     * Cumulate {@link ByteBuf}s.
     */
    public interface Cumulator {
        /**
         * Cumulate the given {@link ByteBuf}s and return the {@link ByteBuf} that holds the cumulated bytes.
         * The implementation is responsible to correctly handle the life-cycle of the given {@link ByteBuf}s and so
         * call {@link ByteBuf#release()} if a {@link ByteBuf} is fully consumed.
         */
        ByteBuf cumulate(ByteBufAllocator alloc, ByteBuf cumulation, ByteBuf in);
    }

Cumulator有两种实现，MERGE_CUMULATOR和COMPOSITE_CMUMULATOR。MERGE_CUMULATOR通过memory copy的方法将in中的数据复制写入到cumulation中。COMPOSITE_CUMULATOR采取的是类似链表的方式，没有进行memory copy, 通过一种CompositeByteBuf来实现，在某些场景下会更适合。默认采用的是MERGE_CUMULATOR。

public static final Cumulator MERGE_CUMULATOR = new Cumulator() {
        @Override
        public ByteBuf cumulate(ByteBufAllocator alloc, ByteBuf cumulation, ByteBuf in) {
            final ByteBuf buffer;
            if (cumulation.writerIndex() > cumulation.maxCapacity() - in.readableBytes()
                    || cumulation.refCnt() > 1 || cumulation.isReadOnly()) {
                // Expand cumulation (by replace it) when either there is not more room in the buffer
                // or if the refCnt is greater then 1 which may happen when the user use slice().retain() or
                // duplicate().retain() or if its read-only.
                // 如果cumulation是只读的、或者要超过capacity了，或者还有其他地方在引用, 则都通过创建一个新的byteBuf的方式来扩容ByteBuf
                buffer = expandCumulation(alloc, cumulation, in.readableBytes());
            } else {
                buffer = cumulation;
            }
            buffer.writeBytes(in);
            in.release();
            return buffer;
        }
    };

ByteToMessageDecoder中最主要的部分在channelRead处理上

1. 收到一个msg后先判断是否是ByteBuf类型，是的情况创建一个CodecOutputList(也是一种list)保存转码后的对象列表
2. 如果cumulation为null则把msg设置为cumulation，否则合并到cumulation里
3. 调用callDecode方法，尝试解码
4. finally中如果cumulation已经读完了，就release并置为null等待gc
5. 调用fireChannelRead将解码后的out传递给后面的Handler

public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        if (msg instanceof ByteBuf) {
            CodecOutputList out = CodecOutputList.newInstance();
            try {
                ByteBuf data = (ByteBuf) msg;
                first = cumulation == null;
                if (first) {
                    cumulation = data;
                } else {
                    cumulation = cumulator.cumulate(ctx.alloc(), cumulation, data);
                }
                callDecode(ctx, cumulation, out);
            } catch (DecoderException e) {
                throw e;
            } catch (Exception e) {
                throw new DecoderException(e);
            } finally {
                if (cumulation != null && !cumulation.isReadable()) {
                    numReads = 0;
                    cumulation.release();
                    cumulation = null;
                } else if (++ numReads >= discardAfterReads) {
                    // We did enough reads already try to discard some bytes so we not risk to see a OOME.
                    // See https://github.com/netty/netty/issues/4275
                    numReads = 0;
                    discardSomeReadBytes();
                }
                int size = out.size();
                decodeWasNull = !out.insertSinceRecycled();
                fireChannelRead(ctx, out, size);
                out.recycle();
            }
        } else {
            ctx.fireChannelRead(msg);
        }
    }

callDecode中不断执行抽象decode(ctx, in, out)方法直到in可读数据没有减少或当前handler被remove。

 protected void callDecode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) {
        try {
            while (in.isReadable()) {
                int outSize = out.size();
                if (outSize > 0) {
                    fireChannelRead(ctx, out, outSize);
                    out.clear();
                   // 检查当前handler是否被remove了
                    if (ctx.isRemoved()) {
                        break;
                    }
                    outSize = 0;
                }

                int oldInputLength = in.readableBytes();
                decodeRemovalReentryProtection(ctx, in, out);
                // 检查当前handler是否被remove了
                if (ctx.isRemoved()) {
                    break;
                }

                if (outSize == out.size()) {
                    if (oldInputLength == in.readableBytes()) {
                        break;
                    } else {
                        continue;
                    }
                }

                if (oldInputLength == in.readableBytes()) { // 这种情况是解码出了对象但是并没有移动in的readIndex
                    throw new DecoderException(
                            StringUtil.simpleClassName(getClass()) +
                                    ".decode() did not read anything but decoded a message.");
                }

                if (isSingleDecode()) {
                    break;
                }
            }
        } ...
    }

fireChannelRead(ctx, msgs, numElements)的处理方式是对每个解码后的消息进行fireChannelRead,交给下一个Handler处理

    static void fireChannelRead(ChannelHandlerContext ctx, List<Object> msgs, int numElements) {
        if (msgs instanceof CodecOutputList) {
            fireChannelRead(ctx, (CodecOutputList) msgs, numElements);
        } else {
            for (int i = 0; i < numElements; i++) {
                ctx.fireChannelRead(msgs.get(i));
            }
        }
    }
    static void fireChannelRead(ChannelHandlerContext ctx, CodecOutputList msgs, int numElements) {
        for (int i = 0; i < numElements; i ++) {
            ctx.fireChannelRead(msgs.getUnsafe(i));
        }
    }

以上就是ByteToMessageDecoder的主要处理部分。关于Netty，面试中会喜欢问道“粘包/拆包”问题，指的是一个消息在网络中是二进制byte流的形式传过去的，接收方如何判断一个消息是否读完、哪里是分割点等，这些可以通过Netty中提供的一些Decoder来实现，例如DelimiterBasedFrameDecoder，FixedLengthFrameDecoder, LengthFieldBasedFrameDecoder。其中最常见的应该是LengthFieldBasedFrameDecoder了，因为自定义的协议中通常会有一个协议头，里面有一个字段描述一个消息的大小长度，然后接收方就能知道消息读到什么时候是读完一个Frame了。这些解码器会在后续的文章中介绍。