class  TransportServer

 bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
RpcHandler rpcHandler = appRpcHandler;
for (TransportServerBootstrap bootstrap : bootstraps) {
rpcHandler = bootstrap.doBootstrap(ch, rpcHandler);
}
context.initializePipeline(ch, rpcHandler);
}
});

每次有新client连接完成,server都会调用匿名类的initChannel方法,在调用context.initializePipeline(ch, rpcHandler)时,通过createChannelHandler方法,创建TransportResponseHandler用于处理响应消息(Channel直接被隐藏在TransportResponseHandler中),创建TransportRequestHandler 用于处理请求消息(Channel被封装在TransportClient中,TransportClient又被隐藏在TransportRequestHandler),最后返回的TransportChannelHandler包含三者。

public TransportChannelHandler initializePipeline(
SocketChannel channel,
RpcHandler channelRpcHandler) {
try {
TransportChannelHandler channelHandler = createChannelHandler(channel, channelRpcHandler);
channel.pipeline()
.addLast("encoder", encoder)
.addLast(TransportFrameDecoder.HANDLER_NAME, NettyUtils.createFrameDecoder())
.addLast("decoder", decoder)
.addLast("idleStateHandler", new IdleStateHandler(0, 0, conf.connectionTimeoutMs() / 1000))
// NOTE: Chunks are currently guaranteed to be returned in the order of request, but this
// would require more logic to guarantee if this were not part of the same event loop.
.addLast("handler", channelHandler);
return channelHandler;
} catch (RuntimeException e) {
logger.error("Error while initializing Netty pipeline", e);
throw e;
}
}
private TransportChannelHandler createChannelHandler(Channel channel, RpcHandler rpcHandler) {
TransportResponseHandler responseHandler = new TransportResponseHandler(channel);
TransportClient client = new TransportClient(channel, responseHandler);
TransportRequestHandler requestHandler = new TransportRequestHandler(channel, client,
rpcHandler);
return new TransportChannelHandler(client, responseHandler, requestHandler,
conf.connectionTimeoutMs(), closeIdleConnections);
}
class TransportChannelHandler
//原始消息到达,根据消息类型调用不同处理方式。此处以RequestMessage消息为例
public void channelRead0(ChannelHandlerContext ctx, Message request) throws Exception {
if (request instanceof RequestMessage) {
requestHandler.handle((RequestMessage) request); //上文实例的TransportRequestHandler
} else {
responseHandler.handle((ResponseMessage) request);
}
}
class  TransportRequestHandler 

  @Override
public void handle(RequestMessage request) {
if (request instanceof ChunkFetchRequest) {
processFetchRequest((ChunkFetchRequest) request);
} else if (request instanceof RpcRequest) {
processRpcRequest((RpcRequest) request); //以rpc请求为例
} else if (request instanceof OneWayMessage) {
processOneWayMessage((OneWayMessage) request);
} else if (request instanceof StreamRequest) {
processStreamRequest((StreamRequest) request);
} else {
throw new IllegalArgumentException("Unknown request type: " + request);
}
} private void processRpcRequest(final RpcRequest req) {
try {
rpcHandler.receive(reverseClient, req.body().nioByteBuffer(), new RpcResponseCallback() { //reverseClient上文传递的值
@Override
public void onSuccess(ByteBuffer response) {
respond(new RpcResponse(req.requestId, new NioManagedBuffer(response)));
} @Override
public void onFailure(Throwable e) {
respond(new RpcFailure(req.requestId, Throwables.getStackTraceAsString(e)));
}
});
} catch (Exception e) {
logger.error("Error while invoking RpcHandler#receive() on RPC id " + req.requestId, e);
respond(new RpcFailure(req.requestId, Throwables.getStackTraceAsString(e)));
} finally {
req.body().release();
}
} class NettyRpcHandler override def receive(
client: TransportClient,
message: ByteBuffer,
callback: RpcResponseCallback): Unit = {
val messageToDispatch = internalReceive(client, message) //将client封装在messageToDispatch 中
dispatcher.postRemoteMessage(messageToDispatch, callback) //此方法是将消息封装成inbox形式,放入inbox消息队列
} internalReceive调用
val requestMessage = nettyEnv.deserialize[RequestMessage](client, message) class NettyRpcEnv
private[netty] def deserialize[T: ClassTag](client: TransportClient, bytes: ByteBuffer): T = {
NettyRpcEnv.currentClient.withValue(client) {
deserialize { () =>
javaSerializerInstance.deserialize[T](bytes) //反序列化时调用NettyRpcEndpointRef实例的readObject方法,而下面两个DynamicVariable在此处被替换为新值
}
}
} object NettyRpcEnv
//DynamicVariable可以用来在指定作用域替换数据
private[netty] val currentEnv = new DynamicVariable[NettyRpcEnv](null)
private[netty] val currentClient = new DynamicVariable[TransportClient](null) class NettyRpcEndpointRef private def readObject(in: ObjectInputStream): Unit = { //反序列化,为client、nettyEnv填充新值,而非通过网络传送过来的空值
in.defaultReadObject()
nettyEnv = NettyRpcEnv.currentEnv.value
client = NettyRpcEnv.currentClient.value
}

spark-rpc是如何实现将netty的Channel隐藏在inbox中的的更多相关文章

  1. spark2.1源码分析3:spark-rpc如何实现将netty的Channel隐藏在inbox中

    class TransportServer bootstrap.childHandler(new ChannelInitializer<SocketChannel>() { @Overri ...

  2. Spark RPC框架源码分析(一)简述

    Spark RPC系列: Spark RPC框架源码分析(一)运行时序 Spark RPC框架源码分析(二)运行时序 Spark RPC框架源码分析(三)运行时序 一. Spark rpc框架概述 S ...

  3. Spark RPC框架源码分析(二)RPC运行时序

    前情提要: Spark RPC框架源码分析(一)简述 一. Spark RPC概述 上一篇我们已经说明了Spark RPC框架的一个简单例子,Spark RPC相关的两个编程模型,Actor模型和Re ...

  4. org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse

    跑sparkPis示例程序 [root@node01 bin]# ./spark-submit --master spark://node01:7077 --class org.apache.spar ...

  5. spark RPC详解

    前段时间看spark,看着迷迷糊糊的.最近终于有点头绪,先梳理了一下spark rpc相关的东西,先记录下来. 1,概述 个人认为,如果把分布式系统(HDFS, HBASE,SPARK等)比作一个人, ...

  6. Spark RPC

    在Spark中,对于网络调用的底层封装(粘包拆包,编解码,链路管理等)都是在common/network-common包中实现的(详见[common/network-common]).在common/ ...

  7. spark 源码分析之六--Spark RPC剖析之Dispatcher和Inbox、Outbox剖析

    在上篇 spark 源码分析之五 -- Spark内置RPC机制剖析之一创建NettyRPCEnv 中,涉及到了Diapatcher 内容,未做过多的剖析.本篇来剖析一下它的工作原理. Dispatc ...

  8. spark 源码分析之八--Spark RPC剖析之TransportContext和TransportClientFactory剖析

    spark 源码分析之八--Spark RPC剖析之TransportContext和TransportClientFactory剖析 TransportContext 首先官方文档对Transpor ...

  9. spark 源码分析之十--Spark RPC剖析之TransportResponseHandler、TransportRequestHandler和TransportChannelHandler剖析

    spark 源码分析之十--Spark RPC剖析之TransportResponseHandler.TransportRequestHandler和TransportChannelHandler剖析 ...

随机推荐

  1. Toggle Slow Animations

    Toggle Slow Animations iOS Simulator has a feature that slows animations, you can toggle it either b ...

  2. md5Util

    package com.rscode.credits.util; /** * MD5加密类 * @author 12 */ import java.security.MessageDigest; pu ...

  3. 使用Angular cli创建组件报错: Unexpected token / in JSON at position....

    之前为了熟悉流程一直都是手动创建组件,今天试着用cli创建组件,居然报错了,报错大致为: Unexpected token / in JSON at position.... ,并且错误指向了.ang ...

  4. 20155208徐子涵 Exp 6 信息搜集与漏洞扫描

    20155208徐子涵 Exp 6 信息搜集与漏洞扫描 实验目的 掌握信息搜集的最基础技能与常用工具的使用方法 实验内容 (1)各种搜索技巧的应用 (2)DNS IP注册信息的查询 (3)基本的扫描技 ...

  5. 前端代理----whistle

    场景一:如何将本地的请求代理到服务器上(如果接口没有校验登陆的情况) 最简单的方法:在项目文件中找到webpack开发环境的配置文件,配置devServer对象 devServer: { conten ...

  6. 搭建开发环境3) Debian8 安装Netbeans

    1.下载最新版的Netbeans.sh(netbeans-8.2-linux.sh) 2.进入管理员账户,移动到下载的目录 ./netbeans-8.2-linux.sh //执行安装脚本 出现界面安 ...

  7. hql和sql的一些区别

    日期格式化查询的区别: selectSql.append(" AND DATE_FORMAT(o.createDate,\'%Y-%m-%d\') = :createDate"); ...

  8. robot framework中的返回值

    1.若想要再setup中有返回值,给后续的操作使用 A)在setup的关键词中需要的返回值,设置为global variable或者suit variable:如下图:但是在编译器中,会报错,但是执行 ...

  9. Evosuite使用方法入门

    ​ Evosuite使用方法入门 ​ 1.简要介绍 EvoSuite开源工具可以基于Eclipse进行测试用例的自动生成,生成的测试用例符合Junit标准(直接生成可进行Junit的java文件),满 ...

  10. java 大数运算[转]

    用JAVA 实现算术表达式(1234324234324 + 8938459043545)/5 + 343434343432.59845 因为JAVA语言中的long 定义的变量值的最大数受到限制,例如 ...