class  TransportServer

 bootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
RpcHandler rpcHandler = appRpcHandler;
for (TransportServerBootstrap bootstrap : bootstraps) {
rpcHandler = bootstrap.doBootstrap(ch, rpcHandler);
}
context.initializePipeline(ch, rpcHandler);
}
});

每次有新client连接完成,server都会调用匿名类的initChannel方法,在调用context.initializePipeline(ch, rpcHandler)时,通过createChannelHandler方法,创建TransportResponseHandler用于处理响应消息(Channel直接被隐藏在TransportResponseHandler中),创建TransportRequestHandler 用于处理请求消息(Channel被封装在TransportClient中,TransportClient又被隐藏在TransportRequestHandler),最后返回的TransportChannelHandler包含三者。

public TransportChannelHandler initializePipeline(
SocketChannel channel,
RpcHandler channelRpcHandler) {
try {
TransportChannelHandler channelHandler = createChannelHandler(channel, channelRpcHandler);
channel.pipeline()
.addLast("encoder", encoder)
.addLast(TransportFrameDecoder.HANDLER_NAME, NettyUtils.createFrameDecoder())
.addLast("decoder", decoder)
.addLast("idleStateHandler", new IdleStateHandler(0, 0, conf.connectionTimeoutMs() / 1000))
// NOTE: Chunks are currently guaranteed to be returned in the order of request, but this
// would require more logic to guarantee if this were not part of the same event loop.
.addLast("handler", channelHandler);
return channelHandler;
} catch (RuntimeException e) {
logger.error("Error while initializing Netty pipeline", e);
throw e;
}
}
private TransportChannelHandler createChannelHandler(Channel channel, RpcHandler rpcHandler) {
TransportResponseHandler responseHandler = new TransportResponseHandler(channel);
TransportClient client = new TransportClient(channel, responseHandler);
TransportRequestHandler requestHandler = new TransportRequestHandler(channel, client,
rpcHandler);
return new TransportChannelHandler(client, responseHandler, requestHandler,
conf.connectionTimeoutMs(), closeIdleConnections);
}
class TransportChannelHandler
//原始消息到达,根据消息类型调用不同处理方式。此处以RequestMessage消息为例
public void channelRead0(ChannelHandlerContext ctx, Message request) throws Exception {
if (request instanceof RequestMessage) {
requestHandler.handle((RequestMessage) request); //上文实例的TransportRequestHandler
} else {
responseHandler.handle((ResponseMessage) request);
}
}
class  TransportRequestHandler 

  @Override
public void handle(RequestMessage request) {
if (request instanceof ChunkFetchRequest) {
processFetchRequest((ChunkFetchRequest) request);
} else if (request instanceof RpcRequest) {
processRpcRequest((RpcRequest) request); //以rpc请求为例
} else if (request instanceof OneWayMessage) {
processOneWayMessage((OneWayMessage) request);
} else if (request instanceof StreamRequest) {
processStreamRequest((StreamRequest) request);
} else {
throw new IllegalArgumentException("Unknown request type: " + request);
}
} private void processRpcRequest(final RpcRequest req) {
try {
rpcHandler.receive(reverseClient, req.body().nioByteBuffer(), new RpcResponseCallback() { //reverseClient上文传递的值
@Override
public void onSuccess(ByteBuffer response) {
respond(new RpcResponse(req.requestId, new NioManagedBuffer(response)));
} @Override
public void onFailure(Throwable e) {
respond(new RpcFailure(req.requestId, Throwables.getStackTraceAsString(e)));
}
});
} catch (Exception e) {
logger.error("Error while invoking RpcHandler#receive() on RPC id " + req.requestId, e);
respond(new RpcFailure(req.requestId, Throwables.getStackTraceAsString(e)));
} finally {
req.body().release();
}
} class NettyRpcHandler override def receive(
client: TransportClient,
message: ByteBuffer,
callback: RpcResponseCallback): Unit = {
val messageToDispatch = internalReceive(client, message) //将client封装在messageToDispatch 中
dispatcher.postRemoteMessage(messageToDispatch, callback) //此方法是将消息封装成inbox形式,放入inbox消息队列
} internalReceive调用
val requestMessage = nettyEnv.deserialize[RequestMessage](client, message) class NettyRpcEnv
private[netty] def deserialize[T: ClassTag](client: TransportClient, bytes: ByteBuffer): T = {
NettyRpcEnv.currentClient.withValue(client) {
deserialize { () =>
javaSerializerInstance.deserialize[T](bytes) //反序列化时调用NettyRpcEndpointRef实例的readObject方法,而下面两个DynamicVariable在此处被替换为新值
}
}
} object NettyRpcEnv
//DynamicVariable可以用来在指定作用域替换数据
private[netty] val currentEnv = new DynamicVariable[NettyRpcEnv](null)
private[netty] val currentClient = new DynamicVariable[TransportClient](null) class NettyRpcEndpointRef private def readObject(in: ObjectInputStream): Unit = { //反序列化,为client、nettyEnv填充新值,而非通过网络传送过来的空值
in.defaultReadObject()
nettyEnv = NettyRpcEnv.currentEnv.value
client = NettyRpcEnv.currentClient.value
}

spark-rpc是如何实现将netty的Channel隐藏在inbox中的的更多相关文章

  1. spark2.1源码分析3:spark-rpc如何实现将netty的Channel隐藏在inbox中

    class TransportServer bootstrap.childHandler(new ChannelInitializer<SocketChannel>() { @Overri ...

  2. Spark RPC框架源码分析(一)简述

    Spark RPC系列: Spark RPC框架源码分析(一)运行时序 Spark RPC框架源码分析(二)运行时序 Spark RPC框架源码分析(三)运行时序 一. Spark rpc框架概述 S ...

  3. Spark RPC框架源码分析(二)RPC运行时序

    前情提要: Spark RPC框架源码分析(一)简述 一. Spark RPC概述 上一篇我们已经说明了Spark RPC框架的一个简单例子,Spark RPC相关的两个编程模型,Actor模型和Re ...

  4. org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse

    跑sparkPis示例程序 [root@node01 bin]# ./spark-submit --master spark://node01:7077 --class org.apache.spar ...

  5. spark RPC详解

    前段时间看spark,看着迷迷糊糊的.最近终于有点头绪,先梳理了一下spark rpc相关的东西,先记录下来. 1,概述 个人认为,如果把分布式系统(HDFS, HBASE,SPARK等)比作一个人, ...

  6. Spark RPC

    在Spark中,对于网络调用的底层封装(粘包拆包,编解码,链路管理等)都是在common/network-common包中实现的(详见[common/network-common]).在common/ ...

  7. spark 源码分析之六--Spark RPC剖析之Dispatcher和Inbox、Outbox剖析

    在上篇 spark 源码分析之五 -- Spark内置RPC机制剖析之一创建NettyRPCEnv 中,涉及到了Diapatcher 内容,未做过多的剖析.本篇来剖析一下它的工作原理. Dispatc ...

  8. spark 源码分析之八--Spark RPC剖析之TransportContext和TransportClientFactory剖析

    spark 源码分析之八--Spark RPC剖析之TransportContext和TransportClientFactory剖析 TransportContext 首先官方文档对Transpor ...

  9. spark 源码分析之十--Spark RPC剖析之TransportResponseHandler、TransportRequestHandler和TransportChannelHandler剖析

    spark 源码分析之十--Spark RPC剖析之TransportResponseHandler.TransportRequestHandler和TransportChannelHandler剖析 ...

随机推荐

  1. 【JMeter】生成报告-Dashboard Report

    Dashboard Report 用于生成HTML页面格式图形化报告 1.在JMmeter性能测试结束时,自动生成本次测试的HTML图形化报告 2.使用一个已有的结果文件(如CSV)来生成该次的HTM ...

  2. JS 获取图片标签和所有的图片中的src的正则表达式

    var imgReg = /<img.*?(?:>|\/>)/gi; //匹配所有img标签的正则表达式规则 var srcReg = /src=[\'\"]?([^\'\ ...

  3. struts1和struts2和springMVC的区别和介绍

    MVC是web开发常用的模式,M即模型层(Model):主要由javabean来实现.V即视图层(View):主要由jsp.velocity.freemarker等.C即控制层(Controller) ...

  4. 小程序使用animation实现跑马灯

    html: <view class="marquee"> <view class="content"> <text>{{te ...

  5. react+umi+dva+antd中dva的数据流图解

  6. JPA、Hibernate框架、通用mapper之间的关系及通用mapper的具体实现

    JPA是描述对象-关系表的映射关系,将运行期实体对象持久化到数据库中,提出以面向对象方式操作数据库的思想. Hibernate框架核心思想是ORM-实现自动的关系映射.缺点:由于关联操作提出Hql语法 ...

  7. Java中如何拆分字符串为字符数组

    题目:输入一串字符,由(){}[]组成,判断是否所有的括号都是闭括号,是的返回TRUE,不是返回FALSE. /*输入字符串,拆解为字符数组 * 用函数s.charAt(i)来完成 * * */imp ...

  8. mysql 两张表的数据设置主外健关联删除

    image_group 主表 image 副表 alter table image add constraint fk_group_idforeign key (group_id)references ...

  9. vue使用axios 时 后台接收不到数据

    后台用django 时,默认接收的数据格式为formdata ,前端如果传了其他格式会出现后台收不到参数的情况. 前端参数转 fromdata 代码如下 let formData = new Form ...

  10. robotframework中的清除输入框输入值

    业务需求 当该输入框输入之后,联动某一个按钮高亮,输入框为空的时候,该按钮置灰 需要将输入框清空,清空的办法 1.直接将输入框赋值为${empty} 如:input Text ${loactor} $ ...