从协议VersionedProtocol开始4——AdminOperationsProtocol、InterTrackerProtocol、JobSubmissionProtocol、TaskUmbilicalProtocol
1、package org.apache.hadoop.mapred这四个协议都在这个包下。
2、从最简单的AdminOperationsProtocol看,
void refreshQueues() throws IOException;
void refreshNodes() throws IOException;
boolean setSafeMode(JobTracker.SafeModeAction safeModeAction) throws IOException;
3、相对于hdfs里边那几个协议,这几个协议都经常修改,不是很稳定
InterTrackerProtocol
HeartbeatResponse heartbeat(TaskTrackerStatus status,
boolean restarted,
boolean initialContact,
boolean acceptNewTasks,
short responseId)
throws IOException;
public String getFilesystemName() throws IOException;
public void reportTaskTrackerError(String taskTracker,
String errorClass,
String errorMessage) throws IOException;
TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid, int fromEventId
, int maxEvents) throws IOException;
public String getSystemDir();
public String getBuildVersion() throws IOException;
public String getVIVersion() throws IOException;
JobSubmissionProtocol
public JobID getNewJobId() throws IOException;
public JobStatus submitJob(JobID jobName, String jobSubmitDir, Credentials ts)
throws IOException;
public ClusterStatus getClusterStatus(boolean detailed) throws IOException;
public AccessControlList getQueueAdmins(String queueName) throws IOException;
public void killJob(JobID jobid) throws IOException;
public void setJobPriority(JobID jobid, String priority)
throws IOException;
public boolean killTask(TaskAttemptID taskId, boolean shouldFail) throws IOException;
public JobProfile getJobProfile(JobID jobid) throws IOException;
public JobStatus getJobStatus(JobID jobid) throws IOException;
public Counters getJobCounters(JobID jobid) throws IOException;
public TaskReport[] getMapTaskReports(JobID jobid) throws IOException;
public TaskReport[] getReduceTaskReports(JobID jobid) throws IOException;
public TaskReport[] getCleanupTaskReports(JobID jobid) throws IOException;
public TaskReport[] getSetupTaskReports(JobID jobid) throws IOException;
public String getFilesystemName() throws IOException;
public JobStatus[] jobsToComplete() throws IOException;
public JobStatus[] getAllJobs() throws IOException;
public TaskCompletionEvent[] getTaskCompletionEvents(JobID jobid
, int fromEventId, int maxEvents) throws IOException;
public String[] getTaskDiagnostics(TaskAttemptID taskId)
throws IOException;
public String getSystemDir();
public String getStagingAreaDir() throws IOException;
public JobQueueInfo[] getQueues() throws IOException;
public JobQueueInfo getQueueInfo(String queue) throws IOException;
public JobStatus[] getJobsFromQueue(String queue) throws IOException;
public QueueAclsInfo[] getQueueAclsForCurrentUser() throws IOException;
public
Token<DelegationTokenIdentifier> getDelegationToken(Text renewer
) throws IOException,
InterruptedException;
public long renewDelegationToken(Token<DelegationTokenIdentifier> token
) throws IOException,
InterruptedException;
public void cancelDelegationToken(Token<DelegationTokenIdentifier> token
) throws IOException,InterruptedException;
TaskUmbilicalProtocol
JvmTask getTask(JvmContext context) throws IOException;
boolean statusUpdate(TaskAttemptID taskId, TaskStatus taskStatus,
JvmContext jvmContext) throws IOException, InterruptedException;
void reportDiagnosticInfo(TaskAttemptID taskid, String trace,
JvmContext jvmContext) throws IOException;
void reportNextRecordRange(TaskAttemptID taskid, SortedRanges.Range range,
JvmContext jvmContext) throws IOException;
boolean ping(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void done(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void commitPending(TaskAttemptID taskId, TaskStatus taskStatus,
JvmContext jvmContext) throws IOException, InterruptedException;
boolean canCommit(TaskAttemptID taskid, JvmContext jvmContext) throws IOException;
void shuffleError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
void fsError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
void fatalError(TaskAttemptID taskId, String message, JvmContext jvmContext)
throws IOException;
MapTaskCompletionEventsUpdate getMapCompletionEvents(JobID jobId,
int fromIndex,
int maxLocs,
TaskAttemptID id,
JvmContext jvmContext)
throws IOException;
void updatePrivateDistributedCacheSizes(org.apache.hadoop.mapreduce.JobID jobId,
long[] sizes) throws IOException;
从协议VersionedProtocol开始4——AdminOperationsProtocol、InterTrackerProtocol、JobSubmissionProtocol、TaskUmbilicalProtocol的更多相关文章
- 从协议VersionedProtocol开始
VersionedProtocol协议是Hadoop的最顶层协议接口的抽象:5--3--3共11个协议,嘿嘿 1)HDFS相关 ClientDatanodeProtocol:client与datano ...
- 从协议VersionedProtocol开始3——ClientProtocol、DatanodeProtocol、NamenodeProtocol、RefreshAuthorizationPolicyProtocol、RefreshUserMappingsProtocol
1.ClientProtocol这个玩意的版本号是61L:DatanodeProtocol 是26L:NamenodeProtocol是 3L;RefreshAuthorizationPolicyPr ...
- 从协议VersionedProtocol开始2——ClientDatanodeProtocol和InterDatanodeProtocol
1.首先,我看的是hadoop1.2.1 这个里边,有点奇怪ClientDatanodeProtocol的versionID是4,但是InterDatanodeProtocol的versionID是3 ...
- 从协议VersionedProtocol开始1
Phase 0: Make a plan You must first decide what steps you're going to have in your process. It sound ...
- Hadoop系列番外篇之一文搞懂Hadoop RPC框架及细节实现
@ 目录 Hadoop RPC 框架解析 1.Hadoop RPC框架概述 1.1 RPC框架特点 1.2 Hadoop RPC框架 2.Java基础知识回顾 2.1 Java反射机制与动态代理 2. ...
- MapReduce剖析笔记之四:TaskTracker通过心跳机制获取任务的流程
上一节分析到了JobTracker把作业从队列里取出来并进行了初始化,所谓的初始化,主要是获取了Map.Reduce任务的数量,并统计了哪些DataNode所在的服务器可以处理哪些Split等等,将这 ...
- MapReduce剖析笔记之二:Job提交的过程
上一节以WordCount分析了MapReduce的基本执行流程,但并没有从框架上进行分析,这一部分工作在后续慢慢补充.这一节,先剖析一下作业提交过程. 在分析之前,我们先进行一下粗略的思考,如果要我 ...
- hadoop之JobTracker功能分析
JobTracker是整个MapReduce计算框架中的主服务,相当于集群的“管理者”,负责整个集群的作业控制和资源管理.本文对JobTracker的启动过程及心跳接收与应答两个主要功能进行分析. 1 ...
- 【Hadoop代码笔记】通过JobClient对Jobtracker的调用详细了解Hadoop RPC
Hadoop的各个服务间,客户端和服务间的交互采用RPC方式.关于这种机制介绍的资源很多,也不难理解,这里不做背景介绍.只是尝试从Jobclient向JobTracker提交作业这个最简单的客户端服务 ...
随机推荐
- features recreate的含义
就是从数据库里导出这个features的代码块.
- php imagecreatetruecolor输出字符符或验证码
$img = imagecreatetruecolor(100,100); //创建真彩图像资源 $color = imagecolorAllocate($img,200,200,200); //分配 ...
- flasCC技术点记录
[待测试特性] 一.C接口导出相关 1.重载函数. 2.虚函数. 3.template相关 二.内存相关 1.as直接往c分配的内存写数据. 2.c直接往as对象写数据. 三.C访问AS 1.访问as ...
- 日期操作类--GregorianCalendar类
GregorianCalendar--API JavaTM Platform Standard Ed. 6 GregorianCalendar类 Calendar类实现了公历日历,GregorianC ...
- DBUtils学习
1. DBUtils是JDBC的简单封装,可以和JDBC混合使用. 2. DBUtils对结果集自动封装为JavaBean是有着苛刻要求的:必须满足JavaBean的规范,其次 ...
- Android利用V4包中的SwipeRefreshLayout实现上拉加载
基本原理 上拉加载或者说滚动到底部时自动加载,都是通过判断是否滚动到了ListView或者其他View的底部,然后触发相应的操作,这里我们以 ListView来说明.因此我们需要在监听ListView ...
- ubuntu下python3安装类库
ubuntu是默认安装了python2的,所以直接使用 pip install XXX 是默认安装到python2的,安装到python3 的指令是 pip3 install XXXX 或者 pyth ...
- Ubuntu配置Open BlockChain
/* 以前只是听说过BlockChain,没怎么了解过,最近导师在搞Blockchain,让我配一下Open BlockChain的环境,于是就硬着头皮配了一下,还挺顺利的. 由于没接触过BlockC ...
- TCP与UDP
TCP(Transmission Control Protocol,传输控制协议)是面向连接的协议:可靠.保证正确性:顺序到达:流量控制.拥塞控制:重传机制.窗口机制:对系统资源.时间要求多:流模式S ...
- Tarjan--LCA算法的个人理解即模板
tarjan---LCA算法的步骤是(当dfs到节点u时): 实际: 并查集+dfs 具体步骤: 1 在并查集中建立仅有u的集合,设置该集合的祖先为u 1 对u的每个孩子v: 1.1 tarj ...