Hadoop Ecosystem related ports

本文总结了Hadoop生态系统中各个组件使用的端口，包括了HDFS，Map Reduce，HBase，Hive，Spark，WebHCat，Impala，Alluxio，Sqoop等，后续会持续更新。

HDFS Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?	Configuration Parameters
NameNode WebUI	Master Nodes (NameNode and any back-up NameNodes)		http	Web UI to look at current status of HDFS, explore file system	Yes (Typically admins, Dev/Support teams)	dfs.http.address
			https	Secure http service		dfs.https.address
NameNode metadata service	Master Nodes (NameNode and any back-up NameNodes)	8020/9000	IPC	File system metadata operations	Yes (All clients who directly need to interact with the HDFS)	Embedded in URI specified by fs.default.name
DataNode	All Slave Nodes		http	DataNode WebUI to access the status, logs etc.	Yes (Typically admins, Dev/Support teams)	dfs.datanode.http.address
			https	Secure http service		dfs.datanode.https.address
				Data transfer		dfs.datanode.address
			IPC	Metadata operations	No	dfs.datanode.ipc.address
Secondary NameNode	Secondary NameNode and any backup Secondary NameNode		http	Checkpoint for NameNode metadata	No	dfs.secondary.http.address

Map Reduce Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?	Configuration Parameters
JobTracker  WebUI	Master Nodes (JobTracker Node and any back-up Job­Tracker node )		http	Web UI for JobTracker	Yes	mapred.job.tracker.http.address
JobTracker	Master Nodes (JobTracker Node)		IPC	For job submissions	Yes (All clients who need to submit the MapReduce jobs  including Hive, Hive server, Pig)	Embedded in URI specified by mapred.job.tracker
Task­Tracker Web UI and Shuffle	All Slave Nodes		http	DataNode Web UI to access status, logs, etc.	Yes (Typically admins, Dev/Support teams)	mapred.task.tracker.http.address
History Server WebUI			http	Web UI for Job History	Yes	mapreduce.history.server.http.address

HBase Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?	Configuration Parameters
HMaster	Master Nodes (HBase Master Node and any back-up HBase Master node)				Yes	hbase.master.port
HMaster Info Web UI	Master Nodes (HBase master Node and back up HBase Master node if any)		http	The port for the HBase­Master web UI. Set to -1 if you do not want the info server to run.	Yes	hbase.master.info.port
Region Server	All Slave Nodes				Yes (Typically admins, dev/support teams)	hbase.regionserver.port
Region Server	All Slave Nodes		http		Yes (Typically admins, dev/support teams)	hbase.regionserver.info.port
	All ZooKeeper Nodes			Port used by ZooKeeper peers to talk to each other.Seehere for more information.	No	hbase.zookeeper.peerport
	All ZooKeeper Nodes			Port used by ZooKeeper peers to talk to each other.Seehere for more information.		hbase.zookeeper.leaderport
				Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect.		hbase.zookeeper.property.clientPort

Hive Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?	Configuration Parameters
Hive Server2	Hive Server machine (Usually a utility machine)		thrift	Service for programatically (Thrift/JDBC) connecting to Hive	Yes (Clients who need to connect to Hive either programatically or through UI SQL tools that use JDBC)	ENV Variable HIVE_PORT
Hive Metastore			thrift		Yes (Clients that run Hive, Pig and potentially M/R jobs that use HCatalog)	hive.metastore.uris

WebHCat Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?
WebHCat Server	Any utility machine		http	Web API on top of HCatalog and other Hadoop services	Yes

Spark Ports：

Service	Servers	Default Ports Used	Description
Spark GUI	Nodes running spark		Spark web interface for monitoring and troubleshooting

Impala Ports：

Service	Servers	Default Ports Used	Description
Impala Daemon	Nodes running impala daemon		Used by transmit commands and receive results by impala-shell
Impala Daemon	Nodes running impala daemon		Used by applications through JDBC
Impala Daemon	Nodes running impala daemon		Impala web interface for monitoring and troubleshooting
Impala StateStore Daemon	Nodes running impala StateStore daemon		StateStore web interface for monitoring and troubleshooting
Impala Catalog Daemon	Nodes running impala catalog daemon		Catalog service web interface for monitoring and troubleshooting

Alluxio Ports：

Service	Servers	Default Ports Used	Protocol	Description	Need End User Access?
Alluxio Web GUI	Any utility machine		http	Web GUI to check alluxio status	Yes
Alluxio API	Any utility machine		Tcp	Api to access data on alluxio	No

Sqoop Ports：

Service	Servers	Default Ports Used	Description
Sqoop server	Nodes running Sqoop		Used by Sqoop client to access the sqoop server