Spark job 部署模式

---

　　Spark job 的部署有两种模式，Client && Cluster
　　spark-submit .. --deploy-mode client | cluster

　　【上传 Jar 包】

[centos@s101 ~]$ hdfs dfs -put myspark.jar data

　　【Client】
　　默认值，Driver 运行在 Client 端主机上。

spark-submit --class com.share.scala.mr.TaggenCluster --master spark://s101:7077 myspark.jar /user/centos/temptags.txt

　　【cluster】
　　Driver 运行在某个 Worker 节点上。客户端值负责提交 job。

spark-submit --class com.share.scala.mr.TaggenCluster --master spark://s101:7077 --deploy-mode cluster hdfs://mycluster/user/centos/data/myspark.jar /user/centos/temptags.txt

[centos@s101 ~]$ xcall.sh jps
==================== s101 jps ===================
 Master
 NameNode
 DFSZKFailoverController
 Jps
==================== s102 jps ===================
 CoarseGrainedExecutorBackend
 Worker
 JournalNode
 DataNode
 QuorumPeerMain
 Jps
==================== s103 jps ===================
 Worker
 JournalNode
 QuorumPeerMain
 CoarseGrainedExecutorBackend
 DataNode
 Jps
==================== s104 jps ===================
 QuorumPeerMain
 DriverWrapper
 Jps
 Worker
 JournalNode
 DataNode
 CoarseGrainedExecutorBackend

[centos@s105 /soft/spark-2.1.0-bin-hadoop2.7/bin]$ ./spark-submit --class com.share.scala.mr.TaggenCluster --master spark://s101:7077 --deploy-mode cluster hdfs://s101/user/centos/data/myspark.jar /user/centos/temptags.txt

---