spark在yarn-cluster上面执行报错

在单机模式下执行成功的spark程序，在yarn上面就报错。异常信息如下：

 // :: INFO DAGScheduler: Completed ResultTask(, )
 // :: INFO DAGScheduler: Stage  (saveAsTextFile at FileUtil.scala:) finished in 0.179 s
 // :: INFO SparkContext: Job finished: saveAsTextFile at FileUtil.scala:, took 0.331739293 s
 // :: INFO TaskSetManager: Finished TID  in  ms on localhost (progress: /)
 // :: INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool
 // :: INFO ApplicationMaster: finishApplicationMaster with SUCCEEDED
 Exception in thread "main" java.lang.AssertionError: assertion failed
     at scala.Predef$.assert(Predef.scala:)
     at org.apache.spark.deploy.yarn.ApplicationMaster.waitForSparkContextInitialized(ApplicationMaster.scala:)
     at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:)
     at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$main$.apply$mcV$sp(ApplicationMaster.scala:)
     at org.apache.spark.deploy.SparkHadoopUtil$$anon$.run(SparkHadoopUtil.scala:)
     at org.apache.spark.deploy.SparkHadoopUtil$$anon$.run(SparkHadoopUtil.scala:)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
     at org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:)
     at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:)
     at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
 // :: INFO ApplicationMaster: AppMaster received a signal.
 // :: INFO ApplicationMaster: Deleting staging directory .sparkStaging/application_1408004797389_0007

从日志上面分析，job执行成功了。但貌似是没有获取到yarn返回的结果信息。

debug后发现是下面的问题：

spark-submit --class org.andy.hadoop.ETL --master yarn-cluster  ../lib/rdbms-0.0.-SNAPSHOT-jar-with-dependencies.jar /dest/ETL2

job以yarn-cluster形式执行，但代码中初始化的为：

  var conf = new SparkConf().setAppName("testFilter").setMaster("local")
  var sc = new SparkContext(conf)

以local的形式初始化的。所以接收不到yarn的返回结果。修改后：

     var conf = new SparkConf().setAppName("testFilter").setMaster("yarn-cluster")
     var sc = new SparkContext(conf)

执行成功！