//统计单词top10def main(args: Array[String]): Unit = { val conf = new SparkConf().setAppName("tst").setMaster("local[3]") val sc = new SparkContext(conf) //wc val res = sc.textFile("D:\\test\\spark\\urlCount").flatMap(_.split(&qu
val path = "/usr/data/lfw-a/*" val rdd = sc.wholeTextFiles(path) val first = rdd.first println(first) val files = rdd.map { case (fileName, content) => fileName.replace("file:", "") } println(files.first)println(files.coun
问题描述1 使用spark-shell ,sc.textFile("hdfs://test02.com:8020/tmp/w").count 出现如下异常: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) at org.apache.hadoop.util.Refle