[root@hadoop1 bin]# ./spark-submit --class myprojectpackaging.App /usr/local/hadoop/spark-2.2.-bin-hadoop2./mycode/myprojectname/target/myprojectname-1.0-SNAPSHOT.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/spark-2.2.-bin-hadoop2./jars/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/hadoop-2.6./share/hadoop/common/lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Hello World!
// :: INFO spark.SparkContext: Running Spark version 2.2.
// :: WARN util.Utils: Your hostname, hadoop1 resolves to a loopback address: 127.0.0.1; using 192.168.2.51 instead (on interface eno1)
// :: WARN util.Utils: Set SPARK_LOCAL_IP if you need to bind to another address
// :: INFO spark.SparkContext: Submitted application: MongoSparkConnectorIntro
// :: INFO spark.SecurityManager: Changing view acls to: root
// :: INFO spark.SecurityManager: Changing modify acls to: root
// :: INFO spark.SecurityManager: Changing view acls groups to:
// :: INFO spark.SecurityManager: Changing modify acls groups to:
// :: INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
// :: INFO util.Utils: Successfully started service 'sparkDriver' on port .
// :: INFO spark.SparkEnv: Registering MapOutputTracker
// :: INFO spark.SparkEnv: Registering BlockManagerMaster
// :: INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
// :: INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
// :: INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-a5531bf0-ae40-44be-9a91-a2aca6c81267
// :: INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
// :: INFO spark.SparkEnv: Registering OutputCommitCoordinator
// :: INFO util.log: Logging initialized @1291ms
// :: INFO server.Server: jetty-9.3.z-SNAPSHOT
// :: INFO server.Server: Started @1353ms
// :: INFO server.AbstractConnector: Started ServerConnector@6548bb7d{HTTP/1.1,[http/1.1]}{0.0.0.0:}
// :: INFO util.Utils: Successfully started service 'SparkUI' on port .
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7668d560{/jobs,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@438bad7c{/jobs/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4fdf8f12{/jobs/job,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6979efad{/jobs/job/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4a67318f{/stages,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@17f9344b{/stages/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@54e81b21{/stages/stage,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@50cf5a23{/stages/stage/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@273c947f{/stages/pool,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1af1347d{/stages/pool/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@20765ed5{/storage,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2899a8db{/storage/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c1a4620{/storage/rdd,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@130a0f66{/storage/rdd/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@12365c88{/environment,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2237bada{/environment/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5710768a{/executors,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6e0d4a8{/executors/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@{/executors/threadDump,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5bf61e67{/executors/threadDump/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b273a59{/static,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30865a90{/,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@777c9dc9{/api,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22175d4f{/jobs/job/kill,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b809711{/stages/stage/kill,null,AVAILABLE,@Spark}
// :: INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.2.51:4040
// :: INFO spark.SparkContext: Added JAR file:/usr/local/hadoop/spark-2.2.-bin-hadoop2./mycode/myprojectname/target/myprojectname-1.0-SNAPSHOT.jar at spark://192.168.2.51:56112/jars/myprojectname-1.0-SNAPSHOT.jar with timestamp 1512096533656
// :: INFO executor.Executor: Starting executor ID driver on host localhost
// :: INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port .
// :: INFO netty.NettyBlockTransferService: Server created on 192.168.2.51:
// :: INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
// :: INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.2.51: with 366.3 MB RAM, BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44c5a16f{/metrics/json,null,AVAILABLE,@Spark}
// :: INFO internal.SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/usr/local/hadoop/spark-2.2.0-bin-hadoop2.7/bin/spark-warehouse/').
// :: INFO internal.SharedState: Warehouse path is 'file:/usr/local/hadoop/spark-2.2.0-bin-hadoop2.7/bin/spark-warehouse/'.
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70e3f36f{/SQL,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@23e44287{/SQL/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@61f3fbb8{/SQL/execution,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@432034a{/SQL/execution/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@173373b4{/static/sql,null,AVAILABLE,@Spark}
// :: INFO state.StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint
Exception in thread "main" java.lang.NoClassDefFoundError: com/mongodb/spark/MongoSpark
at myprojectpackaging.App.main(App.java:)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:)
at java.lang.reflect.Method.invoke(Method.java:)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.mongodb.spark.MongoSpark
at java.net.URLClassLoader.findClass(URLClassLoader.java:)
at java.lang.ClassLoader.loadClass(ClassLoader.java:)
at java.lang.ClassLoader.loadClass(ClassLoader.java:)
... more
// :: INFO spark.SparkContext: Invoking stop() from shutdown hook
// :: INFO server.AbstractConnector: Stopped Spark@6548bb7d{HTTP/1.1,[http/1.1]}{0.0.0.0:}
// :: INFO ui.SparkUI: Stopped Spark web UI at http://192.168.2.51:4040
// :: INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
// :: INFO memory.MemoryStore: MemoryStore cleared
// :: INFO storage.BlockManager: BlockManager stopped
// :: INFO storage.BlockManagerMaster: BlockManagerMaster stopped
// :: INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
// :: INFO spark.SparkContext: Successfully stopped SparkContext
// :: INFO util.ShutdownHookManager: Shutdown hook called
// :: INFO util.ShutdownHookManager: Deleting directory /tmp/spark-94c896a5-3f94-48ce-92ff-2e5b94204d0b

mvn 引入自定义jar 解决 mongo-spark 报错的更多相关文章

  1. larave5.6 引入自定义函数库时,报错不能重复定义

    方法一:使用function_exists判断 方法二:使用命名空间 namespace test; function test(){ echo 'test/test'; } namespace te ...

  2. idea lib下有jar包但是仍然报错 找不到类

    现象: idea lib下有jar包但是仍然报错 找不到类 但是有个奇怪现象 同样的配置下项目在eclipse中可以正常编译 启动. package com.puhui.car.aspect; imp ...

  3. maven仓库中有jar包pom还报错

    maven仓库中有jar包pom还报错 就报错,咋啦? 这个包来源不明,自己拷贝进来的吧?你当我mvn是傻子?我要去网上验证一下: 我自己有个_remote.respositories文件,如果自己用 ...

  4. 解决:MySQL 报错:1045 - Access denied for user 'root'@'localhost'(using password YES)

    一.前言 今年疯狂迷上了开源,只要看到好的开源项目,就会不顾一切一股脑扎进去研究,五一期间发现一个很好的关于众筹的开源项目,但不巧,这个项目竟然是 PHP 写的,没学过 PHP,自然对这个开源项目毫无 ...

  5. IDEA导出jar包后运行报错 找不到或无法加载主类

    开发工具:IDEA16 运行环境:ubuntu 问题:根据网上的Idea导出jar包的方法,将我的项目导出jar包后运行报错:找不到或无法加载主类.   为了找到这个原因,我重新搭建了一个测试例子,在 ...

  6. spark报错处理

    Spark报错处理 1.问题:org.apache.spark.SparkException: Exception thrown in awaitResult 分析:出现这个情况的原因是spark启动 ...

  7. 解决kylin查询报错:org.apache.kylin.rest.exception.InternalErrorException

    报错信息: -- ::, ERROR [Query 12e9c054-760c---b1f06724c9b6-] service.QueryService: : Exception when exec ...

  8. homestead虚拟机,通过npm下载依赖包和解决运行gulp报错问题 yarn出错问题

    homestead虚拟机,通过npm下载依赖包和解决运行gulp报错问题 yarn出错问题 1. 在虚拟器运行 npm 下载依赖组件时报错: npm ERR! EPROTO: protocol err ...

  9. spark报错:invalid token

    启动spark报错,启动container失败,去看yarn的日志,显示invalid token, 经过排查是hadoop子节点的配置和主节点的配置不一致导致的,同步之后,问题解决.

随机推荐

  1. cf468C Hack it!

    Little X has met the following problem recently. Let's define f(x) as the sum of digits in decimal r ...

  2. ElasticSearch聚合入门(续)

    主要理解聚合中的terms. 参考:http://www.cnblogs.com/xing901022/p/4947436.html Terms聚合 记录有多少F,多少M { "size&q ...

  3. .net面试题汇总-第二篇

    本篇主要关注下,.net面试题中经常用的算法问题 1.有一群猴子,它们每天要吃桃子,它们第一天吃的数量是总量的一半再多一个,第二天吃的是第一天剩下的一半再多一个,第三天吃的是第二天剩下的一半多一个,以 ...

  4. Codeforces Round #268 (Div. 2) D. Two Sets [stl - set + 暴力]

    8161957                 2014-10-10 06:12:37     njczy2010     D - Two Sets             GNU C++     A ...

  5. sql 注入 及 in 注入

    原文地址: http://www.cnblogs.com/lzrabbit/archive/2012/04/22/2465313.html 除了校验参数内容,过滤长度和sql关键字. 解决in条件拼接 ...

  6. xgboost调参

    The overall parameters have been divided into 3 categories by XGBoost authors: General Parameters: G ...

  7. NBUT 1457 Sona (莫队算法)

    题目大意: 求一段区间内 出现的数字的次数的三次方的和 思路分析: 这要水过去的题目真是难,各种优化. 不能用map , 要离散化之后 先处理lowerbound. 优化输入. . . 时间卡的非常紧 ...

  8. C语言的代码内存布局具体解释

    一个程序本质上都是由 BSS 段.data段.text段三个组成的.这种概念在当前的计算机程序设计中是非常重要的一个基本概念,并且在嵌入式系统的设计中也非常重要,牵涉到嵌入式系统执行时的内存大小分配, ...

  9. People seldom do what they believe in. They do what is convenient, then repent.

    People seldom do what they believe in. They do what is convenient, then repent. 人们很少真正实践他们的理想.他们只做比较 ...

  10. 系统重装 WIN7如何创建和使用VHD文件

    1 在磁盘管理中,点击操作-创建VHD,然后可以创建一个空的VHD文件   2 右击这个磁盘,点击初始化磁盘,然后可以新建简单卷   3 右击这个磁盘,设置为脱机或者联机就可以在计算机中显示和隐藏这个 ...