[root@hadoop1 bin]# ./spark-submit --class myprojectpackaging.App /usr/local/hadoop/spark-2.2.-bin-hadoop2./mycode/myprojectname/target/myprojectname-1.0-SNAPSHOT.jar
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/spark-2.2.-bin-hadoop2./jars/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/hadoop-2.6./share/hadoop/common/lib/slf4j-log4j12-1.7..jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Hello World!
// :: INFO spark.SparkContext: Running Spark version 2.2.
// :: WARN util.Utils: Your hostname, hadoop1 resolves to a loopback address: 127.0.0.1; using 192.168.2.51 instead (on interface eno1)
// :: WARN util.Utils: Set SPARK_LOCAL_IP if you need to bind to another address
// :: INFO spark.SparkContext: Submitted application: MongoSparkConnectorIntro
// :: INFO spark.SecurityManager: Changing view acls to: root
// :: INFO spark.SecurityManager: Changing modify acls to: root
// :: INFO spark.SecurityManager: Changing view acls groups to:
// :: INFO spark.SecurityManager: Changing modify acls groups to:
// :: INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
// :: INFO util.Utils: Successfully started service 'sparkDriver' on port .
// :: INFO spark.SparkEnv: Registering MapOutputTracker
// :: INFO spark.SparkEnv: Registering BlockManagerMaster
// :: INFO storage.BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
// :: INFO storage.BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
// :: INFO storage.DiskBlockManager: Created local directory at /tmp/blockmgr-a5531bf0-ae40-44be-9a91-a2aca6c81267
// :: INFO memory.MemoryStore: MemoryStore started with capacity 366.3 MB
// :: INFO spark.SparkEnv: Registering OutputCommitCoordinator
// :: INFO util.log: Logging initialized @1291ms
// :: INFO server.Server: jetty-9.3.z-SNAPSHOT
// :: INFO server.Server: Started @1353ms
// :: INFO server.AbstractConnector: Started ServerConnector@6548bb7d{HTTP/1.1,[http/1.1]}{0.0.0.0:}
// :: INFO util.Utils: Successfully started service 'SparkUI' on port .
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@7668d560{/jobs,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@438bad7c{/jobs/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4fdf8f12{/jobs/job,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6979efad{/jobs/job/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@4a67318f{/stages,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@17f9344b{/stages/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@54e81b21{/stages/stage,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@50cf5a23{/stages/stage/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@273c947f{/stages/pool,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@1af1347d{/stages/pool/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@20765ed5{/storage,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2899a8db{/storage/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@c1a4620{/storage/rdd,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@130a0f66{/storage/rdd/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@12365c88{/environment,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@2237bada{/environment/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5710768a{/executors,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@6e0d4a8{/executors/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@{/executors/threadDump,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@5bf61e67{/executors/threadDump/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@b273a59{/static,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@30865a90{/,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@777c9dc9{/api,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@22175d4f{/jobs/job/kill,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@3b809711{/stages/stage/kill,null,AVAILABLE,@Spark}
// :: INFO ui.SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.2.51:4040
// :: INFO spark.SparkContext: Added JAR file:/usr/local/hadoop/spark-2.2.-bin-hadoop2./mycode/myprojectname/target/myprojectname-1.0-SNAPSHOT.jar at spark://192.168.2.51:56112/jars/myprojectname-1.0-SNAPSHOT.jar with timestamp 1512096533656
// :: INFO executor.Executor: Starting executor ID driver on host localhost
// :: INFO util.Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port .
// :: INFO netty.NettyBlockTransferService: Server created on 192.168.2.51:
// :: INFO storage.BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
// :: INFO storage.BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManagerMasterEndpoint: Registering block manager 192.168.2.51: with 366.3 MB RAM, BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO storage.BlockManager: Initialized BlockManager: BlockManagerId(driver, 192.168.2.51, , None)
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@44c5a16f{/metrics/json,null,AVAILABLE,@Spark}
// :: INFO internal.SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir ('file:/usr/local/hadoop/spark-2.2.0-bin-hadoop2.7/bin/spark-warehouse/').
// :: INFO internal.SharedState: Warehouse path is 'file:/usr/local/hadoop/spark-2.2.0-bin-hadoop2.7/bin/spark-warehouse/'.
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@70e3f36f{/SQL,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@23e44287{/SQL/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@61f3fbb8{/SQL/execution,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@432034a{/SQL/execution/json,null,AVAILABLE,@Spark}
// :: INFO handler.ContextHandler: Started o.s.j.s.ServletContextHandler@173373b4{/static/sql,null,AVAILABLE,@Spark}
// :: INFO state.StateStoreCoordinatorRef: Registered StateStoreCoordinator endpoint
Exception in thread "main" java.lang.NoClassDefFoundError: com/mongodb/spark/MongoSpark
at myprojectpackaging.App.main(App.java:)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:)
at java.lang.reflect.Method.invoke(Method.java:)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: com.mongodb.spark.MongoSpark
at java.net.URLClassLoader.findClass(URLClassLoader.java:)
at java.lang.ClassLoader.loadClass(ClassLoader.java:)
at java.lang.ClassLoader.loadClass(ClassLoader.java:)
... more
// :: INFO spark.SparkContext: Invoking stop() from shutdown hook
// :: INFO server.AbstractConnector: Stopped Spark@6548bb7d{HTTP/1.1,[http/1.1]}{0.0.0.0:}
// :: INFO ui.SparkUI: Stopped Spark web UI at http://192.168.2.51:4040
// :: INFO spark.MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
// :: INFO memory.MemoryStore: MemoryStore cleared
// :: INFO storage.BlockManager: BlockManager stopped
// :: INFO storage.BlockManagerMaster: BlockManagerMaster stopped
// :: INFO scheduler.OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
// :: INFO spark.SparkContext: Successfully stopped SparkContext
// :: INFO util.ShutdownHookManager: Shutdown hook called
// :: INFO util.ShutdownHookManager: Deleting directory /tmp/spark-94c896a5-3f94-48ce-92ff-2e5b94204d0b

mvn 引入自定义jar 解决 mongo-spark 报错的更多相关文章

  1. larave5.6 引入自定义函数库时,报错不能重复定义

    方法一:使用function_exists判断 方法二:使用命名空间 namespace test; function test(){ echo 'test/test'; } namespace te ...

  2. idea lib下有jar包但是仍然报错 找不到类

    现象: idea lib下有jar包但是仍然报错 找不到类 但是有个奇怪现象 同样的配置下项目在eclipse中可以正常编译 启动. package com.puhui.car.aspect; imp ...

  3. maven仓库中有jar包pom还报错

    maven仓库中有jar包pom还报错 就报错,咋啦? 这个包来源不明,自己拷贝进来的吧?你当我mvn是傻子?我要去网上验证一下: 我自己有个_remote.respositories文件,如果自己用 ...

  4. 解决:MySQL 报错:1045 - Access denied for user 'root'@'localhost'(using password YES)

    一.前言 今年疯狂迷上了开源,只要看到好的开源项目,就会不顾一切一股脑扎进去研究,五一期间发现一个很好的关于众筹的开源项目,但不巧,这个项目竟然是 PHP 写的,没学过 PHP,自然对这个开源项目毫无 ...

  5. IDEA导出jar包后运行报错 找不到或无法加载主类

    开发工具:IDEA16 运行环境:ubuntu 问题:根据网上的Idea导出jar包的方法,将我的项目导出jar包后运行报错:找不到或无法加载主类.   为了找到这个原因,我重新搭建了一个测试例子,在 ...

  6. spark报错处理

    Spark报错处理 1.问题:org.apache.spark.SparkException: Exception thrown in awaitResult 分析:出现这个情况的原因是spark启动 ...

  7. 解决kylin查询报错:org.apache.kylin.rest.exception.InternalErrorException

    报错信息: -- ::, ERROR [Query 12e9c054-760c---b1f06724c9b6-] service.QueryService: : Exception when exec ...

  8. homestead虚拟机,通过npm下载依赖包和解决运行gulp报错问题 yarn出错问题

    homestead虚拟机,通过npm下载依赖包和解决运行gulp报错问题 yarn出错问题 1. 在虚拟器运行 npm 下载依赖组件时报错: npm ERR! EPROTO: protocol err ...

  9. spark报错:invalid token

    启动spark报错,启动container失败,去看yarn的日志,显示invalid token, 经过排查是hadoop子节点的配置和主节点的配置不一致导致的,同步之后,问题解决.

随机推荐

  1. 30分钟学会如何使用Shiro(转自:http://www.cnblogs.com/learnhow/p/5694876.html)

    本篇内容大多总结自张开涛的<跟我学Shiro>原文地址:http://jinnianshilongnian.iteye.com/blog/2018936 我并没有全部看完,只是选择了一部分 ...

  2. 【转】Python + Android + Uiautomator自动化测试

    1.首先来介绍下UIAutomator工具 UIAutomator是Android官方推出的安卓应用界面自动化测试工具,是最理想的针对APK进行自动化功能回归测试的利器. 2.UIAutomator测 ...

  3. 【强化学习】百度云BCC openai gym 环境配置

    openai gym是RL算法的测试床.为了直观和便于调试,我们需要可视化界面.然而远程服务器使用openGL的问题比较棘手,这篇博客记录在百度云BCC上配置openai gym环境的注意事项. 按照 ...

  4. 货车运输(codevs 3287)

    题目描述 Description A 国有 n 座城市,编号从 1 到 n,城市之间有 m 条双向道路.每一条道路对车辆都有重量限制,简称限重.现在有 q 辆货车在运输货物,司机们想知道每辆车在不超过 ...

  5. 【Codevs1907】方格取数3(最小割)

    题意:在一个有m*n 个方格的棋盘中,每个方格中有一个正整数.现要从方格中取数,使任意2 个数所在方格没有公共边,且取出的数的总和最大.试设计一个满足要求的取数算法. n,m<=30 思路:如果 ...

  6. 【Git】Git 本地的撤销修改和删除操作

    一:撤销操作 比如我现在在readme.txt文件里面增加一行 内容为555555555555,我们先通过命令查看如下: 在我未提交之前,我发现添加5555555555555内容有误,所以我得马上恢复 ...

  7. Permission denied: user=administrator, access=WRITE, inode="/":root:supergroup:drwxr-xr-x

    参考博文:http://blog.csdn.net/xiaoshunzi111/article/details/52062640 http://www.udpwork.com/item/7047.ht ...

  8. Yii2之创建定时任务

    yii开发的项目需要使用定时任务其实也可以使用一些单独的脚本文件来完成,但若是定时任务代码中需要使用到项目中的一些类,特别是需要使用应用对象Yii::$app的时候,单独的脚本想要完成就比较麻烦了.这 ...

  9. elasticsearch入门使用(五) kibana&x-pack安装使用

    Kibana User Guide 一.UI安装 https://www.elastic.co/downloads/kibana 下载rpm直接运行即可 二.参数配置 find / -name kib ...

  10. python socket非阻塞及python队列Queue

    一. python非阻塞编程的settimeout与setblocking+select 原文:www.th7.cn/Program/Python/201406/214922.shtml 侧面认证Py ...