Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli2/Option
今天,在做canopy算法实例时,遇到这个问题,所以记录下来。下面是源码:
package czx.com.mahout; import java.io.IOException; import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat;
import org.apache.hadoop.util.ToolRunner;
import org.apache.mahout.common.AbstractJob;
import org.apache.mahout.math.RandomAccessSparseVector;
import org.apache.mahout.math.VectorWritable; public class TextVecWrite extends AbstractJob { public static void main(String[] args) throws Exception {
ToolRunner.run(new Configuration(), new TextVecWrite(), args);
} /**
* TextVecWriterMapper
* @author czx
*;
*/
public static class TextVecWriterMapper extends Mapper<LongWritable, Text, LongWritable,VectorWritable >{
@SuppressWarnings("unchecked")
@Override
protected void map(LongWritable key, Text value,
@SuppressWarnings("rawtypes") org.apache.hadoop.mapreduce.Mapper.Context context)
throws IOException, InterruptedException {
String[] split = value.toString().split("\\s{1,}");
RandomAccessSparseVector vector = new RandomAccessSparseVector(split.length);
for(int i=0;i<split.length;++i){
vector.set(i, Double.parseDouble(split[i]));
}
VectorWritable vectorWritable = new VectorWritable(vector);
context.write(key, vectorWritable);
}
} /**
* TextVectorWritableReducer
* @author czx
*
*/
public static class TextVectorWritableReducer extends Reducer<LongWritable, VectorWritable, LongWritable , VectorWritable >{
@Override
protected void reduce(LongWritable arg0, Iterable<VectorWritable> arg1,
Context arg2)
throws IOException, InterruptedException {
for(VectorWritable v:arg1){
arg2.write(arg0, v);
}
}
} @Override
public int run(String[] arg0) throws Exception {
addInputOption();
addOutputOption();
if(parseArguments(arg0)==null){
return -1;
}
Path input = getInputPath();
Path output = getOutputPath();
Configuration conf = getConf(); Job job = new Job(conf,"textvectorWritable with input:"+input.getName());
job.setOutputFormatClass(SequenceFileOutputFormat.class);
job.setMapperClass(TextVecWriterMapper.class);
job.setReducerClass(TextVectorWritableReducer.class);
job.setMapOutputKeyClass(LongWritable.class);
job.setOutputValueClass(VectorWritable.class);
job.setOutputKeyClass(LongWritable.class);
job.setOutputValueClass(VectorWritable.class);
job.setJarByClass(TextVecWrite.class); FileInputFormat.addInputPath(job, input);
SequenceFileOutputFormat.setOutputPath(job, output); if(!job.waitForCompletion(true)){
throw new InterruptedException("Canopy Job failed processing "+input);
}
return 0;
}
}
将程序编译打包成JAR并运行如下:
hadoop jar ClusteringUtils.jar czx.com.mahout.TextVecWrite -i /user/hadoop/testdata/synthetic_control.data -o /home/czx/1
但出现如下错误:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli2/Option
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.util.RunJar.main(RunJar.java:205)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli2.Option
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 3 more
最后,发现是将mahout根目录下的相应的jar包复制到hadoop-2.4.1/share/hadoop/common/lib文件夹下时,少复制了mahout-core-0.9-job.jar,于是复制mahout-core-0.9-job.jar后,重新启动hadoop即可。
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli2/Option的更多相关文章
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
学习架构探险,从零开始写Java Web框架时,在学习到springAOP时遇到一个异常: "C:\Program Files\Java\jdk1.7.0_40\bin\java" ...
- MyEclipse8.5集成Tomcat7时的启动错误:Exception in thread “main” java.lang.NoClassDefFoundError org/apache/commons/logging/LogFactory
今天,安装Tomcat7.0.21后,单独用D:\apache-tomcat-7.0.21\bin\startup.bat启动web服务正常.但在MyEclipse8.5中集成配置Tomcat7后,在 ...
- MyEclipse8.5集成Tomcat7时的启动错误:Exception in thread “main” java.lang.NoClassDefFoundError org/apache/commons/logging/LogFactory
今天,安装Tomcat7.0.21后,单独用D:\apache-tomcat-7.0.21\bin\startup.bat启动web服务正常.但 在MyEclipse8.5中集成配置Tomcat7后, ...
- 【转】MyEclipse8.5集成Tomcat7时的启动错误:Exception in thread “main” java.lang.NoClassDefFoundError org/apache/commons/logging/LogFactory
http://www.cnblogs.com/newsouls/p/4021198.html 今天,安装Tomcat7.0.21后,单独用D:\apache-tomcat-7.0.21\bin\sta ...
- shiro报错SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".和Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
未能加载类"org.slf4j.impl.StaticLoggerBinder" 解决方案: <dependency> <groupId>org.slf4j ...
- spark-shell报错:Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream
环境: openSUSE42.2 hadoop2.6.0-cdh5.10.0 spark1.6.0-cdh5.10.0 按照网上的spark安装教程安装完之后,启动spark-shell,出现如下报错 ...
- 报错:Exception in thread "main" java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/FileSystem
报错现象: Exception in thread "main" java.lang.NoClassDefFoundError: Lorg/apache/hadoop/fs/Fil ...
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/CanUnbuffer
在执行spark on hive 的时候在 sql.show()处报错 : Exception in thread "main" java.lang.NoClassDefFoun ...
- Exception in thread main java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFacto
报错: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/juli/logging/Log ...
随机推荐
- Arduino连接MPU6050陀螺仪
一.线路连接 Arduino MPU6050 VCC 3.3V/5V GND GND SCL A5 SDA A4 INT D2 二.库下载 https://pan.baidu.com/s/1nvt75 ...
- kafka2.12_1.0.1生产者示范代码
import java.util.Properties;import kafka.javaapi.producer.Producer;import kafka.producer.KeyedMessag ...
- 对JavaScript 引擎基础:原型优化的研究 -----------------------引用
一.优化层级与执行效率的取舍 介绍了现代 JavaScript 引擎通用的工作流程: 我们也指出,尽管从高级抽象层面来看,引擎之间的处理流程都很相似,但他们在优化流程上通常都存在差异.为什么呢?为什么 ...
- linux 配置内网yum源
一.yum服务器端配置1.安装FTP软件#yum install vsftpd #service vsftpd start#chkconfig --add vsftpd#chkconfig vsftp ...
- BZOJ 4373: 算术天才⑨与等差数列 线段树
Description 算术天才⑨非常喜欢和等差数列玩耍. 有一天,他给了你一个长度为n的序列,其中第i个数为a[i]. 他想考考你,每次他会给出询问l,r,k,问区间[l,r]内的数从小到大排序后能 ...
- sublime的python开发环境搭建
至少很长一段时间内,我个人用的一直是pycharm,也感觉挺好用的,也没啥大毛病 但是pycharm确实有点笨重,啥功能都有,但是有很多可能这辈子我也不会用到,并且pycharm打开的速度确实不敢恭维 ...
- Vue.js 使用 Font Awesome 小图标
1.安装 Font Awesome npm i --save @fortawesome/fontawesome-svg-core npm i --save @fortawesome/free-soli ...
- Python3学习笔记(七):字典
在python中,有一种通过名字来引用值的数据结构,这种类型的数据结构成为映射. 字典是Python中唯一内建的映射类型,具有以下特点: 字典中的值是无序的 值存在特定的键(key)下 键(key)可 ...
- python3学习笔记(三):注释和字符串
一.注释 为了让别人能够更容易理解程序,使用注释是非常有效的,即使是自己回头再看旧代码也是一样. # 打印圆的周长: print(2* pi* r) 在python 中用井号(#)表示注释.井号(#) ...
- go语言系列--前言
我为什么要学golang语言 绝不是一时兴起,也不是人云亦云,这是我规划了很久的事了. 我曾自学过C语言,C++语言,Python语言,可都学的不精,原因我想是不知道为了什么而学的,可是这就是缺少学习 ...