Lucene分词报错:”TokenStream contract violation: close() call missing”
Lucene使用IKAnalyzer分词时报错:”TokenStream contract violation: close() call missing” 解决办法是每次完成后必须调用关闭方法。
如果报错:java.lang.illegalstateexception: tokenstream contract violation: reset()/close() call missing,则要在tokenStream.incrementToken(),原因是lucene从4.6.0开始tokenstream使用方法更改的问题,在使用incrementtoken方法前必须调用reset方法,详见api http://lucene.apache.org/core/4_6_0/core/index.html 。
以下正确示例代码(第10行和22行调用reset()和close()方法):
public Set<String> slicing(String text){
Set<String> result = new HashSet<>();
StringReader reader = null;
TokenStream tokenStream = null;
try {
reader = new StringReader(text);
tokenStream = analyzer.tokenStream("", reader);
CharTermAttribute charTermAttribute = tokenStream.getAttribute(CharTermAttribute.class);
OffsetAttribute offsetAttribute = tokenStream.addAttribute(OffsetAttribute.class);
tokenStream.reset();
while (tokenStream.incrementToken()) {
int startOffset = offsetAttribute.startOffset();
int endOffset = offsetAttribute.endOffset();
if((endOffset - startOffset) > 1){
String term = charTermAttribute.toString();
result.add(term);
}
}
} catch (IOException e) {
e.printStackTrace();
} finally{
IOs.close(tokenStream, reader);
}
return result;
}
http://www.lizi.pw/archives/56
org.wltea.analyzer.lucene.IKAnalyzer
Exception in thread "main" java.lang.IllegalStateException: 词典尚未初始化,请先调用initial方法
at org.wltea.analyzer.dic.Dictionary.getSingleton(Dictionary.java:137)
at org.wltea.analyzer.core.CJKSegmenter.analyze(CJKSegmenter.java:80)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:116)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:88)
Lucene分词报错:”TokenStream contract violation: close() call missing”的更多相关文章
- Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation
这是旧代码在新版本Lucene中出现的异常,异常如下: Exception in thread "main" java.lang.IllegalStateException: To ...
- mysql报错sql injection violation, syntax error: syntax error, expect RPAREN, actual IDENTIFIER
mysql报错sql injection violation, syntax error: syntax error, expect RPAREN, actual IDENTIFIER 处理,在控制台 ...
- 修改umask后apache报错:because search permissions are missing on a component of the path,
0.修改umask后apache报错:because search permissions are missing on a component of the path, 1.ls -lrth ./h ...
- Jfinal报错sql injection violation, multi-statement not allow
Jfinal报错: com.jfinal.plugin.activerecord.ActiveRecordException: java.sql.SQLException: sql injection ...
- 安卓模拟器的报错This AVD's configuration is missing a kernel file!!
安卓模拟器的报错: 可能的原因是target设置问题:
- idea中 参数没有描述报错 @param XX tag description is missing错误,去除黄色警告
最近在使用idea开发工具,在方法备注中参数没有描述报错就会报一些黄色警告: @param XX tag description is missing,下面展示去除黄色警告的方法 File--sett ...
- [已解决]报错:Required request body is missing
问题代码: res = requests.post(getXxxxList_url, headers=headers, data={}) 对象网站: angular4 apache 通过验证 (coo ...
- DELLR720服务器更换硬盘,启动系统报错:there are offline or missing virtual drivers with preserved cache
linux系统启动过程中给出错误: There are offline or missing virtual drives with preserved cache. Please check the ...
- Glibc编译报错:*** These critical programs are missing or too old: as ld gcc
Binutils版本升级 这里是binutils版本过低导致, 查看已部署版本 上传离线升级包 [root@sdw1 glibc]# tar -zxvf binutils-2.32.tar.gz [r ...
随机推荐
- JS学习笔记 - cookie设置、读取、删除
<script> // 设置cookie function setCookie(name, value, iDay) { var oDate = new Date(); oDate.set ...
- 3、在编译过程中出现no space left on device
原因:通过df -h查看发现磁盘空间不错 删掉不需要的文件后执行sudo apt-get clean
- axios采坑之路
POST请求设置Content-Type 由于后端采用的是form表单形式上送参数,需要设置Content-Type axios设置如下 const _axios = axios.create(con ...
- POJ 2590 Steps (ZOJ 1871)
http://poj.org/problem?id=2590 http://acm.zju.edu.cn/onlinejudge/showProblem.do?problemCode=1871 题目大 ...
- 8、linux下输入子系统
input_sync(button_dev); /*通知接收者,一个报告发送完毕*/ 参考:http://www.51hei.com/bbs/dpj-27652-1.html 很详细说明 in ...
- 【BZOJ 3675】[Apio2014]序列分割
[链接] 链接 [题意] 在这里输入题意 [题解] 模拟一下样例. 会发现.切的顺序不影响最后的答案. 只要切点确定了. 答案就确定了. 则设f[i][j]表示前i段,第i段保留到j的最大值. \(f ...
- UVALive - 4960 Sensor network(生成树+LCA)
题目大意:给出N个点.M条边.问这N个点形成的生成树的最大权值边-最小权值边的最小值 解题思路:先排序,然后按生成树的kruscal算法进行加边,再维护一个最小权值边 加边的时候要考虑一下加下去的边是 ...
- HDoj-1163- Digital Roots
Problem Description The digital root of a positive integer is found by summing the digits of the int ...
- SpringBoot学习:获取yml和properties配置文件的内容(转)
项目下载地址:http://download.csdn.net/detail/aqsunkai/9805821 (一)yml配置文件: pom.xml加入依赖: <!-- 支持 @Configu ...
- Linux文件编辑命令具体整理
刚接触Linux,前几天申请了个免费体验的阿里云server,选择的是Ubuntu系统.配置jdk环境变量的时候须要编辑文件. vi命令编辑文件,百度了一下,非常多回答不是非常全面,因此编辑文件话了一 ...