Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation
这是旧代码在新版本Lucene中出现的异常,异常如下:
Exception in thread "main" java.lang.IllegalStateException: TokenStream contract violation: reset()/close() call missing, reset() called multiple times, or subclass does not call super.reset(). Please see Javadocs of TokenStream class for more information about the correct consuming workflow.
at org.apache.lucene.analysis.Tokenizer$1.read(Tokenizer.java:110)
at java.io.Reader.read(Reader.java:140)
at org.wltea.analyzer.core.AnalyzeContext.fillBuffer(AnalyzeContext.java:124)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:122)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:78)
at com.hankcs.train.IKHelper.parse(IKHelper.java:36)
at com.hankcs.train.AnalysisAdjuster.handleFile(AnalysisAdjuster.java:44)
at com.hankcs.train.AnalysisAdjuster.main(AnalysisAdjuster.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)Process finished with exit code 1
旧代码:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
IKAnalyzer ss = new IKAnalyzer(); StringReader reader = new StringReader(str); try { TokenStream tokenStream = ss.tokenStream( "" , reader); while (tokenStream.incrementToken()) { CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute. class ); System.out.println(termAttribute.toString()); } } catch (IOException e) { e.printStackTrace(); } |
根据新的API文档,调用TokenStream API的流程必须是:
The workflow of the new
TokenStream
API is as follows:
Instantiation of
TokenStream
/TokenFilter
s which add/get attributes to/from theAttributeSource
.The consumer calls
reset()
.The consumer retrieves attributes from the stream and stores local references to all attributes it wants to access.
The consumer calls
incrementToken()
until it returns false consuming the attributes after each call.The consumer calls
end()
so that any end-of-stream operations can be performed.The consumer calls
close()
to release any resource when finished using theTokenStream
.
所以代码必须在incrementToken()
之前调用一次reset()
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
IKAnalyzer ss = new IKAnalyzer(); StringReader reader = new StringReader(str); try { TokenStream tokenStream = ss.tokenStream( "" , reader); tokenStream.reset(); while (tokenStream.incrementToken()) { CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute. class ); System.out.println(termAttribute.toString()); } } catch (IOException e) { e.printStackTrace(); } |
转载请注明:码农场 » Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation
Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation的更多相关文章
- Lucene分词报错:”TokenStream contract violation: close() call missing”
Lucene使用IKAnalyzer分词时报错:”TokenStream contract violation: close() call missing” 解决办法是每次完成后必须调用关闭方法. ...
- ES failed to notify ClusterStateListener java.lang.IllegalStateException: environment is not locked
ES出现异常: failed to notify ClusterStateListenerjava.lang.IllegalStateException: environment is not loc ...
- myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).
把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...
- java.lang.IllegalStateException:Couldn't read row 0, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.xxx...}: java.lang.IllegalSta ...
- java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead
java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...
- java.lang.IllegalStateException: getOutputStream() has already been called for this response
ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...
- 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this response
1. 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this ...
- eclipse启动报错java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' befo
报错: java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invo ...
- java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data fr
Android中操作Sqlite遇到的错误:java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. ...
随机推荐
- 手动创建一个Spring Boot 2.x项目
spring boot 2.1.9版本quick start参考文档地址:https://docs.spring.io/spring-boot/docs/2.1.9.RELEASE/reference ...
- flask之日志的配置
1. 项目中,日志和配置文件都是单独在一个文件夹中,一般log文件夹和config文件夹,两个文件夹和manage.py在同一个目录下. 2. 配置日志前,先给flask装上script脚本扩展,Fl ...
- Debian10+OpenMediaVault(OMV)安装
前言:测试打造NAS平台,以下是步骤. 安装Debian10 注:请下载amd64,不要下载i836平台,因为OMV外挂插件不支持I836所以不建议用i836,如只使用官方插件可以无视 安装前-安装, ...
- Jackson常用工具类
原创转载请注明出处:https://www.cnblogs.com/agilestyle/p/11983194.html Demo package org.fool.util; import com. ...
- 6398. 【NOIP2018模拟10.30】Generator(树状数组区间修改)
题目描述 Description Input Output 输出 q 行,第 i 行表示数据 Di 的答案. Sample Input 4 3 2 1 1 2 4 2 1 2 1 1 3 5 2 2 ...
- Comet OJ - Contest #6 C 一道树题 数学 + 推导
Code: #include <bits/stdc++.h> #define setIO(s) freopen(s".in","r",stdin) ...
- apicloud直接上传图片
function getPicture() { api.confirm({ title : "提示", msg : "选择图片", buttons : [&qu ...
- EQS 自定义Context 如何用Testing Pawn 进行测试?
比如自定义了一个玩家的Context, 那么需要把这个玩家直接放置到场景中 在Context中override Provide Single Actor函数,按类型获取所有的Actor,其中第一个作为 ...
- 小程序cover-view
cover-view包裹的元素设置定位,元素内容长短会影响cover-view的位置,即使设置的left,top一致 最佳解决方法,就是给cover-view设置宽度
- 前端学习记录 week 1
前端学习记录 week 1 基础知识 CSS盒模型 所有HTML元素可以看作盒子,在CSS中,"box model"这一术语是用来设计和布局时使用.CSS盒模型本质上是一个盒子,封 ...