这是旧代码在新版本Lucene中出现的异常,异常如下:

Exception in thread "main" java.lang.IllegalStateException: TokenStream contract violation: reset()/close() call missing, reset() called multiple times, or subclass does not call super.reset(). Please see Javadocs of TokenStream class for more information about the correct consuming workflow.
at org.apache.lucene.analysis.Tokenizer$1.read(Tokenizer.java:110)
at java.io.Reader.read(Reader.java:140)
at org.wltea.analyzer.core.AnalyzeContext.fillBuffer(AnalyzeContext.java:124)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:122)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:78)
at com.hankcs.train.IKHelper.parse(IKHelper.java:36)
at com.hankcs.train.AnalysisAdjuster.handleFile(AnalysisAdjuster.java:44)
at com.hankcs.train.AnalysisAdjuster.main(AnalysisAdjuster.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Process finished with exit code 1

旧代码:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
IKAnalyzer ss = new IKAnalyzer();
        StringReader reader = new StringReader(str);
        try
        {
            TokenStream tokenStream = ss.tokenStream("", reader);
            while (tokenStream.incrementToken())
            {
                CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute.class);
                System.out.println(termAttribute.toString());
 
            }
        catch (IOException e)
        {
            e.printStackTrace();
        }

根据新的API文档,调用TokenStream API的流程必须是:

The workflow of the new TokenStream API is as follows:

  1. Instantiation of TokenStream/TokenFilters which add/get attributes to/from the AttributeSource.

  2. The consumer calls reset().

  3. The consumer retrieves attributes from the stream and stores local references to all attributes it wants to access.

  4. The consumer calls incrementToken() until it returns false consuming the attributes after each call.

  5. The consumer calls end() so that any end-of-stream operations can be performed.

  6. The consumer calls close() to release any resource when finished using the TokenStream.

所以代码必须在incrementToken()之前调用一次reset()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
  IKAnalyzer ss = new IKAnalyzer();
        StringReader reader = new StringReader(str);
        try
        {
            TokenStream tokenStream = ss.tokenStream("", reader);
            tokenStream.reset();
            while (tokenStream.incrementToken())
            {
                CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute.class);
                System.out.println(termAttribute.toString());
 
            }
        catch (IOException e)
        {
            e.printStackTrace();
        }

转载请注明:码农场 » Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation

Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation的更多相关文章

  1. Lucene分词报错:”TokenStream contract violation: close() call missing”

    Lucene使用IKAnalyzer分词时报错:”TokenStream contract violation: close() call missing”  解决办法是每次完成后必须调用关闭方法. ...

  2. ES failed to notify ClusterStateListener java.lang.IllegalStateException: environment is not locked

    ES出现异常: failed to notify ClusterStateListenerjava.lang.IllegalStateException: environment is not loc ...

  3. myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).

    把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...

  4. java.lang.IllegalStateException:Couldn't read row 0, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.

    java.lang.RuntimeException: Unable to start activity ComponentInfo{com.xxx...}: java.lang.IllegalSta ...

  5. java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

    java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...

  6. java.lang.IllegalStateException: getOutputStream() has already been called for this response

    ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...

  7. 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this response

    1. 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this ...

  8. eclipse启动报错java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' befo

    报错: java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invo ...

  9. java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data fr

    Android中操作Sqlite遇到的错误:java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. ...

随机推荐

  1. 前端每日实战:11# 视频演示如何用纯 CSS 创作一个荧光脉冲 loader 特效

    效果预览 按下右侧的"点击预览"按钮在当前页面预览,点击链接全屏预览. https://codepen.io/zhang-ou/pen/erRzzR 可交互视频教程 此视频是可以交 ...

  2. CCF201412-2 Z字形扫描 java(100分)

    试题编号: 201412-2 试题名称: Z字形扫描 时间限制: 2.0s 内存限制: 256.0MB 问题描述: 问题描述 在图像编码的算法中,需要将一个给定的方形矩阵进行Z字形扫描(Zigzag ...

  3. 【LuoguP3270】[JLOI2016] 成绩比较

    题目链接 题目描述 G系共有n位同学,M门必修课.这N位同学的编号为0到N-1的整数,其中B神的编号为0号.这M门必修课编号为0到M-1的整数.一位同学在必修课上可以获得的分数是1到Ui中的一个整数. ...

  4. git clone项目失败,Host key verification failed.

    在码云上创建了一个项目,配置好公钥后,克隆到我本地出现以下失败 百度了好久也没有找到解决办法,困扰了好久,后来还是百度到了, 原来是在提示 ey fingerprint is SHA256:FQGC9 ...

  5. iView栅格的使用

    一般情况下栅格系统都会把每行row分为12列,但是iview是采用了24栅格系统,将区域进行24等分 基础用法 实例代码: <template> <Row> <Col s ...

  6. Linux学习-NFS服务

    一.NFS服务相关介绍 1.NFS简介 NFS (Network File System) 网络文件系统,基于内核的文件系统.Sun公司开发,通过使用NFS,用户和程序可以像访问本地文件一样访问远端系 ...

  7. js自执行函数前加个分号是什么意思?

    1.(function(){alert("1")})()(function(){alert("2")})()报错 2.(function(){alert(&qu ...

  8. django快速搭建blog

    python版本:3.5.4: Django版本:2.0 创建项目 创建mysite项目和 blog应用: django-admin startproject mysite # 创建mysite项目 ...

  9. getCurrentPages

    解释:getCurrentPages 全局函数用于获取当前页面栈的实例,以数组形式按栈的顺序给出,第一个元素为首页,最后一个元素为当前页面. 示例: // index.jsPage({ onShow( ...

  10. 获取第几个class

    假如类名为a,如果是同级(兄弟元素)的,如:<ul><li class='a'></li><li class='a'></li></u ...