java 使用CRF遇到的问题汇总
1、libCRFPP.so放在idea项目 resources下,打jar包时打在jar中。
jar包工具类
/*
* Class NativeUtils is published under the The MIT License:
*
* Copyright (c) 2012 Adam Heinrich <adam@adamh.cz>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in all
* copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
package cz.adamh.utils; import java.io.*; /**
* A simple library class which helps with loading dynamic libraries stored in the
* JAR archive. These libraries usualy contain implementation of some methods in
* native code (using JNI - Java Native Interface).
*
* @see http://adamheinrich.com/blog/2012/how-to-load-native-jni-library-from-jar
* @see https://github.com/adamheinrich/native-utils
*
*/
public class NativeUtils { /**
* Private constructor - this class will never be instanced
*/
private NativeUtils() {
} /**
* Loads library from current JAR archive
*
* The file from JAR is copied into system temporary directory and then loaded. The temporary file is deleted after exiting.
* Method uses String as filename because the pathname is "abstract", not system-dependent.
*
* @param path The path of file inside JAR as absolute path (beginning with '/'), e.g. /package/File.ext
* @throws IOException If temporary file creation or read/write operation fails
* @throws IllegalArgumentException If source file (param path) does not exist
* @throws IllegalArgumentException If the path is not absolute or if the filename is shorter than three characters (restriction of {@see File#createTempFile(java.lang.String, java.lang.String)}).
*/
public static void loadLibraryFromJar(String path) throws IOException { if (!path.startsWith("/")) {
throw new IllegalArgumentException("The path has to be absolute (start with '/').");
} // Obtain filename from path
String[] parts = path.split("/");
String filename = (parts.length > 1) ? parts[parts.length - 1] : null; // Split filename to prexif and suffix (extension)
String prefix = "";
String suffix = null;
if (filename != null) {
parts = filename.split("\\.", 2);
prefix = parts[0];
suffix = (parts.length > 1) ? "."+parts[parts.length - 1] : null; // Thanks, davs! :-)
} // Check if the filename is okay
if (filename == null || prefix.length() < 3) {
throw new IllegalArgumentException("The filename has to be at least 3 characters long.");
} // Prepare temporary file
File temp = File.createTempFile(prefix, suffix);
temp.deleteOnExit(); if (!temp.exists()) {
throw new FileNotFoundException("File " + temp.getAbsolutePath() + " does not exist.");
} // Prepare buffer for data copying
byte[] buffer = new byte[1024];
int readBytes; // Open and check input stream
InputStream is = NativeUtils.class.getResourceAsStream(path);
if (is == null) {
throw new FileNotFoundException("File " + path + " was not found inside JAR.");
} // Open output stream and copy data between source file in JAR and the temporary file
OutputStream os = new FileOutputStream(temp);
try {
while ((readBytes = is.read(buffer)) != -1) {
os.write(buffer, 0, readBytes);
}
} finally {
// If read/write fails, close streams safely before throwing an exception
os.close();
is.close();
} // Finally, load the library
System.load(temp.getAbsolutePath());
}
}
2、需要安装CRF相关信息
网上找到两种方式:
出现这种情况的原因是找不到libcrfpp.so.0等库文件,解决方案一为(貌似此方法对root用户不管用):
- 修改/etc/ld.so.conf文件
- 加入include /usr/local/lib
- 执行/sbin/ldconfig -v,刷新LIB库
解决方案二为建立以下符号链接:
ln -s /usr/local/lib/libcrfpp.a /usr/lib/libcrfpp.a
ln -s /usr/local/lib/libcrfpp.so /usr/lib/libcrfpp.so
ln -s /usr/local/lib/libcrfpp.so.0 /usr/lib/libcrfpp.so.0
连接 https://zxdcs.github.io/post/16/crf_java/
python 用户连接 http://midday.me/article/94d6bd4973264e1a801f8445904a810d
公司线上环境是docker容器方式不可用,实际用的方式一。
3、再有是连接库使用训练出来的model文件。路径网上均采用相对路劲,实际容器中不可用,采用绝对路径后解决。
Caused by: java.lang.RuntimeException: feature_index.cpp(193) [mmap_.open(model_filename)] mmap.h(153) [(fd = ::open(filename, flag | O_BINARY)) >= 0] open failed: model
at org.chasen.crfpp.CRFPPJNI.new_Tagger(Native Method)
at org.chasen.crfpp.Tagger.<init>(Tagger.java:183)
at com.jd.app.server.LoadCRFModel.<clinit>(LoadCRFModel.java:89)
... 63 more
这个错误可以采用3解决。
java 使用CRF遇到的问题汇总的更多相关文章
- Hanlp中使用纯JAVA实现CRF分词
Hanlp中使用纯JAVA实现CRF分词 与基于隐马尔可夫模型的最短路径分词.N-最短路径分词相比,基于条件随机场(CRF)的分词对未登录词有更好的支持.本文(HanLP)使用纯Java实现CRF模型 ...
- Java基础知识常见面试题汇总第一篇
[Java面试题系列]:Java基础知识常见面试题汇总 第一篇 文中面试题从茫茫网海中精心筛选,如有错误,欢迎指正! 1.前言 参加过社招的同学都了解,进入一家公司面试开发岗位时,填写完个人信息后 ...
- Java中常用修饰符使用汇总
修饰符汇总: 一:public protected default private 修饰类,修饰方法,修饰属性,修饰代码块. 类: 顶级类只能用public 修饰,顶级类不能使用private 和p ...
- Java远程调试代码不一致问题汇总
欢迎和大家交流技术相关问题: 邮箱: jiangxinnju@163.com 博客园地址: http://www.cnblogs.com/jiangxinnju GitHub地址: https://g ...
- Java最常用的变量定义汇总
Java最常用的数据类型有基本数据类型,字符串对象,数组,基本数据类型又分为:数值型(包括整形和浮点型),字符型,布尔型,下面用一个简单的程序把这些数据类型汇总一下 public class Java ...
- Java最全文件操作实例汇总
本文实例汇总了Java文件操作.分享给大家供大家参考,具体如下: 1.创建文件夹 ? 1 2 3 4 5 6 7 8 9 10 11 //import java.io.*; File myFolder ...
- 阿里巴巴JAVA常考面试题及汇总答案
一.String,StringBuffer, StringBuilder 的区别是什么?String为什么是不可变的? 答: 1.String是字符串常量,StringBuffer和StringB ...
- 2017-2018-1 Java演绎法 小组会议及交互汇总
第一周会议 今天我们小组开展了第一次团队例会活动.我们小组将<构建之法>分为了六个部分并由六位成员先分别学习并向组长上传学习收获,这次的活动内容便是 交流前两周小组成员学习阅读<构建 ...
- Windows 环境下Java调用CRF++详解
1.步骤一览 2.步骤详情 2.1.环境准备 Swig(Simplified Wrapper and Interface Generator)下载,Windows操作系统直接解压即可使用 CRF++( ...
随机推荐
- java-HTML&javaSkcript&CSS&jQuery&ajax
CSS 伪装 1.<style>a;link{color:#000000} a:visited{color:#000000; a.:hover{color:#FF00FF} a:acti ...
- anaconda利用pip安装module
开始_程序 中搜索:anaconda prompt (控制台) 输入pip 出现pip的一些信息,可以忽略 接着输入 pip install 模块名称 例如:pip install alphalens ...
- cf1140E 回文串+染色方案dp
有点硬核的dp..要用到一个结论.. /* 把原串拆成奇偶串,再拆成极大连续的-1串:该串两端都是非-1数,中间都是-1,并且下标要么都是偶数,要么都是技术 然后对所有这些串进行dp,dp[i][0] ...
- re模块(正则)
一, 什么是正则? 正则就是用一些具有特殊含义的符号组合到一起(称为正则表达式)来描述字符或者字符串的方法. 在python中,正则内嵌在python中,并通过re模块实现,正则表达模式被编译成一系列 ...
- 升级centos6.8内核
1.查看默认版本:uname -r 2.更新nss 3.安装elrepo的yum源,升级内核需要使用elrepo的yum源,在安装yum源之前还需要我们导入elrepo的key rpm --impor ...
- Python+selenium自动化测试中Windows窗口跳转方法
Python+selenium自动化测试中Windows窗口跳转方法 #第一种方法 #获得当前窗口 nowhandle=driver.current_window_handle #打开弹窗 drive ...
- 论文阅读笔记三十六:Mask R-CNN(CVPR2017)
论文源址:https://arxiv.org/pdf/1703.06870.pdf 开源代码:https://github.com/matterport/Mask_RCNN 摘要 Mask R-CNN ...
- tp5.0整合七牛云图片上传
转:https://www.cnblogs.com/adobe-lin/p/7699638.html 这里以上传图片为例 上传其他文件也是大同小异 使用composer安装gmars/tp5-qini ...
- ASP.NET CORE Swagger
Package 添加并配置Swagger中间件 将Swagger生成器添加到方法中的服务集合中Startup.ConfigureServices: public void ConfigureServi ...
- MVC异常处理(异常捕获)
1.cshtml页面异常 2.Controller异常 3.路由参数异常. 4.页面不存在404 页面不存在404,可以通过配置config来处理 <customErrors mode=&quo ...