madlib centos yum 包安装

使用centos 测试安装madlib sql 机器学习类库

安装步骤

添加pg 10 repo

yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-centos10-10-2.noarch.rpm

安装pg python 基本包

yum -y install postgresql10-plpython supervisor

安装madlib 依赖包
注意python 版本,我使用python 2.7 安装时候失败了，修改为了python34

yum update -y && yum install -y \

                    git \

                    gcc \

                    wget \

                    postgresql10-devel \

                    openssl \

                    m4 \

                    vim \

                    flex \

                    bison \

                    graphviz \

                    java \

                    epel-release \

                    python34-devel

安装pip 包
默认一般是包含的

yum install -y python34-pip

pg_conf 配置(环境变量)

PATH="$PATH:/usr/pgsql-10/bin"

安装python 依赖（通过pip）

 pip3 install awscli pygresql paramiko --upgrade

安装apache-madlib

下载rpm 包

wget   https://dist.apache.org/repos/dist/release/madlib/1.15.1/apache-madlib-1.15.1-bin-Linux.rpm

安装

yum install -y apache-madlib-1.15.1-bin-Linux.rpm

启动pg 数据库

/usr/pgsql-10/bin/postgresql-10-setup initdb

systemctl enable postgresql-10

systemctl start postgresql-10

修改pg_hba.conf 添加访问支持
之后修改之后，需要重启服务，systemctl restart postgresql-10

/var/lib/pgsql/10/data/pg_hba.conf

修改如下：

# "local" is for Unix domain socket connections only

local all all trust

# IPv4 local connections:

host all all 127.0.0.1/32 trust

# IPv6 local connections:

host all all ::1/128 trust

初始化madlib 服务

/usr/local/madlib/bin/madpack -s madlib -p postgres -c postgres@localhost:5432/postgres install

安装结果如下：

madpack.py: INFO : Detected PostgreSQL version 10.6.

madpack.py: INFO : *** Installing MADlib ***

madpack.py: INFO : MADlib tools version = 1.15.1 (/usr/local/madlib/Versions/1.15.1/bin/../madpack/madpack.py)

madpack.py: INFO : MADlib database version = None (host=localhost:5432, db=postgres, schema=madlib)

madpack.py: INFO : Testing PL/Python environment...

madpack.py: INFO : > Creating language PL/Python...

madpack.py: INFO : > PL/Python environment OK (version: 2.7.5)

madpack.py: INFO : > Preparing objects for the following modules:

madpack.py: INFO : > - array_ops

madpack.py: INFO : > - bayes

madpack.py: INFO : > - crf

madpack.py: INFO : > - elastic_net

madpack.py: INFO : > - linalg

madpack.py: INFO : > - pmml

madpack.py: INFO : > - prob

madpack.py: INFO : > - sketch

madpack.py: INFO : > - svec

madpack.py: INFO : > - svm

madpack.py: INFO : > - tsa

madpack.py: INFO : > - stemmer

madpack.py: INFO : > - conjugate_gradient

madpack.py: INFO : > - knn

madpack.py: INFO : > - lda

madpack.py: INFO : > - stats

madpack.py: INFO : > - svec_util

madpack.py: INFO : > - utilities

madpack.py: INFO : > - assoc_rules

madpack.py: INFO : > - convex

madpack.py: INFO : > - glm

madpack.py: INFO : > - graph

madpack.py: INFO : > - linear_systems

madpack.py: INFO : > - recursive_partitioning

madpack.py: INFO : > - regress

madpack.py: INFO : > - sample

madpack.py: INFO : > - summary

madpack.py: INFO : > - kmeans

madpack.py: INFO : > - pca

madpack.py: INFO : > - validation

madpack.py: INFO : Installing MADlib:

madpack.py: INFO : > Created madlib schema

madpack.py: INFO : > Created madlib.MigrationHistory table

madpack.py: INFO : > Wrote version info in MigrationHistory table

madpack.py: INFO : MADlib 1.15.1 installed successfully in madlib schema.

检查madlib 服务的安装

/usr/local/madlib/bin/madpack -s madlib -p postgres -c postgres@localhost:5432/postgres install-check

输出结果如下：

madpack.py: INFO : Detected PostgreSQL version 10.6.

TEST CASE RESULT|Module: bayes|bayes.ic.sql_in|PASS|Time: 117 milliseconds

TEST CASE RESULT|Module: crf|crf_train_small.ic.sql_in|PASS|Time: 112 milliseconds

TEST CASE RESULT|Module: crf|crf_test_small.ic.sql_in|PASS|Time: 131 milliseconds

TEST CASE RESULT|Module: elastic_net|elastic_net.ic.sql_in|PASS|Time: 123 milliseconds

TEST CASE RESULT|Module: linalg|linalg.ic.sql_in|PASS|Time: 43 milliseconds

TEST CASE RESULT|Module: linalg|svd.ic.sql_in|PASS|Time: 151 milliseconds

TEST CASE RESULT|Module: linalg|matrix_ops.ic.sql_in|PASS|Time: 231 milliseconds

TEST CASE RESULT|Module: prob|prob.ic.sql_in|PASS|Time: 21 milliseconds

TEST CASE RESULT|Module: svm|svm.ic.sql_in|PASS|Time: 151 milliseconds

TEST CASE RESULT|Module: tsa|arima.ic.sql_in|PASS|Time: 130 milliseconds

TEST CASE RESULT|Module: conjugate_gradient|conj_grad.ic.sql_in|PASS|Time: 35 milliseconds

TEST CASE RESULT|Module: knn|knn.ic.sql_in|PASS|Time: 107 milliseconds

TEST CASE RESULT|Module: lda|lda.ic.sql_in|PASS|Time: 109 milliseconds

TEST CASE RESULT|Module: stats|correlation.ic.sql_in|PASS|Time: 93 milliseconds

TEST CASE RESULT|Module: stats|f_test.ic.sql_in|PASS|Time: 25 milliseconds

TEST CASE RESULT|Module: stats|robust_and_clustered_variance_coxph.ic.sql_in|PASS|Time: 116 milliseconds

TEST CASE RESULT|Module: stats|pred_metrics.ic.sql_in|PASS|Time: 98 milliseconds

TEST CASE RESULT|Module: stats|wsr_test.ic.sql_in|PASS|Time: 29 milliseconds

TEST CASE RESULT|Module: stats|mw_test.ic.sql_in|PASS|Time: 26 milliseconds

TEST CASE RESULT|Module: stats|cox_prop_hazards.ic.sql_in|PASS|Time: 101 milliseconds

TEST CASE RESULT|Module: stats|ks_test.ic.sql_in|PASS|Time: 28 milliseconds

TEST CASE RESULT|Module: stats|chi2_test.ic.sql_in|PASS|Time: 28 milliseconds

TEST CASE RESULT|Module: stats|t_test.ic.sql_in|PASS|Time: 26 milliseconds

TEST CASE RESULT|Module: stats|anova_test.ic.sql_in|PASS|Time: 29 milliseconds

TEST CASE RESULT|Module: utilities|minibatch_preprocessing.ic.sql_in|PASS|Time: 104 milliseconds

TEST CASE RESULT|Module: utilities|pivot.ic.sql_in|PASS|Time: 74 milliseconds

TEST CASE RESULT|Module: utilities|path.ic.sql_in|PASS|Time: 88 milliseconds

TEST CASE RESULT|Module: utilities|sessionize.ic.sql_in|PASS|Time: 74 milliseconds

TEST CASE RESULT|Module: utilities|text_utilities.ic.sql_in|PASS|Time: 78 milliseconds

TEST CASE RESULT|Module: utilities|utilities.ic.sql_in|PASS|Time: 77 milliseconds

TEST CASE RESULT|Module: utilities|transform_vec_cols.ic.sql_in|PASS|Time: 93 milliseconds

TEST CASE RESULT|Module: utilities|encode_categorical.ic.sql_in|PASS|Time: 97 milliseconds

TEST CASE RESULT|Module: assoc_rules|assoc_rules.ic.sql_in|PASS|Time: 114 milliseconds

TEST CASE RESULT|Module: convex|mlp.ic.sql_in|PASS|Time: 226 milliseconds

TEST CASE RESULT|Module: convex|lmf.ic.sql_in|PASS|Time: 118 milliseconds

TEST CASE RESULT|Module: glm|glm.ic.sql_in|PASS|Time: 234 milliseconds

TEST CASE RESULT|Module: graph|graph.ic.sql_in|PASS|Time: 218 milliseconds

TEST CASE RESULT|Module: linear_systems|dense_linear_sytems.ic.sql_in|PASS|Time: 88 milliseconds

TEST CASE RESULT|Module: linear_systems|sparse_linear_sytems.ic.sql_in|PASS|Time: 91 milliseconds

TEST CASE RESULT|Module: recursive_partitioning|random_forest.ic.sql_in|PASS|Time: 155 milliseconds

TEST CASE RESULT|Module: recursive_partitioning|decision_tree.ic.sql_in|PASS|Time: 130 milliseconds

TEST CASE RESULT|Module: regress|clustered.ic.sql_in|PASS|Time: 115 milliseconds

TEST CASE RESULT|Module: regress|robust.ic.sql_in|PASS|Time: 99 milliseconds

TEST CASE RESULT|Module: regress|logistic.ic.sql_in|PASS|Time: 101 milliseconds

TEST CASE RESULT|Module: regress|multilogistic.ic.sql_in|PASS|Time: 98 milliseconds

TEST CASE RESULT|Module: regress|marginal.ic.sql_in|PASS|Time: 279 milliseconds

TEST CASE RESULT|Module: regress|linear.ic.sql_in|PASS|Time: 24 milliseconds

TEST CASE RESULT|Module: sample|train_test_split.ic.sql_in|PASS|Time: 76 milliseconds

TEST CASE RESULT|Module: sample|sample.ic.sql_in|PASS|Time: 21 milliseconds

TEST CASE RESULT|Module: sample|stratified_sample.ic.sql_in|PASS|Time: 76 milliseconds

TEST CASE RESULT|Module: sample|balance_sample.ic.sql_in|PASS|Time: 81 milliseconds

TEST CASE RESULT|Module: summary|summary.ic.sql_in|PASS|Time: 86 milliseconds

TEST CASE RESULT|Module: kmeans|kmeans.ic.sql_in|PASS|Time: 155 milliseconds

TEST CASE RESULT|Module: pca|pca_project.ic.sql_in|PASS|Time: 152 milliseconds

TEST CASE RESULT|Module: pca|pca.ic.sql_in|PASS|Time: 242 milliseconds

TEST CASE RESULT|Module: validation|cross_validation.ic.sql_in|PASS|Time: 110 milliseconds

添加数据库测试函数
创建数据表&&添加数据：

CREATE TABLE array_tbl (

    id integer,

    array1 integer[],

    array2 integer[]

);

INSERT INTO "public"."array_tbl"("id","array1","array2")

VALUES

(1,E'{1,2,3,4,5,6}',E'{6,5,4,3,2,1}'),

(2,E'{1,1,0,0,99,8}',E'{0,0,0,-5,2,1}');

查看插入的结果

select * from array_tbl;

 id | array1 | array2

----+----------------+----------------

  1 | {1,2,3,4,5,6} | {6,5,4,3,2,1}

  2 | {1,1,0,0,99,8} | {0,0,0,-5,2,1}

(2 行记录)

使用madlib API:
说明：主要是计算数组的最大、最小值

select id,madlib.array_min(array1) min, madlib.array_max(array1) max from array_tbl;

select id,madlib.array_min(array1) min, madlib.array_max(array1) max from array_tbl;

 id | min | max

----+-----+-----

  1 | 1 | 6

  2 | 0 | 99

(2 行记录)

说明

这个只是简单的安装，实际生产还需要好多东西需要调整，同时注意madlib schema 的安装，是对应到数据库的，当然
测试环境我们可以直接使用docker 版本的，尽管有点大，但是还是很方便的

参考资料

https://cwiki.apache.org/confluence/display/MADLIB/Installation+Guide#InstallationGuide-Prerequisites
https://www.postgresql.org/download/linux/redhat/

madlib centos yum 包安装的更多相关文章

centos yum方式安装net-snmp v3
SNMPv3的安全报头采用用户安全模式(USM),其提供具有机密性和完整性的网络管理通信.机密性通过采用数据加密标准(DES)来提供.尽管这一算法以脆弱性著称(由于它采用的是40位的密钥),但与明文 ...
CentOs yum源安装 nginx
1 更新源 [root@server ~]#rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.no ...
CentOS yum如何安装php7.4
centos系统下使用yum安装php7.4正式版,当前基于WLNMP提供的一键安装包来安装 1.添加epel源 yum install epel-release 2.添加WLNMP一键安装包源 rp ...
CentOS常用包安装
安装rz和sz命令: yum install lrzsz 安装unzip:yum install unzip
yum -------包安装库
elrepo: http://elrepo.org/tiki/tiki-index.php CentOSPlus: http://wiki.centos.org/zh/AdditionalR ...
查看yum包安装地址
首先找到包含版本号在内的全包名 rpm -qa|grep t_dp_apsara_exstoret_dp_apsara_exstore-1.0.5-56 然后就可以查询到了 rpm -ql t_dp_ ...
Centos rpm包安装PHP所需包
yum -y install php php-devel php-fpm php-xml php-pdo php-ldap php-mysql
查询yum包安装路径
rpm -ql php71-php yum install json yum install libcurl
CentOS 使用yum命令安装Java SDK(openjdk)
CentOS 6.X 和 5.X 自带有OpenJDK runtime environment (openjdk).它是一个在linux上实现开源的java 平台.CentOS yum 命令安装 ...

随机推荐

[Leetcode221]最大面积 Maximal Square
[题目] Given a 2D binary matrix filled with 0's and 1's, find the largest square containing only 1's a ...
Html.DropDownList()的用法
页面代码如下: <%= Html.DropDownList("Category", ViewData["Categories"] as SelectLis ...
leetcode第26题：删除排序数组的重复项
给定一个排序数组,你需要在原地删除重复出现的元素,使得每个元素只出现一次,返回移除后数组的新长度. 不要使用额外的数组空间,你必须在原地修改输入数组并在使用 O(1) 额外空间的条件下完成. 给定数组 ...
jQuery中AJAX同步如何实现?
jax请求默认的都是异步的如果想同步 async设置为false就可以(默认是true) var html = $.ajax({ url: "some.php", async: ...
hadoop day 1
hadoop是什么? 解决的问题: 海量数据的存储(HDFS):供hbase,mapreduce进行处理海量数据的分析(MapReduce) 资源管理调度(YARN) 搜索引擎: 爬虫系统+站内搜索 ...
查询表Or列的注释信息
需求:开发人员需要DBA支持,查询表的注释说明,用于明确表的用途. 1.测试 session 1 创建测试表SQL> create table a_emp as select * from sc ...
Django之模板层-语法：{{ }}
模版语法的深度查询(.) views.py def index(request): name = 'name' lis = [1,2,3,4,5,6] dic = {"name": ...
sharpkeys键盘按键重映射
/********************************************************************** * sharpkeys键盘按键重映射 * 说明: * 键 ...
mac下python安装MySQLdb模块
参考:http://blog.csdn.net/yelyyely/article/details/41114449 1.调整到anaconda下的python 2.安装有关程序 brew instal ...
c# 移动鼠标到指定位置
/// <summary> /// 引用user32.dll动态链接库(windows api), /// 使用库中定义 API:SetCursorPos /// </summary ...

madlib centos yum 包安装

安装步骤

说明

参考资料

madlib centos yum 包安装的更多相关文章

随机推荐

热门专题