vertica 8.0 新特性

前言:

<<line>> 表明在vertica 8.0文档中的title

 

正文:

1-支持平台

1.1-操作系统

<<Vertica Server and Vertica Management Console>>

When there are multiple minor versions supported for a major operating system release, Hewlett Packard Enterprise recommends that you run Vertica on the latest minor version listed in the supported versions list. For example, if you run Vertica on a Red Hat Enterprise Linux 6.x release, Hewlett Packard Enterprise recommends you upgrade to or be running the latest supported RHEL 6.x release, which is 6.8.

官方推荐rhel6.8或centos6.8版本

 

1.2- 文件系统

Vertica Analytic Database Enterprise Edition has been tested on all supported Linux platforms running the ext4 file system. For the Vertica Analytic Database I/O profile, the ext4 file system is considerably faster than ext3.

官方推荐ext4方式，不要使用LVM

 

1.3-hadoop支持

<<Vertica Integrations for Hadoop>>

支持CDH 5.6-5.8,其中5.6逐渐不再支持

 

1.4-kafka支持

<<Vertica Integration for Apache Kafka>>

支持Kafka 0.8.x-0.9,其中0.8.x逐渐不再支持

 

1.5-java/R支持

jdk 1.6-1.8

R语言3.0

 

2-新特性-

2.1-机器学习的资源池单独设置

<<New Resource Pool for BLOBDATA>>

blobdata资源池的设置

 

2.2-异常值检测

outliner过程

<<DETECT_OUTLIERS>>

基于标准差(待定是不是标准差)来筛选数据

If data point > ( value * STDDEV( data ) ) then data point is an outlier.

dbadmin=> SELECT * FROM baseball_roster;

id | last_name | hr | avg

----+-----------+--------+--------

1 | Polo | 7 | 0.233

2 | Gloss | 45 | 0.17

3 | Gus | 12 | 0.345

4 | Gee | 1 | 0.125

5 | Laus | 3 | 0.095

6 | Hilltop | 16 | 0.222

7 | Wicker | 78 | 0.333

8 | Scooter | 0 | 0.121

9 | Hank | 999999 | 0.8888

10 | Popup | 35 | 0.378

(10 rows)

 

dbadmin=> SELECT * FROM baseball_outliers;

id | last_name | hr | avg

----+-----------+--------+--------

7 | Wicker | 78 | 0.333

9 | Hank | 999999 | 0.8888

(2 rows)

 

2.3-机器学习相关

2.3.1-期初导入

<<Downloading the Machine Learning Example Data>>

yum install git

git clone https://github.com/vertica/Machine-Learning-Examples

vsql -d dbname -f load_ml_data.sql

 

2.3.2-训练模型

<<NAIVE_BAYES>>

#训练基础表,党派人士的投票倾向

select * from public.house84_train;

#训练model//训练基础表//要预测哪一列//基于那些数据来预测//排除非数据列

SELECT NAIVE_BAYES

('naive_house84_model', 'house84_train', 'party', '*','--exclude_columns="party, id"');

 

2.3.3-验证模型

<<Classifying Data Using Naive Bayes>>

SELECT SUMMARIZE_MODEL('naive_house84_model', 'dbadmin');

基于模型来检验是民主党还是共和党

CREATE TABLE predicted_party_naive

AS SELECT party,

PREDICT_NAIVE_BAYES (vote1, vote2, vote3, vote4, vote5,

vote6, vote7, vote8, vote9, vote10, vote11, vote12, vote13, vote14,vote15, vote16

USING PARAMETERS model_name = 'naive_house84_model',owner = 'dbadmin',type = 'response') AS Predicted_Party

FROM house84_test;

 

 

每个预测的可能性百分比

SELECT PREDICT_NAIVE_BAYES_CLASSES (id, vote1, vote2, vote3, vote4, vote5,

vote6, vote7, vote8, vote9, vote10,

vote11, vote12, vote13, vote14,

vote15, vote16

USING PARAMETERS model_name = 'naive_house84_model',

owner = 'dbadmin',

key_columns = 'id', exclude_columns = 'id',

classes = 'democrat, republican')

OVER() FROM house84_test;

 

基于训练集的前三个vote,来预测党派

SELECT party, PREDICT_NAIVE_BAYES (vote1, vote2, vote3

USING PARAMETERS model_name = 'naive_house84_model',

owner = 'dbadmin',

type = 'response') AS Predicted_Party

FROM house84_test;