hive 3.1.0 安装配置
环境:
hadoop 3.1.1
hive 3.1.0
mysql 8.0.11
安装前准备:
准备好mysql-connector-java-8.0.12.jar驱动包
上传hive的tar包并解压
第一步:
进入hive/conf,拷贝hive-env.sh.template 为hive-env.sh,修改部分为第48,51,54行
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License. # Set Hive and Hadoop environment variables here. These variables can be used
# to control the execution of Hive. It should be used by admins to configure
# the Hive installation (so that users do not have to set environment variables
# or set command line parameters to get correct behavior).
#
# The hive service being invoked (CLI etc.) is available via the environment
# variable SERVICE # Hive Client memory usage can be an issue if a large number of clients
# are running at the same time. The flags below have been useful in
# reducing memory usage:
#
# if [ "$SERVICE" = "cli" ]; then
# if [ -z "$DEBUG" ]; then
# export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit"
# else
# export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit"
# fi
# fi # The heap size of the jvm stared by hive shell script can be controlled via:
#
# export HADOOP_HEAPSIZE=
#
# Larger heap size may be required when running queries over large number of files or partitions.
# By default hive shell scripts use a heap size of (MB). Larger heap size would also be
# appropriate for hive server. # Set HADOOP_HOME to point to a specific hadoop install directory
HADOOP_HOME=/opt/module/hadoop-3.1. # Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/opt/module/hive/conf # Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/opt/module/hive/lib
第二步,拷贝hive-default.xml.template为hive-site.xml,主要是一些连接数据库的信息,包括用户名,密码,注意把hive.metastore.schema.verification设置为false
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=GMT</value>
<description>JDBC connect string for a JDBC metastore</description>
</property> <property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.cj.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property> <property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property> <property>
<name>javax.jdo.option.ConnectionPassword</name>
<value></value>
<description>password to use against metastore database</description>
</property>
<property>
<name>datanucleus.schema.autoCreateAll</name>
<value>true</value>
</property>
</configuration>
第三步:上传mysql驱动包到hive/lib目录下
第四步:在mysql中创建hive数据库
create database hive;
第五步:进入bin目录执行(指定元数据库并进行初始化)
./schematool -dbType mysql -initSchema
第六步:启动hive(要先启动hadoop)
./hive
启动完成后show databases;
注意事项:
1.hive-site.xml中 mysql的驱动名称为com.mysql.cj.jdbc.Driver
2.xml文档中javax.jdo.option.ConnectionURL中&要用&替代,一定要指定字符集,时区
3.我已经提前对数据库的root用户进行授权
4.如果测试hive插入数据,要在hdfs上创建/user/hive/warehouse路径
常见问题:
1../schematool -dbType mysql -initSchema提示server code 255之类的是连接数据库的字符集没指定,failed的话删除hive数据库,重新创建,再次执行此命令即可
2.The server time zone value 'PDT' is unrecognized or represents more than one
时区问题.写成jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=GMT即可
参考https://blog.csdn.net/m0_37520980/article/details/80364884
hive 3.1.0 安装配置的更多相关文章
- CentOS 7.0安装配置Vsftp服务器
一.配置防火墙,开启FTP服务器需要的端口 CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙. 1.关闭firewall: systemctl stop fi ...
- CentOS 7.0安装配置LAMP服务器(Apache+PHP+MariaDB)
CentOS 7.0默认使用的是firewall作为防火墙,这里改为iptables防火墙. 1.关闭firewall: systemctl stop firewalld.service #停止fir ...
- CentOS 7.0安装配置Vsftp服务器步骤详解
安装Vsftp讲过最多的就是在centos6.x版本中了,这里小编看到有朋友写了一篇非常不错的CentOS 7.0安装配置Vsftp服务器教程,下面整理分享给各位. 一.配置防火墙,开启FTP服务器需 ...
- CentOS 7.0 安装配置LAMP服务器方法(Apache+PHP+MariaDB)(转)
转自:http://www.jb51.net/os/188488.html 作者:佚名 字体:[增加 减小] 来源:osyunwei 准备篇: CentOS 7.0系统安装配置图解教程 http:/ ...
- Win10 64位+VS2015+Opencv3.3.0安装配置
Win10 64位+VS2015+Opencv3.3.0安装配置 1.我们首先下载VS2015.OpenCV3.3.0. 1.1 VS2015下载 在官网https://visualstudio.mi ...
- 实战:ADFS3.0单点登录系列-ADFS3.0安装配置
本文为系列第三章,主要讲下ADFS3.0的安装和配置.本文和前面的文章是一个系列,因此有些地方是有前后关联,比如本文中使用的通配符证书就是第二篇讲解的,因此需要连贯的进行阅读. 全文目录如下: 实战: ...
- 【转发】【linux】【ftp】CentOS 7.0安装配置Vsftp服务器
adduser -d /var/www/android -g ftp -s /sbin/nologin ftp2 一.配置防火墙,开启FTP服务器需要的端口 CentOS 7.0默认使用的是firew ...
- Hive 2.1.1安装配置
##前期工作 安装JDK 安装Hadoop 安装MySQL ##安装Hive ###下载Hive安装包 可以从 Apache 其中一个镜像站点中下载最新稳定版的 Hive, apache-hive-2 ...
- Linux Tomcat 6.0安装配置实践总结
系统环境: Red Hat Enterprise Linux Server release 5.7 (Tikanga) 64位 Tomcat下载 从官方网站 http://tomcat.apache ...
随机推荐
- linux下pthread_cancel无法取消线程的原因
一个线程能够调用pthread_cancel终止同一进程中的还有一个线程,可是值得强调的是:同一进程的线程间,pthread_cancel向还有一线程发终止信号.系统并不会立即关闭被取消线程,仅仅有在 ...
- LeetCode Algorithm 133_Clone Graph
Clone an undirected graph. Each node in the graph contains a label and a list of its neighbors. OJ's ...
- Microsoft Bot Framework 链接至微信公共号
如何将 Microsoft Bot Framework 链接至微信公共号 说到 Microsoft Bot Framework 其实微软发布了已经有一段时间了,有很多朋友可能还不太了解,微软Bot ...
- uiview关联xib
1,在需要实例的地方 //加载一个uiview的作法 [LotteryInvestigationView *lotteryInvestigationView=[[[NSBundle mainBundl ...
- 【hdu 6194】string string string
[链接]h在这里写链接 [题意] 给你一个字符串s以及一个整数k; 让你找出这个字符串里面,恰好出现了k次的子串的个数. k>=1 [题解] 后缀数组题. 对于输入的字符串.求出它的Height ...
- YASM User Manual
This document is the user manual for the Yasm assembler. It is intended as both an introduction and ...
- Linq读取XML数据
1.XML数据格式:<?xml version="1.0"?><customers> <customer> <id>ALFK ...
- hdu 4811 数学 不难
http://acm.hdu.edu.cn/showproblem.php? pid=4811 由于看到ball[0]>=2 && ball[1]>=2 && ...
- [TypeStyle] Compose CSS classes using TypeStyle
We will demonstrate composing classes using the utility classes function. classes is also what we re ...
- 将String转化为Long,并将Long转化为Date 分类: B1_JAVA 2014-06-30 16:23 1249人阅读 评论(0) 收藏
package org.ljh.test.javaee; import java.text.SimpleDateFormat; import java.util.Date; public class ...