hive 3.1.0 安装配置

环境:

hadoop 3.1.1

hive 3.1.0

mysql 8.0.11

安装前准备:

准备好mysql-connector-java-8.0.12.jar驱动包

上传hive的tar包并解压

第一步:

进入hive/conf,拷贝hive-env.sh.template 为hive-env.sh,修改部分为第48,51,54行

 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
 # regarding copyright ownership.  The ASF licenses this file
 # to you under the Apache License, Version 2.0 (the
 # "License"); you may not use this file except in compliance
 # with the License.  You may obtain a copy of the License at
 #
 #     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.

 # Set Hive and Hadoop environment variables here. These variables can be used
 # to control the execution of Hive. It should be used by admins to configure
 # the Hive installation (so that users do not have to set environment variables
 # or set command line parameters to get correct behavior).
 #
 # The hive service being invoked (CLI etc.) is available via the environment
 # variable SERVICE

 # Hive Client memory usage can be an issue if a large number of clients
 # are running at the same time. The flags below have been useful in
 # reducing memory usage:
 #
 # if [ "$SERVICE" = "cli" ]; then
 #   if [ -z "$DEBUG" ]; then
 #     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit"
 #   else
 #     export HADOOP_OPTS="$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit"
 #   fi
 # fi

 # The heap size of the jvm stared by hive shell script can be controlled via:
 #
 # export HADOOP_HEAPSIZE=
 #
 # Larger heap size may be required when running queries over large number of files or partitions.
 # By default hive shell scripts use a heap size of  (MB).  Larger heap size would also be
 # appropriate for hive server.

 # Set HADOOP_HOME to point to a specific hadoop install directory
  HADOOP_HOME=/opt/module/hadoop-3.1.

 # Hive Configuration Directory can be controlled by:
  export HIVE_CONF_DIR=/opt/module/hive/conf

 # Folder containing extra libraries required for hive compilation/execution can be controlled by:
  export HIVE_AUX_JARS_PATH=/opt/module/hive/lib

第二步,拷贝hive-default.xml.template为hive-site.xml,主要是一些连接数据库的信息,包括用户名,密码,注意把hive.metastore.schema.verification设置为false

 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
    Licensed to the Apache Software Foundation (ASF) under one or more
    contributor license agreements.  See the NOTICE file distributed with
    this work for additional information regarding copyright ownership.
    The ASF licenses this file to You under the Apache License, Version 2.0
    (the "License"); you may not use this file except in compliance with
    the License.  You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

    Unless required by applicable law or agreed to in writing, software
    distributed under the License is distributed on an "AS IS" BASIS,
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    See the License for the specific language governing permissions and
    limitations under the License.
 -->
 <configuration>
 <property>
 <name>javax.jdo.option.ConnectionURL</name>
 <value>jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&amp;characterEncoding=utf8&amp;useSSL=false&amp;serverTimezone=GMT</value>
 <description>JDBC connect string for a JDBC metastore</description>
 </property>

 <property>
       <name>hive.metastore.schema.verification</name>
       <value>false</value>
   </property>
 <property>
 <name>javax.jdo.option.ConnectionDriverName</name>
 <value>com.mysql.cj.jdbc.Driver</value>
 <description>Driver class name for a JDBC metastore</description>
 </property>

 <property>
 <name>javax.jdo.option.ConnectionUserName</name>
 <value>root</value>
 <description>username to use against metastore database</description>
 </property>

 <property>
 <name>javax.jdo.option.ConnectionPassword</name>
 <value></value>
 <description>password to use against metastore database</description>
 </property>
 <property>
     <name>datanucleus.schema.autoCreateAll</name>
     <value>true</value>
  </property>
 </configuration>

第三步:上传mysql驱动包到hive/lib目录下

第四步:在mysql中创建hive数据库

create database hive;

第五步:进入bin目录执行(指定元数据库并进行初始化)

./schematool -dbType mysql -initSchema

第六步:启动hive(要先启动hadoop)

./hive

启动完成后show databases;

注意事项:

1.hive-site.xml中 mysql的驱动名称为com.mysql.cj.jdbc.Driver

2.xml文档中javax.jdo.option.ConnectionURL中&要用&替代,一定要指定字符集,时区

3.我已经提前对数据库的root用户进行授权

4.如果测试hive插入数据,要在hdfs上创建/user/hive/warehouse路径

常见问题:

1../schematool -dbType mysql -initSchema提示server code 255之类的是连接数据库的字符集没指定,failed的话删除hive数据库,重新创建,再次执行此命令即可

2.The server time zone value 'PDT' is unrecognized or represents more than one

时区问题.写成jdbc:mysql://127.0.0.1:3306/hive?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=GMT即可

参考https://blog.csdn.net/m0_37520980/article/details/80364884