Sqoop环境安装

环境下载

　　首先将下载的 sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz放到 /usr/hadoop/目录下（该目录可以自定义，一般为Hadoop集群安装目录），然后对安装包解压、修改文件名和修改用户权限。

[hadoop@centpy /]$ cd /usr/hadoop/
[hadoop@centpy hadoop]$ ls
flume  hadoop-2.6.
[hadoop@centpy hadoop]$ rz   //上传安装包

[hadoop@centpy hadoop]$ ls
flume  hadoop-2.6.  sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
[hadoop@centpy hadoop]$ tar -zxf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz   //解压
[hadoop@centpy hadoop]$ mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop   //修改安装文件目录
[hadoop@centpy hadoop]$ ls
flume  hadoop-2.6.  sqoop  sqoop-1.4..bin__hadoop-2.0.-alpha.tar.gz
[hadoop@centpy hadoop]$ rm -f sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz   //删除安装包
[hadoop@centpy hadoop]$ ls
flume  hadoop-2.6.  sqoop

环境配置

1.配置hadoop版本号

　　切换到/sqoop/conf 目录下，执行以下命令修改对应Hadoop版本号为自己所安装的集群的版本号。

[hadoop@centpy hadoop]$ cd sqoop/
[hadoop@centpy sqoop]$ ls
bin            COMPILING.txt  ivy      LICENSE.txt  README.txt             sqoop-test-1.4..jar
build.xml      conf           ivy.xml  NOTICE.txt   sqoop-1.4..jar        src
CHANGELOG.txt  docs           lib      pom-old.xml  sqoop-patch-review.py  testdata
　[hadoop@centpy sqoop]$ vi build.xml

修改为：

2.配置sqoop-env.sh文件

　　执行以下命令生成sqoop-env.sh配置文件。

[hadoop@centpy sqoop]$ cd conf/
[hadoop@centpy conf]$ ls
oraoop-site-template.xml  sqoop-env-template.sh    sqoop-site.xml
sqoop-env-template.cmd    sqoop-site-template.xml
[hadoop@centpy conf]$ mv sqoop-env-template.sh sqoop-env.sh

　　然后使用 vi sqoop-env.sh 命令，打开文件添加如下内容。

　　如果数据读取不涉及hbase和hive，那么相关hbase和hive的配置可以不加；如果集群有独立的zookeeper集群，那么配置zookeeper，反之，不用配置。

3.配置环境变量　

[hadoop@centpy conf]$ su
Password:
[root@centpy conf]# vim /etc/profile   //追加以下内容
　　#sqoop
　　export SQOOP_HOME=/usr/hadoop/sqoop
　　export PATH=$PATH:$SQOOP_HOME/bin
[root@centpy conf]# source /etc/profile

4.上传jar包

　　将相关的驱动 jar 包拷贝到 sqoop/lib 目录下。数据库驱动 jar 包需要导入，这里我们使用的是 mysql 数据库，所以需要导入mysql-connector-java-5.1.32.jar包。

5.确定用户组

　　需要确定sqoop所属用户组是否为hadoop

[hadoop@centpy hadoop]$ ll
total
drwxrwxr-x.   hadoop hadoop  Jun  : flume
drwxr-xr-x.  hadoop hadoop  Jun  : hadoop-2.6.
drwxr-xr-x.   hadoop hadoop  Jun  : sqoop

　　如果不是请通过以下语句修改：

chown –R hadoop:hadoop sqoop 

测试运行

//首先启动hadoop集群
[hadoop@centpy sqoop]$ cd ../hadoop-2.6.0/
[hadoop@centpy hadoop-2.6.]$ sbin/start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [centpy]
centpy: starting namenode, logging to /usr/hadoop/hadoop-2.6./logs/hadoop-hadoop-namenode-centpy.out
centpy: starting datanode, logging to /usr/hadoop/hadoop-2.6./logs/hadoop-hadoop-datanode-centpy.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/hadoop/hadoop-2.6./logs/hadoop-hadoop-secondarynamenode-centpy.out
starting yarn daemons
starting resourcemanager, logging to /usr/hadoop/hadoop-2.6./logs/yarn-hadoop-resourcemanager-centpy.out
centpy: starting nodemanager, logging to /usr/hadoop/hadoop-2.6./logs/yarn-hadoop-nodemanager-centpy.out
[hadoop@centpy hadoop-2.6.]$ jps
 SecondaryNameNode
 Jps
 ResourceManager
 NameNode
 NodeManager
 DataNode
//测试sqoop
[hadoop@centpy hadoop-2.6.]$ cd ../sqoop/
[hadoop@centpy sqoop]$ sqoop list-databases --connect jdbc:mysql://202.193.60.117/dataweb --username root --password 20134997
Warning: /usr/hadoop/sqoop/../hbase does not exist! HBase imports will fail.
Please set $HBASE_HOME to the root of your HBase installation.
Warning: /usr/hadoop/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/hadoop/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/hadoop/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
// :: INFO sqoop.Sqoop: Running Sqoop version: 1.4.
// :: WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
// :: INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
dataweb
mysql
performance_schema
test

　　sqoop 命令执行成功，代表安装成功。

sqoop list-databases --connect jdbc:mysql://202.193.60.117/dataweb --username root --password 20134997

　　该语句中使用我们上传的mysql-connector-java-5.1.32.jar包链接本地数据库，其中ip地址可以通过命令行中输入ipconfig进行查看。

　　执行该命令之前要先启动数据库服务，若出现2003错误时因为mysql服务没有成功启动，解决方案如下：

　　选中后右键开启服务即可。

　　到此，sqoop的安装和配置就结束了！

 以上就是博主为大家介绍的这一板块的主要内容，这都是博主自己的学习过程，希望能给大家带来一定的指导作用，有用的还望大家点个支持，如果对你没用也望包涵，有错误烦请指出。如有期待可关注博主以第一时间获取更新哦，谢谢！