CDH6.3 Centos7

按照官方文档安装即可

CentOS7 上搭建 CDH（6.3.0）

官方文档：https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/install_cm_cdh.html

LZO安装：https://blog.csdn.net/lingeio/article/details/94438582

Sqoop

官方问文档：https://docs.cloudera.com/documentation/enterprise/latest/topics/cm_mc_sqoop1_client.html#concept_xtn_35m_n2b

一共分三步

• Adding the Sqoop 1 Client
  • 
• Installing the JDBC Drivers for Sqoop 1
  • 下载JDBC驱动，放到目录/var/lib/sqoop/中
• Setting HADOOP_MAPRED_HOME for Sqoop 1
  • 在/etc/profile中添加环境变量HADOOP_MAPRED_HOME

export HADOOP_MAPRED_HOME=/opt/cloudera/parcels/CDH-6.3.-.cdh6.3.2.p0./bin
export PATH=$PATH:$JAVA_HOME/bin:HADOOP_MAPRED_HOME

在/opt/cloudera/parcels/CDH-6.3.2-1.cdh6.3.2.p0.1605554/bin中有mapred文件

深入分析CDH的安装目录 https://blog.csdn.net/wj1314250/article/details/86494703

hadoop管理工具---CDH的目录结构了解 https://blog.csdn.net/zzq900503/article/details/79045955

测试

sqoop list-databases --connect jdbc:mysql://localhost:3306 --username root --password 000000

oozie

报错：

WARN org.apache.oozie.command.wf.ActionStartXCommand: SERVER[node01] USER[yarn] GROUP[-] TOKEN[] APP[gmv] JOB[--oozie-oozi-W] ACTION[--oozie-oozi-W@shell-e6c8] Error starting action [shell-e6c8]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: Invalid resource request! Cannot allocate containers as requested resource is greater than maximum allowed allocation. Requested resource type=[memory-mb], Requested resource=<memory:, vCores:>, maximum allowed allocation=<memory:, vCores:>, please note that maximum allowed allocation is calculated by scheduler based on maximum resource of registered NodeManagers, which might be less than configured maximum allocation=<memory:, vCores:>

原因是yarn-site.xml中的两处配置值太小了，不满足作业的申请条件

把yarn-site.xml中的两处配置加大一点：

容器内存 yarn.nodemanager.resource.memory-mb

最大容器内存 yarn.scheduler.maximum-allocation-mb

报错：

WARN org.apache.oozie.action.hadoop.ShellActionExecutor: SERVER[node01] USER[yarn] GROUP[-] TOKEN[] APP[gmv] JOB[--oozie-oozi-W] ACTION[--oozie-oozi-W@shell-9dc9] Launcher exception: output.properties data exceeds its limit []
java.io.IOException: output.properties data exceeds its limit []
    at org.apache.oozie.action.hadoop.LocalFsOperations.getLocalFileContentAsString(LocalFsOperations.java:)
    at org.apache.oozie.action.hadoop.LauncherAM.processActionData(LauncherAM.java:)
    at org.apache.oozie.action.hadoop.LauncherAM.handleActionData(LauncherAM.java:)
    at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:)
    at org.apache.oozie.action.hadoop.LauncherAM$.run(LauncherAM.java:)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
    at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:)

输出大小默认是2048，在oozie-site.xml修改配置，重启
<property>
    <name>oozie.action.max.output.data</name>
    <value>204800</value>
</property>

CDH中