hadoop+zookeeper+hbase伪分布式安装

基本安装步骤##

安装包下载###

从大数据组件下载地址下载以下组件安装包

1. hadoop-2.6.0-cdh5.6.0.tar.gz
2. hbase-1.0.0-cdh5.6.0.tar.gz
3. zookeeper-3.4.5-cdh5.6.0.tar.gz
   
   从java下载地址下载以下组件安装包
4. jdk-8u144-linux-x64.tar.gz

java环境###

1. 解压 tar zxvf jdk-8u144-linux-x64.tar.gz
2. 配置环境变量
   1. vim /etc/profile

export JAVA_HOME=/opt/cdh/jdk1.8.0_144
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=.:$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

2. `source /etc/profile`

SSH免密登陆###

• 教程链接

hadoop安装###

1. 解压 tar zxvf hadoop-2.6.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hadoop-2.6.0-cdh5.6.0/etc/hadoop/下
   • 修改core-site.xml

<configuration>
	<property>
    	<name>hadoop.tmp.dir</name>
    	<value>/opt/cdh/hadoop/tmp</value>
	</property>
	<property>
    	<name>dfs.name.dir</name>
    	<value>/opt/cdh/hadoop/name</value>
	</property>
	<property>
    	<name>fs.default.name</name>
    	<value>hdfs://ubuntu:9000</value>
	</property>
</configuration>

- 修改mapred-site.xml

<configuration>
    <property>
        <name>mapred.job.tracker</name>
        <value>ubuntu:9001</value>
    </property>
</configuration>

- 修改hdfs-site.xml

<configuration>
    <property>
        <name>dfs.data.dir</name>
        <value>/opt/cdh/hadoop/data</value>
    </property>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
    <property>
        <name>dfs.permissions</name>
        <value>false</value>
    </property>
</configuration>

- 修改hadoop-env.sh

export JAVA_HOME=/opt/cdh/jdk1.8.0_144

1. 格式化namenode

hadoop-2.6.0-cdh5.6.0/bin/hadoop namenode -format

1. 启动hdfs

hadoop-2.6.0-cdh5.6.0/sbin/start-all.sh

1. 验证
   • jps
   • 登录网址: http://ip:50070

zookeeper安装###

1. 解压 tar zxvf zookeeper-3.4.5-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录zookeeper-3.4.5-cdh5.6.0/conf/下
   • zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/opt/cdh/zookeeper
clientPort=2181

1. 启动zookeeper

zookeeper-3.4.5-cdh5.6.0/bin/zkServer.sh start

hbase安装###

1. 解压 tar zxvf hbase-1.0.0-cdh5.6.0.tar.gz
2. 修改配置文件,位于目录hbase-1.0.0-cdh5.6.0/conf/下
   • hbase-env.sh

export JAVA_HOME=/opt/cdh/jdk1.8.0_144
export HBASE_MANAGES_ZK=false

- regionservers

ubuntu

- hbase-site.xml

<configuration>
	<property>
    	<name>hbase.cluster.distributed</name>
    	<value>true</value>
  	</property>
    <property>
        <name>hbase.rootdir</name>
        <value>hdfs://ubuntu:9000/hbase</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>hdfs://ubuntu:60000</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>ubuntu:2181</value>
    </property>
  	<property>
    	<name>hbase.zookeeper.property.dataDir</name>
    	<value>/opt/cdh/zookeeper</value>
  	</property>
</configuration>

1. 启动hbase
   
   hbase-1.0.0-cdh5.6.0/bin/start-hbase.sh
2. 验证
   1. jps
   2. hbase-1.0.0-cdh5.6.0/bin/hbase shell
   3. 登陆网址： http://ip:60010

IDEA连接hbase登陆###

1. 新建maven项目
2. 修改pom.xml文件

	<repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
        </repository>
    </repositories>

    <dependencies>

        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase</artifactId>
            <version>1.0.0-cdh5.6.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>1.0.0-cdh5.6.0</version>
        </dependency>
    </dependencies>

1. 按照百度搜索结果编写demo程序并执行

ERROR解决###

1. 先确认以下地址一致
   
   linux下的/etc/hosts里的ip与windows下C:\Windows\System32\drivers\etc\hosts里的ip映射一致
2. 禁用Hbase自带的zookeeper
   • 修改hbase-env.sh里的export HBASE_MANAGES_ZK=false