ElasticSearch 安装 go-mysql-elasticsearch 同步mysql的数据

一、首先在Centos6.5上安装 go 语言环境

　　下载Golang语言包：https://studygolang.com/dl

 [hoojjack@localhost src]$ ls
 apache-maven-3.5.0  go  go1.6.2.linux-amd64.tar.gz

二、解压.tar.gz安装包。

三、配置安装环境。

[hoojjack@localhost etc]$ vim profile
export GOPATH=/usr/local/src/go/work
export GOROOT=/usr/local/src/go
export PATH=$GOPATH/bin:$PATH

GOARCH（指定系统环境，i386表示x86，amd64表示x64）：amd64

GOROOT：/usr/local/src/go（go的解压路径）

GOBIN：%GOROOT%/bin（exe执行文件路径）

GOOS：（go运行的系统）

GOPATH：/usr/local/src/go/work（用于存放Go语言Package的目录，这个目录不能在Go的安装目录中，放了好像也没什么事，至少我目前没事 ^_^）

我们需要添加的环境变量分别是go的安装路径GOROOT：/usr/local/src/go，PATH：$GOROOT/bin:$PATH，go的工作路径GOPATH:/usr/local/src/go/work

四、配置go环境后配置go-mysql-elasticsearch

首先确保环境中安装了git

安装依赖包：

sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel

也可以直接用yum命令安装：

yum install git

然后：

获取安装包：go get github.com/siddontang/go-mysql-elasticsearch

安装：

cd   $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
         make

 [hoojjack@localhost etc]$ cd /usr/local/src/go/src/github.com/siddontang/go-mysql-elasticsearch
 [hoojjack@localhost go-mysql-elasticsearch]$ ls
 bin  cmd  elastic  etc  Godeps  LICENSE  Makefile  README.md  river  var
 [hoojjack@localhost go-mysql-elasticsearch]$ 

五、修改配置文件

 # MySQL address, user and password
 # user must have replication privilege in MySQL.
 my_addr = "127.0.0.1:3306"     //需要同步的mysql基本设置
 my_user = "root"
 my_pass = "root"

 # Elasticsearch address
 es_addr = "127.0.0.1:9200"     //本地elasticsearch配置

 # Path to store data, like master.info, and dump MySQL data
 data_dir = "./var"             //数据存储的url
  //以下配置保存默认不变
 # Inner Http status address
 stat_addr = "127.0.0.1:12800"

 # pseudo server id like a slave
 server_id = 1001

 # mysql or mariadb
 flavor = "mysql"
 # mysqldump execution path
 mysqldump = "mysqldump"

 # MySQL data source
 [[source]]
 schema = "hoojjack"             //elasticsearch 与 mysql 同步时对应的数据库名称

 # Only below tables will be synced into Elasticsearch.
 # "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
 # I don't think it is necessary to sync all tables in a database.
 tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符，可以指定只复制hoojjack数据库中指定的表数据

 # Below is for special rule mapping
 [[rule]]
 schema = "hoojjack"    //数据库名称
 table = "test_river"   //表名称
 index = "river"        //对应的索引名称
 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
   [rule.field]
 # This will map column title to elastic search my_title
   title="es_title"     //将可以将mysql的某个属性对应指向elasticsearch的某个field, 如test_river的titile属性对应es_title
 # This will map column tags to elastic search my_tags and use array type
   tags="my_tags,list"
 # This will map column keywords to elastic search keywords and use array type
   keywords=",list"

 # wildcard table rule, the wildcard table must be in source tables
   [[rule]]
   schema = "hoojjack"
   table = "test_river_[0-9]{4}"
   index = "river"
   type = "river"

 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
   [[rule.fields]]
   mysql = "title"
   elastic = "es_title"

六、执行go-mysql-elasticsearch同步，首先进入 go-mysql-elasticsearch 路径，然后执行 ： ./bin/go-mysql-elasticsearch -config=./etc/river.toml

 cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch
 ./bin/go-mysql-elasticsearch -config=./etc/river.toml

效果如下：

mysql> select * from test_river;
+----+-----------+-----------------+--------+
| id | name      | title           | number |
+----+-----------+-----------------+--------+
|  1 | hoojjack  | wo shi yi bing  |      1 |
|  2 | xunchuan  | ni shi er ren   |      2 |

七、出现的问题：

1、mysqldump: command not found

mysql的mysqldump命令没有找到，解决办法：创建软连接，查看mysqldump是在哪个路径下：find / -name mysqldump   会获得路径，然后创建软连接：ln -s /usr/local/mysql/bin/mysqldump  /usr/bin

这是由于系统默认会查找/usr/bin下的命令，如果这个命令不在这个目录下，当然会找不到命令

如果没有安装mysql，则可以利用yum安装mysql5.6

1）检查系统是否安装其他版本的MYSQL数据

#yum list installed | grep mysql
#yum -y remove mysql-libs.x86_64

2）安装及配置

# wget http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm
# rpm -ivh mysql-community-release-el6-.noarch.rpm
# yum repolist all | grep mysql

3）安装MYSQL数据库

yum install mysql-community-server -y

2、make build go-mysql-elasticsearch项目的时候出现问题（can’t load package: package .: no buildable Go source files in ）

网上有很多的解决方法，但是我都没看懂，唯一看懂的大概就是要升级下go版本，因为之前go是1.6的，索性尝试升级到1.10，后面问题解决了（人品大爆棚），但是问题的核心没有解决。啊啊。。

【Reference】

[1] http://blog.csdn.net/laoyang360/article/details/51771483