4 datax mysql 和hbase的 相互导入

                                              mysql-->hbase

 

 

0 参考文档：

https://github.com/alibaba/DataX/blob/master/hbase11xwriter/doc/hbase11xwriter.md

 

虽然我是：hbase1.3.3 但是还是可以。

 

1 配置文件

{

  "job": {

    "setting": {

      "speed": {

        "channel": 1

      }

    },

    "content": [

      {

        "reader": {

                    "name": "mysqlreader",

                    "parameter": {

                        "column": ["id","username"],

                        "connection": [

                            {

                                "jdbcUrl": ["jdbc:mysql://192.168.43.20:3306/test"],

                                "table": ["target_user"]

                            }

                        ],

                        "password": "111111",

                        "username": "root",

                        "where": ""

                    }

                },

        "writer": {

          "name": "hbase11xwriter",

          "parameter": {

            "hbaseConfig": {

              "hbase.zookeeper.quorum": "linux01:2181,linux02:2181,linux03:2181"

            },

            "table": "mysql2hbase",

            "mode": "normal",

            "rowkeyColumn": [

                {

                  "index":0,

                  "type":"string"

                },

        {

          "index":1,

          "type":"string"

        }

                {

                  "index":-1,

                  "type":"string",

                  "value":"_"

                }

            ],

            "column": [

              {

                "index":0,

                "name": "group1:id",

                "type": "string"

              },

              {

                "index":1,

                "name": "group1:name",

                "type": "string"

              },

              

            ],

            "versionColumn":{

              "index": -1,

              "value":"123456789"

            },

            "encoding": "utf-8"

          }

        }

      }

    ]

  }

}

 

 

 

2，查看mysql数据

 

3，创建一个hbase表

create "mysql2hbase","group1"

 

4，执行datax脚本

python datax.py mysql2hbase.json

 

5，hbase查看结果

 

 

    hbase-->mysql

0 参考文档： https://github.com/alibaba/DataX/blob/master/hbase11xreader/doc/hbase11xreader.md

 

1 准备配置文件hbase2mysql.json (可以看出，从hbase中拿出group1列簇中拿出2个字段，按照顺序对应下面mysql字段)

{

    "job": {

        "content": [

            {

               "reader": {

                    "name": "hbase11xreader",

                    "parameter": {

                        "hbaseConfig": {

                            "hbase.zookeeper.quorum": "linux01:2181,linux02:2181,linux03:2181"

                        },

                        "table": "mysql2hbase",

                        "encoding": "utf-8",

                        "mode": "normal",

                        "column": [

                            {

                                "name": "group1: id",

                                "type": "string"

                            },

                            {

                                "name": "group1: name",

                                "type": "string"

                            }

                        ],

                        "range": {

                            "startRowkey": "",

                            "endRowkey": "",

                            "isBinaryRowkey": true

                        }

                    }

                },

                "writer": {

                    "name": "mysqlwriter",

                    "parameter": {

                        "column": ["id","username"],

                        "connection": [

                            {

                                "jdbcUrl": "jdbc:mysql://192.168.43.20:3306/test",

                                "table": ["target_user"]

                            }

                        ],

                        "password": "111111",

                        "preSql": [],

                        "session": [],

                        "username": "root",

                        "writeMode": "insert"

                    }

                }

            }

        ],

        "setting": {

            "speed": {

                "channel": "1"

            }

        }

    }

}

 

 

2 准备hbase数据

 

3 准备mysql表结构

 

4，执行文档

    python datax.py  hbase2mysql.json

 

5 查看结果