Sqoop增量导入

Argument	Description
--check-column (col)	Specifies the column to be examined when determining which rows to import. (the column should not be of type CHAR/NCHAR/VARCHAR/VARNCHAR/ LONGVARCHAR/LONGNVARCHAR)
--incremental (mode)	Specifies how Sqoop determines which rows are new. Legal values for mode include append and lastmodified.
--last-value (value)	Specifies the maximum value of the check column from the previous import.

Append模式增量导入

sqoop import --connect 'jdbc:mysql://localhost:3306/test'

--username root

--password root

--table role

-hive-database sqoopdb -hive-table role

--hive-import -m -1

--incremental append

--check-column dateid

--last-value "2017-12-01"

--check-column 可以是一列或者多列，用来检测数据是否按增量数据导入

--last-value   上次导入的检测字段的最大值

--incremental  增量的导入模式，有两种分别为Append和Lastmodified

测试没问题

 Lastmodified增量导入

[root@host ~]# sqoop import --connect 'jdbc:mysql://localhost:3306/test' --username root --password root --table role -hive-database sqoopdb -hive-table role --hive-import -m -1 --incremental lastmodified  --check-column dateid --last-value "2015-12-01"

Warning: /root/sqoop1/sqoop-1.4.7.bin__hadoop-2.6.0/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /root/sqoop1/sqoop-1.4.7.bin__hadoop-2.6.0/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /root/sqoop1/sqoop-1.4.7.bin__hadoop-2.6.0/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
18/06/06 16:59:25 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
18/06/06 16:59:26 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
18/06/06 16:59:26 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
18/06/06 16:59:26 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
--incremental lastmodified option for hive imports is not supported. Please remove the parameter --incremental lastmodified.

发现导入hive不支持