Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://slaver1:9000/user/hadoop/tb_user already exists

1、当时初学Sqoop的时候，mysql导入到hdfs导入命令执行以后，在hdfs上面没有找到对应的数据，今天根据这个bug，顺便解决这个问题吧，之前写的http://www.cnblogs.com/biehongli/p/8039128.html。

 [hadoop@slaver1 sqoop-1.4.-cdh5.3.6]$ bin/sqoop import \
 > --connect jdbc:mysql://slaver1:3306/test \
 > --username root \
 > --password  \
 > --table tb_user \
 > --m
 Warning: /home/hadoop/soft/sqoop-1.4.-cdh5.3.6/../hcatalog does not exist! HCatalog jobs will fail.
 Please set $HCAT_HOME to the root of your HCatalog installation.
 Warning: /home/hadoop/soft/sqoop-1.4.-cdh5.3.6/../accumulo does not exist! Accumulo imports will fail.
 Please set $ACCUMULO_HOME to the root of your Accumulo installation.
 // :: INFO sqoop.Sqoop: Running Sqoop version: 1.4.-cdh5.3.6
 // :: WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
 // :: INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
 // :: INFO tool.CodeGenTool: Beginning code generation
 // :: INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_user` AS t LIMIT
 // :: INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_user` AS t LIMIT
 // :: INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/soft/hadoop-2.5.-cdh5.3.6
 Note: /tmp/sqoop-hadoop/compile/cb147e9deb144db0034d6f38cb47ad68/tb_user.java uses or overrides a deprecated API.
 Note: Recompile with -Xlint:deprecation for details.
 // :: INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/cb147e9deb144db0034d6f38cb47ad68/tb_user.jar
 // :: WARN manager.MySQLManager: It looks like you are importing from mysql.
 // :: WARN manager.MySQLManager: This transfer can be faster! Use the --direct
 // :: WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
 // :: INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
 // :: INFO mapreduce.ImportJobBase: Beginning import of tb_user
 // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 // :: INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
 // :: INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
 // :: INFO client.RMProxy: Connecting to ResourceManager at slaver1/192.168.19.131:
 // :: WARN security.UserGroupInformation: PriviledgedActionException as:hadoop (auth:SIMPLE) cause:org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://slaver1:9000/user/hadoop/tb_user already exists
 // :: ERROR tool.ImportTool: Encountered IOException running import job: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://slaver1:9000/user/hadoop/tb_user already exists
     at org.apache.hadoop.mapreduce.lib.output.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:)
     at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:)
     at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:)
     at org.apache.hadoop.mapreduce.Job$.run(Job.java:)
     at org.apache.hadoop.mapreduce.Job$.run(Job.java:)
     at java.security.AccessController.doPrivileged(Native Method)
     at javax.security.auth.Subject.doAs(Subject.java:)
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
     at org.apache.hadoop.mapreduce.Job.submit(Job.java:)
     at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:)
     at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:)
     at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:)
     at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:)
     at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:)
     at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:)
     at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:)
     at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:)
     at org.apache.sqoop.Sqoop.run(Sqoop.java:)
     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:)
     at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:)
     at org.apache.sqoop.Sqoop.runTool(Sqoop.java:)
     at org.apache.sqoop.Sqoop.main(Sqoop.java:)

 [hadoop@slaver1 sqoop-1.4.-cdh5.3.6]$ 

2、报错说以及存在了，hdfs://slaver1:9000/user/hadoop/tb_user already exists，首先根据路径找到了问题，先将这个路径上面的删除了，然后再执行的时候发现将mysql的数据表数据可以导入到hdfs分布式文件系统上面。

 [hadoop@slaver1 ~]$ hdfs dfs -rm -r /user/hadoop/tb_user
 // :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 // :: INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval =  minutes, Emptier interval =  minutes.
 Deleted /user/hadoop/tb_user
 [hadoop@slaver1 ~]$

3、执行如下所示：

[hadoop@slaver1 sqoop-1.4.-cdh5.3.6]$ bin/sqoop import \
> --connect jdbc:mysql://slaver1:3306/test \
> --username root \
> --password  \
> --table tb_user \
> --m
Warning: /home/hadoop/soft/sqoop-1.4.-cdh5.3.6/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/soft/sqoop-1.4.-cdh5.3.6/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
// :: INFO sqoop.Sqoop: Running Sqoop version: 1.4.-cdh5.3.6
// :: WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
// :: INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
// :: INFO tool.CodeGenTool: Beginning code generation
// :: INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_user` AS t LIMIT
// :: INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `tb_user` AS t LIMIT
// :: INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/soft/hadoop-2.5.-cdh5.3.6
Note: /tmp/sqoop-hadoop/compile/464d9aff412a6285fd9f6f4c6d16b4e6/tb_user.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
// :: INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/464d9aff412a6285fd9f6f4c6d16b4e6/tb_user.jar
// :: WARN manager.MySQLManager: It looks like you are importing from mysql.
// :: WARN manager.MySQLManager: This transfer can be faster! Use the --direct
// :: WARN manager.MySQLManager: option to exercise a MySQL-specific fast path.
// :: INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql)
// :: INFO mapreduce.ImportJobBase: Beginning import of tb_user
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
// :: INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
// :: INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
// :: INFO client.RMProxy: Connecting to ResourceManager at slaver1/192.168.19.131:
// :: INFO db.DBInputFormat: Using read commited transaction isolation
// :: INFO mapreduce.JobSubmitter: number of splits:
// :: INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526642793183_0001
// :: INFO impl.YarnClientImpl: Submitted application application_1526642793183_0001
// :: INFO mapreduce.Job: The url to track the job: http://slaver1:8088/proxy/application_1526642793183_0001/
// :: INFO mapreduce.Job: Running job: job_1526642793183_0001
// :: INFO mapreduce.Job: Job job_1526642793183_0001 running in uber mode : false
// :: INFO mapreduce.Job:  map % reduce %
// :: INFO mapreduce.Job:  map % reduce %
// :: INFO mapreduce.Job: Job job_1526642793183_0001 completed successfully
// :: INFO mapreduce.Job: Counters:
    File System Counters
        FILE: Number of bytes read=
        FILE: Number of bytes written=
        FILE: Number of read operations=
        FILE: Number of large read operations=
        FILE: Number of write operations=
        HDFS: Number of bytes read=
        HDFS: Number of bytes written=
        HDFS: Number of read operations=
        HDFS: Number of large read operations=
        HDFS: Number of write operations=
    Job Counters
        Launched map tasks=
        Other local map tasks=
        Total time spent by all maps in occupied slots (ms)=
        Total time spent by all reduces in occupied slots (ms)=
        Total time spent by all map tasks (ms)=
        Total vcore-seconds taken by all map tasks=
        Total megabyte-seconds taken by all map tasks=
    Map-Reduce Framework
        Map input records=
        Map output records=
        Input split bytes=
        Spilled Records=
        Failed Shuffles=
        Merged Map outputs=
        GC time elapsed (ms)=
        CPU time spent (ms)=
        Physical memory (bytes) snapshot=
        Virtual memory (bytes) snapshot=
        Total committed heap usage (bytes)=
    File Input Format Counters
        Bytes Read=
    File Output Format Counters
        Bytes Written=
// :: INFO mapreduce.ImportJobBase: Transferred  bytes in 38.0904 seconds (4.0168 bytes/sec)
// :: INFO mapreduce.ImportJobBase: Retrieved  records.
[hadoop@slaver1 sqoop-1.4.-cdh5.3.6]$ 

4、数据如下所示：

[hadoop@slaver1 ~]$ hdfs dfs -cat /user/hadoop/tb_user/part-m-
// :: WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
,张三,
,李四,
,王五,
,小明,
,小红,
,小别,
,,
,,
,,
,,
[hadoop@slaver1 ~]$