Incremental import (Simple)

Incremental import means importing the new version of records or the latest inserted records from the RDBMS table into HDFS.

Getting ready

We can control the incremental import by using the arguments listed in the following table:



--check-column <column-name>

The value of this column is used to determine the rows to be imported during the import process.

--incremental <incremental-type>

Specifies the type of incremental mode. Possible values are append and lastmodified.

--last-value <value>

Specifies the last value or the maximum value of the check column from the previous import. All the records whose check column value is greater than the value of the –last-value argument ...

Get Instant Apache Sqoop now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.