Spring Batch 远程分区和远程分块的区别

Partitioning is a master/slave step configuration that allows for partitions of data to be processed in parallel. Each partition is described via some metadata. For example, if you were processing a database table, partition 1 may be ids 0-100, partition 2 being 101-200, etc. For Spring Batch, a master step uses a Partitioner to generate ExecutionContexts that contain the metadata for each partition. These ExecutionContexts are distributed to slave step for processing by a PartitionHandler (for remote partitioning, the MessageChannelPartitionHandler is typically used). The slaves execute their step and return the resulting statuses for aggregation by the master.

Things to note about remote partitioning:

Input and output are local to the slaves. For example, if the input is a file, the slaves need access to the file.
Slaves need access to the JobRepository. Slaves are fully defined Spring Batch steps and so they need JobRepository access.
Remote Chunking

Remote chunking is similar to remote partitioning in that it is a master/slave configuration. However with remote chunking, the data is read at by the master and sent over the wire to the slave for processing. Once the processing is done, the result of the ItemProcessor is returned to the master for writing.

Things to note about remote chunking:

All I/O is done by the master.
The slaves handle processing only and therefore do not need JobRepository access.
Remote chunking is more I/O intensive than remote partitioning since the actual data is sent over the wire instead of metadata describing it.
I did a talk on scaling Spring Batch and do a demonstration of remote partitioning that you can watch here: http://www.youtube.com/watch?v=CYTj5YT7CZU

翻译：

分区是主/从步骤配置，允许并行处理数据分区。每个分区都通过一些元数据来描述。例如，如果您正在处理数据库表，那么分区1可能是ids 0-100，分区2是101-200等。对于Spring Batch，主步骤使用分区程序生成包含每个分区的元数据的ExecutionContext。这些ExecutionContexts被分配给从属步骤以供PartitionHandler处理（对于远程分区，通常使用MessageChannelPartitionHandler）。从站执行它们的步骤并返回由主站汇总的结果状态。

关于远程分区的注意事项：

输入和输出对于从站是本地的。例如，如果输入是文件，则从站需要访问该文件。
从站需要访问JobRepository。从属是完全定义的Spring批处理步骤，因此它们需要JobRepository访问。
远程分块

远程分块与远程分区类似，因为它是主/从配置。但是，使用远程组块时，数据由主站读取并通过线路发送给从站进行处理。处理完成后，ItemProcessor的结果将返回给主设备进行写入。

关于远程分块的注意事项：

所有I / O都由主人完成。
从站仅处理处理，因此不需要JobRepository访问。
远程分块比远程分区的I / O密集程度更高，因为实际数据是通过线路发送的，而不是描述它的元数据。
我做了一次关于缩放Spring Batch的演讲，并演示了可以在此观看的远程分区：http：//www.youtube.com/watch？v = CYTj5YT7CZU