Hands-On Design Patterns with Java

Transform

With the extracted data now accessible to us, we can implement the second step and transform the data into our desired form. A real-world implementation would include source code to convert the data. The transformation process would include the following sequential steps:

1. Determine how to chunk data.
2. Grab chunk.

3. Process records:
   • Convert fields as necessary.
   • Add default values as necessary.
   • Flag incomplete records.
4. Continue step 3 until all records in the current chunk are completed.
5. Repeat step 2 until all chunks have been processed.

The first step, determine how to chunk data, is important because the datasets are very large and server-side scripting would most certainly time out. You might, as an example, process 100 records ...