Pentaho Data Integration Quick Start Guide

Combining two different datasets into a single dataset

Continuing with the same exercise, suppose that we also want to read one of the files with a survey taken in 2017, and create a single dataset with information from both files. If you inspect the files, however, you will notice that the newest files don't have the same format as those with surveys taken in 2015. The fields are in a different order, and also there are new fields. If we intend to read both an old and a new file, we cannot do so using a single step. Let's work on the solution to this, as follows:

1. Open the transformation created previously.
2. Add a new Text file input step and configure it to read the tomslee_airbnb_amsterdam_1476_2017-07-22.csv file.
3. Run a preview, in order ...