For testing, we need to set the expected values, which we can compare with the actual values produced by the application that processes real data. But real data changes slightly from run to run, and an attempt to predict the resultant values either makes the test fragile or forces the introduction of a huge margin of error, which may effectively defeat the purpose of testing.
We cannot even mock the generated data (as we did in the case of unit testing) because we are at the integration stage and have to use the real data.
One possible solution would be to store the incoming real data and the result our application produced in the database. Then, a domain specialist can walk through each record and assert whether the results ...
