Talend Open Studio Cookbook

Using reload at each row to process real-time / near real-time data

Prior to attempting this recipe, you will need to ensure that you have an active MySQL database and have updated the context variables within the context MySQL to contain your database and login details. See the recipe Setting up a database connection in Chapter 7, Working with databases, for details on how to do this.

As we mentioned in the recipe Joining using tMap, tMap will load the join data into memory prior to processing the main input rows. This works fine for a batch processing model, because the overhead of loading large lookups in memory is offset against the efficiency in processing the joins against the data held in memory.

This paradigm does not however work in a real-time ...