O'Reilly logo

Talend Open Studio Cookbook by Rick Barton

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Using reload at each row to process real-time / near real-time data

Prior to attempting this recipe, you will need to ensure that you have an active MySQL database and have updated the context variables within the context MySQL to contain your database and login details. See the recipe Setting up a database connection in Chapter 7, Working with databases, for details on how to do this.

As we mentioned in the recipe Joining using tMap, tMap will load the join data into memory prior to processing the main input rows. This works fine for a batch processing model, because the overhead of loading large lookups in memory is offset against the efficiency in processing the joins against the data held in memory.

This paradigm does not however work in a real-time ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required