August 2017
Beginner to intermediate
284 pages
6h 51m
English
This repartitioning operation enables you to partition the stream on the basis of the fields in the tuples. For example, if you want all the tweets from a particular user to go to the same target partition, then you can partition the tweet stream by applying partitionBy to the username field in the following manner:
mystream.partitionBy(new Fields("username")).each(new Fields("username","text"), new myFilter()).parallelismHint(2)
The partitionBy operation applies the following formula to decide the target partition:
Target Partition = hash(fields) % (number of target partition)
As the preceding formula shows, the partitionBy operation calculates the hash of the input fields to decide the target partition. ...
Read now
Unlock full access