August 2014
Beginner to intermediate
304 pages
7h 10m
English
We can use Redis and execnet together to do distributed word scoring. In the Calculating high information words recipe in Chapter 7, Text Classification, we calculated the information gain of each word in the movie_reviews corpus using a FreqDist and ConditionalFreqDist. Now that we have Redis, we can do the same thing using a RedisHashFreqDist and a RedisConditionalHashFreqDist, and then store the scores in a RedisOrderedDict. We can use execnet to distribute the counting in order to get a better performance out of Redis.
Redis, redis-py, and execnet must be installed, and an instance of redis-server must be running on localhost.
We start by getting a list of (label, words) ...