Learning Data Mining with Python - Second Edition

Extracting character n-grams

We are going to use our CountVectorizer class to extract character n-grams. To do that, we set the analyzer parameter and specify a value for n to extract n-grams with.

The implementation in scikit-learn uses an n-gram range, allowing you to extract n-grams of multiple sizes at the same time. We won't delve into different n-values in this experiment, so we just set the values the same. To extract n-grams of size 3, you need to specify (3, 3) as the value for the n-gram range.

We can reuse the grid search from our previous code. All we need to do is specify the new feature extractor in a new pipeline and run it:

pipeline = Pipeline([('feature_extraction', CountVectorizer(analyzer='char', ngram_range=(3,3))),