Hands-On Markov Models with Python

An HMM-based tagger

Now, we will try to build a POS tagger using HMM and hopefully it will improve our prediction performance. We will first define some helper functions:

def unigram_counts(sequences):    return Counter(sequences)tags = [tag for i, (word, tag) in enumerate(data.training_set.stream())]tag_unigrams = unigram_counts(tags)

def bigram_counts(sequences):    d = Counter(sequences)    return dtags = [tag for i, (word, tag) in enumerate(data.stream())]o = [(tags[i],tags[i+1]) for i in range(0,len(tags)-2,2)]tag_bigrams = bigram_counts(o)

def starting_counts(sequences):    d = Counter(sequences)    return dtags = [tag for i, (word, tag) in enumerate(data.stream())]starts_tag = [i[0] for i in data.Y]tag_starts = starting_counts(starts_tag)

def ending_counts(sequences): ...