R Deep Learning Projects

A simple benchmark implementation

Let's create a simple benchmark implementation and simulate text generation as a Markov chain. The idea is the following, we will estimate the probability of character c appearing after history h has been observed, where h has a fixed length. This length is called memory. For example, if we have a tiny corpus consisting of:

"My name is Pablo"

And we fix a memory length of 4, we get a training set that looks like this:

h	c
My n	a
y na	m
nam	e

Our task is to estimate the conditional probability distribution:

This conditional probability is obtained simply by estimating the number of times c appears ...