O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

The encoder

The encoder takes a sequence of characters as input, where each character is represented by a one-hot vector. An embedder is then used to project the input into a continuous space. Remember that, because of their high-dimensionality and their sparsity, one-hot encoded vectors can lead to computational inefficiency, if they are not used with techniques that exploit these characteristics. An embedder allows for significantly reducing the size of the representation space. Besides, using an embedder allows us to learn about the relationships between the different characters of our vocabulary. 

The embedding layer is then followed by a pre-net, which is a set of non-linear transformations. Basically, it is comprised of two consecutive, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required