O'Reilly logo

Hands-On Natural Language Processing with Python by Rajalingappaa Shanmugamani, Rajesh Arumugam

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Encoder and postprocessing CBHG

To prepare the code for the CBHG module, we first implement two of its major elementary units—the 1-D convolution bank and the highway network:

def get_conv1dbank(K_, input_data):    conv=Conv1D(filters=128, kernel_size=1,                strides=1,padding='same')(input_data)    conv=BatchNormalization()(conv)    conv=Activation('relu')(conv)    for k_ in range(2,K_+1):        conv=Conv1D(filters=128, kernel_size=k_,                    strides=1,padding='same')(conv)        conv=BatchNormalization()(conv)        conv=Activation('relu')(conv)   return conv

With Keras 2, the development team of Keras decided to remove highway networks, probably because they were rarely used, and also because they are quite easy to implement. Since we are using Keras 2, we need to explicitly write ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required