Hands-On Natural Language Processing with Python

Encoder and postprocessing CBHG

To prepare the code for the CBHG module, we first implement two of its major elementary units—the 1-D convolution bank and the highway network:

def get_conv1dbank(K_, input_data):    conv=Conv1D(filters=128, kernel_size=1,                strides=1,padding='same')(input_data)    conv=BatchNormalization()(conv)    conv=Activation('relu')(conv)    for k_ in range(2,K_+1):        conv=Conv1D(filters=128, kernel_size=k_,                    strides=1,padding='same')(conv)        conv=BatchNormalization()(conv)        conv=Activation('relu')(conv)   return conv

With Keras 2, the development team of Keras decided to remove highway networks, probably because they were rarely used, and also because they are quite easy to implement. Since we are using Keras 2, we need to explicitly write ...