When we talk in terms of mixing word embeddings and neural networks, convolutional networks are something that have yielded good results. CNNs are created by applying several layers of convolutions with nonlinear activation functions such as ReLu or tanh applied to the results.
Lets talk a little bit about what a convolution means. A convolution of any function with respect to another function is the integral that expresses the amount of overlap of one function as it is passed over the other function. You can think of this as blending one function into another. In signal theory, this is how experts understand convolutions: the output signal is a convolution of the input signal with the impulse response of the ...
