Data Science with Semantic Technologies

3Latest Trends in Language Processing to Make Semantic Search More Semantic

Devi Kannan, Mamatha T. and Farhana Kausar

DOI: 10.1201/9781003310792-3

Contents

3.1Introduction

3.2Goal of Question and Answering

3.3Open Domain

3.4Open Domain Models for Retriever Reader

3.4.1Retriever Reader Model

3.4.2Limitations and Hypotheses

3.5Open Domain Models for Retriever Generator Model

3.5.1BERT and its Function

3.5.2Model of Masked Language (MLM)

3.5.3Next Sentence Prediction (NSP)

3.5.4Transformer Architecture in BERT Model

3.5.4.1Input to the Encoder

3.5.4.2Token Embedding

3.5.4.3Segment Embedding

3.5.4.4Positional Embedding

3.5.4.5Encoder Architecture

3.5.4.5.1Multi-Head Attention Block

3.5.4.5.2Self-Attention

3.5.4.5.3Muti-Head Attention

3.5.4.6 ...