Creative Approaches Towards Development of Computing and Multidisciplinary IT Solutions for Society

16Generative Pre-Trained Transformer for Vietnamese Community-Based COVID-19 Question Answering

Tam Minh Vo^1,2* and Khiem Vinh Tran^1,2

¹University of Information Technology, Ho Chi Minh City, Vietnam

²Vietnam National University, Ho Chi Minh City, Vietnam

Abstract

Recent studies have provided empirical evidence of the wide-ranging potential of Generative Pre-trained Transformer (GPT), a pre-trained language model, in the field of natural language processing. GPT has been effectively employed as a decoder in state-of-the-art (SOTA) question-answering systems, yielding exceptional performances across various tasks. However, the current research landscape concerning GPT’s application in Vietnamese remains limited. This paper aims to address this gap by presenting an implementation of GPT-2 for community-based question answering, specifically focused on COVID-19 related queries in Vietnamese. We introduce a novel approach by conducting a comparative analysis of different Transformers vs SOTA models in the community-based COVID-19 question answering dataset. The experimental findings demonstrate that the GPT-2 models exhibit highly promising outcomes, outperforming other SOTA models as well as previous community-based COVID-19 question answering models developed for Vietnamese.

Keywords: Benchmark, COVID-19, question answering

16.1 Introduction

Community-based Question Answering (CQA) is a prominent task that involves soliciting answers from the collective intelligence of online ...