Creative Approaches Towards Development of Computing and Multidisciplinary IT Solutions for Society

13Deep Learning Framework for Detecting, Classifying, and Recognizing Invoice Metadata

Nhat Quang Doan¹, Van Tang Nguyen^2*, Anh Tuan Giang¹, Van Trung Doan³ and Dang Bui Hai⁴

¹University of Science and Technology of Hanoi, Vietnam Academy of Science and Technology, Hanoi, Vietnam

²Faculty of Technology and Data Science Hanoi Foreign Trade University, Hanoi, Vietnam

³Faculty of Information Technology, Hanoi University of Industry, Hanoi, Vietnam

⁴KSE Software JSC, Hanoi, Vietnam

Abstract

In general, images or invoice images, are rich information sources; however, processing large amounts of invoices in finance requires extensive human work and material resources. The demand for a technological tool to process invoices is essential. Therefore, this study proposes a framework based on deep learning techniques for recognizing specific invoice information (or metadata). The deep learning model used in this research includes the classic architecture of Convolutional Neural Networks and its other variations, such as VGG-16 and Faster R-CNN. The complete process includes image processing series, model training for field detection and classification, and optical character recognition (OCR) for the detected fields. The proposed framework collects relevant information from invoice images and converts them into a text format to facilitate invoice management and transform them into structured data. The framework performance shows a significant value in recognizing invoice metadata compared ...