Semantic AI in Knowledge Graphs

4MSE**: Multi-Modal Semantic Embeddings for Datasets with Several Positive Matchings¹

Jérémie Huteau, Adrian Basarab, and Florence Dupin de Saint-Cyr

IRIT, Toulouse University, France

DOI: 10.1201/9781003313267-4

Contents

• 4.1 Introduction

• 4.2 Proposed Model: MSE**

  • 4.2.1 Definitions and Notations

  • 4.2.2 Loss Function

• 4.3 Particular Issues Raised within MSE** Framework

  • 4.3.1 Optimizing the Hardest Negative Selection Process

  • 4.3.2 Data Augmentations for Multi-Modal Tuples of Sets

• 4.4 Implementation

  • 4.4.1 Dataset

  • 4.4.2 Model

  • 4.4.3 Triplet Loss

  • 4.4.4 Libraries Used

  • 4.4.5 Model Training

  • 4.4.6 Results and Analysis

• 4.5 Conclusion

• References

4.1 Introduction

Understanding an image or a text, even independently, is an extremely complex task, ...