Human-in-the-Loop Machine Learning

9 Advanced data annotation and augmentation

This chapter covers

• Evaluating annotation quality for subjective tasks
• Optimizing annotation quality control with machine learning
• Treating model predictions as annotations
• Combining embeddings/contextual representations with annotations
• Using search and rule-based systems for data annotation
• Bootstrapping models with lightly supervised machine learning
• Expanding datasets with synthetic data, data creation, and data augmentation
• Incorporating annotation information into machine learning models

For many tasks, simple quality control metrics aren’t enough. Imagine that you need to annotate images for labels like “Cyclist” and “Pedestrian.” Some images, such as a person pushing a bicycle, are inherently ...