
4.5 Wisdom of crowds
In many real-life scenarios, it is hard to collect the actual labels for
training, because it is expensive or the labeling is subjective. To address
this issue, a new direction of research appeared in the last decade,
taking full advantage of the “wisdom of crowds” (Smith et al., 2005).
In simple words, wisdom of crowds enables the fast acquisition of
opinions from multiple annotators/experts.
Based on this intuition, wisdom of crowds was modeled using
Parasocial Consensus Sampling paradigm (Huang et al., 2010) for
data acquisition, which allows multiple crowd members to experience
the same situation. Parasocial Consensus ...