
59
will Crowd solve All problems in testing?
If I have a wrong crowd that is not relevant for me, evaluating my
design and giving me wrong feedback, based on which I decide my
action plans, am I not being misguided by the crowd’s input? In such
open-ended crowd tests, timing is another challenge. How do I know
for how long I should keep my A/B tests open to the crowd? Recently,
there was an example from Google, where the A/B test results were
drastically dierent when they extended the testing duration and had
it run for more time. Had Google made a feature decision based on the
results of the rst round of A/B tests, the impact would have been ...