Incentives Can Reduce Bias in Online Employer Reviews

Online reviews are widely used to propagate the reputations of employers. However, existing research suggests that online reviews often suffer from selection bias - people with extreme opinions are more motivated to select into sharing them than people with moderate opinions, resulting in more extreme ratings. The authors find that providing incentives for reviewing can reduce this selection bias, because incentives can mitigate the motivational deficit of people who hold moderate opinions. Using data from Glassdoor, a leading employer review website, the authors find that incentivized ratings are less extreme than voluntary (self-selected) ratings. The likely bias in the distribution of voluntary reviews can affect workers’ choice of employers. The authors complement their investigation with a randomized experiment that provides a cleaner test of selection bias. The authors again find that voluntary reviews of employers lead to more extreme reviews compared to non-voluntary reviews. Moreover, providing relatively high monetary rewards or a pro-social cue as incentives for reviewing also lowers the relative prevalence of extreme reviews. Although voluntary employer reviews often suffer from selection bias, incentives can reduce it and help workers make more informed employment choices.