Average Profits of Prejudiced Algorithms

We investigate the level of success a firm achieves depending on which of two common scoring algorithms is used to screen qualified applicants belonging to a disadvantaged group. Both algorithms are trained on data generated by a prejudiced decision-maker independently of the firm. One algorithm favors disadvantaged individuals, while the other algorithm exemplifies prejudice in the training data. We deliver sharp guarantees for when the firm finds more success with one algorithm over the other, depending on the prejudice level of the decision-maker.