Google patents method to fix biased data in AI models

1. A computer-implemented method to reduce bias in a machine-learned classification model configured without post-processing bias calibration of model outputs, the method comprising: obtaining, by one or more computing devices, a training dataset comprising a plurality of training examples, each training example comprising an example input and a respective example label applied to the example input, wherein the example labels of the training dataset exhibit a bias against one or more subgroups of the example inputs; initializing, by the one or more computing devices, a plurality of weights that are respectively associated with the plurality of training examples; for each of one or more training iterations: determining, by the one or more computing devices, one or more constraint violation values for the machine-learned classification model on the training dataset relative to one or more fairness constraints applied to the one or more subgroups of the example inputs, wherein the one or more fairness constraints comprise a disparate impact constraint; updating, by the one or more computing devices, one or more re-weighting control values respectively associated with the one or more fairness constraints based at least in part on the one or more constraint violation values; modifying, by the one or more computing devices, at least one of the plurality of weights associated with the plurality of training examples based at least in part on the one or more re-weighting control values to form a plurality of modified weights; and re-training, by the one or more computing devices, the machine-learned classification model using the training dataset weighted according to the plurality of modified weights; wherein both a true positive re-weighting control value and a false positive re-weighting control value are associated with at least one of the one or more fairness constraints.