Microsoft patents automated system to keep hiring AI models accurate

1. A system comprising: a processing circuit; and a memory having instructions stored thereon, which, when executed by the processing circuit, cause the system to: obtain a first plurality of data samples, each of the data samples indicating a value for a first variable and a value for a second variable; identify a generalized linear mixed effect (GLMix) model to train with a first machine learning algorithm, the GLMix model having a global model and a set of different types of random effects models, wherein each type of random effects model corresponds to a different variable in the first plurality of data samples, and wherein a corresponding instance of each type of random effects model is trained using the value for the first variable or the value for the second variable; generate a set of random numbers using a random number generator, the random number generator using output of a hash function as a seed therefor, wherein the hash function takes as input a value for each variable to which a random effects model in the GLMix model corresponds, wherein the hash function is based on a combination of random effects model types; assign a corresponding one of the set of random numbers to the data samples from the first plurality of data samples; randomly select data samples from the first plurality of data samples, based on the corresponding random number assigned thereto, to assign to a first training set or a first holdout set, wherein the global model is trained using data samples in the first training set; train a first iteration of the GLMix model using the first training set; obtain a second plurality of data samples, the data samples in the second plurality of data samples indicating a value for the first variable and a value for the second variable, at least some of the second plurality of data samples being identical to at least some of the first plurality of data samples; randomly select data samples from the second plurality of data samples to assign to a second training set or a second holdout set using output of the hash function as a seed to the random number generator; train a second iteration of the GLMix model using the second training set; test both the first iteration of the GLMix model and the second iteration of the GLMix model using the second holdout set.