Amazon patents method to spot errors in AI models using data fingerprints

1. A computing system comprising: data processing hardware; and memory in communication with the data processing hardware, the memory storing instructions, wherein execution of the instructions by the data processing hardware causes the data processing hardware to: obtain, from a user computing device, an input, wherein the input comprises a request to generate an output using a machine learning model, wherein the computing system generates a prompt for the machine learning model based on the input, and wherein the prompt comprises at least a portion of the input and at least a portion of additional data; provide, to a second computing system, the prompt, wherein the second computing system implements the machine learning model; identify, in real time, a vector embedding of a first plurality of vector embeddings generated by the machine learning model based on the prompt; obtain, from a data store, a first set of data associated with the machine learning model, wherein each machine learning model of a plurality of machine learning models is associated with a respective set of data of a plurality of sets of data stored by the data store, wherein the first set of data comprises a second plurality of vector embeddings; compare the vector embedding generated by the machine learning model with the second plurality of vector embeddings; determine a manner of processing a second set of data associated with the machine learning model based on comparing the vector embedding generated by the machine learning model with the second plurality of vector embeddings and based on one or more of a query vector, a key vector, or a value vector associated with an attention mechanism of the machine learning model, wherein the second set of data comprises a respective input to the machine learning model or a respective output of the machine learning model; and process the second set of data according to the manner of processing the second set of data.