Microsoft patents system to move AI tasks between cloud servers

1. A system for managing artificial intelligence (AI) workloads in a cloud infrastructure platform, the system comprising: a processor of the cloud infrastructure platform; and a memory of the cloud infrastructure platform comprising computer program code, the memory and the computer program code configured to, with the processor, cause the processor to: execute a first AI workload on a first resource, the first resource comprising first AI accelerator hardware, the first AI workload utilizing the first AI accelerator hardware for AI training or inferencing an AI model; execute a graphics processing unit (GPU)-related application programming interface (API) call that generates a physical handle to a GPU object, the GPU object comprising at least one of a stream, an event, or a library context; store the GPU-related API call in a log; intercept, by a checkpointing system, the physical handle generated by the first AI accelerator hardware; return, to the AI workload, a virtual handle in place of the physical handle, wherein the virtual handle is stored in a host-side data structure for use in a subsequent GPU-related API call; maintain, by the checkpointing system, a mapping between the virtual handle and the physical handle generated by the first AI accelerator hardware; checkpointing the AI workload by saving a checkpoint state comprising the log and the mapping; migrate the first AI workload to a second resource, the second resource being distinct from the first resource and comprising second AI accelerator hardware; and restore the checkpoint state of the first AI workload on the second resource, the restoring comprising: replaying the log, including the GPU-related API call, on the second resource; generating a new physical handle corresponding to the GPU object; updating the mapping to associate the virtual handle with the new physical handle; and resuming execution of the AI workload using the virtual handle.