Amazon patents system that auto-updates live data pipelines when code changes

1. A cloud provider system comprising: one or more processors; a code repository service, executed by the one or more processors, storing a plurality of user tasks, wherein each task of the plurality of user tasks when executed by the one or more processors is configured to perform an operation as part of an extract, transform, and load (ETL) workflow for extracting data from one or more data sources, transforming the data, and loading the transformed data onto one or more destinations; and a workflow integration and deployment service comprising a workflow manager with one or more workflow testing rules and in network communication with the code repository service, wherein the workflow integration and deployment service when executed by the one or more processors is configured to at least: receive, from the code repository service, an indication that a code change has been made to a task associated with a user account; determine that the task is part of a live ETL workflow that has been deployed based on the ETL workflow on a first set of computing resources assigned to the user account and configured to receive input data from a source associated with the user account, transform the input data, and output the transformed input data to a destination associated with the user account; determine that the code change made to the task is to be propagated to the live ETL workflow such that any additional data subsequently received by the live ETL workflow is transformed according to the code change made to the task; identify a set of dependencies associated with the task based at least in part on the code change made to the task, wherein the set of dependencies associated with the task includes code dependency information usable to determine a position of the task within an updated ETL workflow, and wherein the set of dependencies includes at least time dependency information usable to determine a time or time range at which the task is to be executed and cluster dependency information usable to determine a cluster of compute capacity in which the task is to be executed; generate, using the workflow manager and based on the code dependency information, the time dependency information, and the cluster dependency information, the updated ETL workflow that includes the code change made to the task and satisfies the set of dependencies associated with the task; test, using the workflow manager, the updated ETL workflow according to the one or more workflow testing rules associated with the user account to lower error rates; update and deploy the live ETL workflow according to the updated ETL workflow based on the testing; and automatically execute the updated live ETL workflow to cause the updated live ETL workflow to receive additional input data from the source, transform the additional input data according to the code change made to the task, and output the transformed additional input data to the destination.