Google patents method to make AI agents learn from less data

1. A method for training an action selection neural network system having a plurality of network parameters and for controlling an agent interacting with an environment, the method comprising: obtaining a set of one or more transition sequences, each transition sequence comprising a respective transition at each of a plurality of time steps, wherein each transition includes: (i) an observation received at the time step, (ii) an action performed in response to the observation, and (iii) one or more rewards received as a result of performing the action; computing a reinforcement learning loss for the set of transition sequences, comprising, for each transition in each transition sequence and for each of one or more return computation schemes in a set of one or more return computation schemes: processing an action selection input comprising the observation at the time step using an action selection neural network of the action selection neural network system that corresponds to the return computation scheme and in accordance with current values of the network parameters to generate an action score for the action performed in response to the observation; processing the action selection input using a target action selection neural network that corresponds to the return computation scheme to generate a target action score for the action performed in response to the observation, wherein the target action selection neural network has target values of the network parameters that are constrained to change more slowly than the current values of the network parameters during the training; and determining whether to include a loss for the transition and for the return computation scheme in the reinforcement learning loss based on the action score for the action performed in response to the observation and the target action score for the action performed in response to the observation; and training the action selection neural network system using the computed reinforcement loss.