Google patents voice assistant that asks before acting on commands

1. A computer-implemented method executed by data processing hardware that causes the data processing hardware to perform operations comprising: receiving audio data captured by a user device, the audio data comprising a first audio segment and a second audio segment, the first audio segment comprising a hotword configured to initiate processing of one or more terms following the hotword, the second audio segment comprising an action to perform on the user device or another device; determining that the first audio segment characterizes a soft hotword event; based on determining that the first audio segment characterizes the soft hotword event, determining that the second audio segment corresponds to a query specifying the action to perform on the user device or the other device; and based on determining that the first audio segment characterizes the soft hotword event and determining that the second audio segment corresponds to the query specifying the action to perform on the user device or the other device: providing a notification for output from the user device, the notification prompting a user associated with the user device to provide an affirmative input indication in order to trigger performance on the user device or the other device; activating a warm word model configured to detect, without performing speech recognition, a presence of both of: one or more spoken terms characterizing the affirmative input indication; and one or more other spoken terms characterizing a negative input indication, the negative input indication indicating that the user does not want the user device or the other device to perform the action specified by the query; receiving a speech input comprising the one or more other spoken terms characterizing the negative input indication; detecting, by the warm word model in the speech input, without performing speech recognition, the one or more other spoken terms characterizing the negative input indication; determining that the user failed to provide the affirmative input indication based on detecting the one or more other spoken terms characterizing the negative input indication; and based on determining that the user failed to provide the affirmative input indication, instructing the user device or the other device to not perform the action specified by the query.