Nvidia patents system to automatically find and fix errors in AI apps
Nvidia describes a system that automatically tests AI applications to check their output quality. It creates detailed logs of the AI's internal steps and uses models to pinpoint exactly why performance dropped, then suggests fixes.
In the filing’s own words
In various examples, systems and methods are provided for automatic evaluation, monitoring, and root cause analysis for generative AI applications. Quality metric(s) may be generated for a generative AI application based at least on output(s) of the generative AI application during evaluation. Log(s) for component(s) of the generative AI application may be generated during evaluation and comprise data flow, processing steps, and performance metric(s) associated with the generative AI application. Indication(s) of a root cause of a regression for the generative AI application may be generated using model(s) based at least on the quality metric(s) for the generative AI application and the log(s) for the component(s) of the generative AI application. Operation(s) for addressing the regression for the generative AI application may be performed based at least on the indication(s) of the root cause of the regression for the generative AI application.
What exactly they claim (claim 1)
1. One or more processors comprising processing circuitry to: trigger an evaluation of a generative artificial intelligence (AI) application; generate one or more quality metrics for the generative AI application during the evaluation based at least on one or more outputs of the generative AI application; generate one or more logs for one or more components of the generative AI application during the evaluation, wherein the one or more logs for the one or more components of the generative AI application comprise data flow and processing steps executed by the one or more components of the generative AI application during the evaluation; detect a regression for the generative AI application based at least on at least one of the one or more quality metrics for the generative AI application or the one or more logs for the one or more components of the generative AI application; generate, using one or more models, a first output indicative of a root cause of the regression for the generative AI application based at least on at least one of the one or more quality metrics for the generative AI application or the one or more logs for the one or more components of the generative AI application; and perform one or more operations for addressing the regression for the generative AI application based at least on the first output indicative of the root cause of the regression for the generative AI application.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- SYSTEMS AND METHODS FOR AUTOMATIC EVALUATION, MONITORING, AND ROOT CAUSE ANALYSIS FOR GENERATIVE ARTIFICIAL INTELLIGENCE APPLICATIONS
- Publication number
- US20260278401A1
- Application number
- 19078729
- Filed
- Mar 13, 2025
- Published
- Sep 17, 2026
- Inventors
- 9
- Claims
- 20
Classified under
G06N 3/10 Physics & computingG06N 3/0475 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.