Nvidia patents system that sorts AI questions to the right server
Nvidia describes a method for directing user questions to the correct AI server in a large system. It uses keywords, text analysis, or a set order to decide where to send the data, and servers can reject questions if they are not the right fit.
In the filing’s own words
In various examples, techniques for routing queries in multi-backend artificial intelligence systems is described herein. Systems and methods described herein may include a frontend that uses various routing techniques to select backends for routing queries. For instance, the frontend may use at least a keyword routing technique that routes queries to backends based at least on identifiers (e.g., names, codenames, etc.) included in the queries, a classification routing technique that routes queries to backends based at least on performing text classification (e.g., using one or more models, embedding comparisons, etc.), and/or a serialized routing technique that routes queries to backends based at least on an order (e.g., either static or dynamic). In some examples, the backends may be configured to initially respond to the frontend to indicate whether the backends accept or reject queries, such that the frontend is able to reroute queries if needed.
What exactly they claim (claim 1)
1. One or more processors comprising: processing circuitry to: store first vectors corresponding to first queries associated with backends; receive input data representative of a second query; generate, based at least on the input data, a second vector corresponding to the second query; compare the second vector to the first vectors to determine scores associated with the first queries; determine, based at least on the scores, that one or more of the first vectors are related to the second vector; select, based at least on the one or more of the first vectors being related to the second vector, a backend from the backends; and provide at least a portion of the input data to the backend for processing.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- TECHNIQUES FOR ROUTING QUERIES IN MULTI-BACKEND ARTIFICIAL INTELLIGENCE SYSTEMS
- Publication number
- US20260277971A1
- Application number
- 19090268
- Filed
- Mar 25, 2025
- Published
- Sep 17, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06F 16/3347 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.