Amazon patents system that picks the right AI model for your request
Amazon has filed a patent for a routing system that decides which specific AI model should handle a user's request. It uses a smaller AI to estimate factors like cost, speed, and output size before sending the task to the best-fit model.
In the filing’s own words
A user of a machine learning model service may lack information about the suitability of particular generative machine learning models for particular tasks. The user may benefit from a machine learning model router that outputs intelligent decisions on which generative machine learning model is more suitable than other generative machine learning models for a particular prompt with respect to known or unknown hardware and process constraints. Suitability may be based on a variety of factors, such as quality of the model for the particular prompt, expected resource expenditure for the particular prompt such as cost or latency, and other factors such as which computing device or devices hosts the generative machine learning model.
What exactly they claim (claim 1)
1. A system, comprising: one or more computing devices configured to: receive a generative machine learning model prompt which is to be processed by at least one of a set of generative machine learning models, wherein the set of generative machine learning models comprises more than two generative machine learning models; estimate consideration factors, using at least one estimator machine learning model which uses a prompt embedding and a set of model identity embeddings which correspond with respective ones of the set of generative machine learning models, wherein: the consideration factors are based on the generative machine learning model prompt; and at least one of the consideration factors is based on an estimated amount of output that will be generated by a given one of the set of generative machine learning models; select, using the consideration factors, the at least one of the set of generative machine learning models; and route the generative machine learning model prompt to the selected at least one of the set of generative machine learning models.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- INTELLIGENT PROMPT ROUTING ACROSS MACHINE LEARNING MODELS
- Publication number
- US20260300841A1
- Application number
- 19096358
- Filed
- Mar 31, 2025
- Published
- Oct 1, 2026
- Inventors
- 10
- Claims
- 20
Classified under
G06N 20/20 Physics & computingMore from Amazon in AI & software
- Amazon patents system that cites sources for AI answers Sep 29, 2026
- Amazon patents tech to swap words in movies using actor's voice Sep 8, 2026
- Amazon patents ML tool to add graphics to live sports from moving cameras Sep 15, 2026
- Amazon patents shopping assistant that remembers your search filters Oct 6, 2026
- Amazon patents system where AI agents split tasks and talk to each other Oct 20, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.