Amazon patents secure way to test multiple AI models in private cloud
Amazon describes a method for securely testing different artificial intelligence models within its cloud network without using the public internet. The system assigns users to specific model versions based on their identity and routes requests through private channels to keep data secure.
In the filing’s own words
Techniques for secure multi-model hosting and experimentation in a cloud provider network are described. A request to perform an action involving one of multiple machine learning (ML) models is received at a cloud provider network. A user associated with the request can be consistently mapped to a particular experience from multiple experiences of an experiment. Based on this experience, a request to use a ML model can be sent by a data plane component that includes a path value associated with the experience. The request can be sent from the data plane component to a private connectivity endpoint, which securely provides the request into another virtual private cloud including the ML models without the request transiting the public internet. The request can be routed via path-based routing and the included path value to a ML model providing the experience.
What exactly they claim (claim 1)
1. A computer-implemented method comprising: receiving, at a service of a cloud provider network, a request from a user to generate source code; selecting, based on a user identifier of the user, an experience associated with the user, wherein the selecting comprises: generating a bucket identifier via use of a deterministic function, and identifying the experience based on use of the bucket identifier; generating an application layer resource identifier comprising a path, wherein the path includes a path component that corresponds to the identified experience; transmitting a request to perform an inference to a private connectivity endpoint, the request including the application layer resource identifier comprising a uniform resource locator (URL) address; routing the request to a first generative machine learning (ML) model, of multiple generative ML models, based on the path component in the URL, wherein: the routing is performed by an application layer load balancer that uses application-layer routing rules, including path-based routing rules, to select among the multiple generative ML models, the private connectivity endpoint provides access to each of the multiple generative ML models and different path components in the URLs route to different generative ML models, and the request is routed entirely within the cloud provider network and does not transit the public internet; generating an output via use of the first generative ML model, the output comprising the source code; and transmitting the output to a computing device of the user to be presented.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- Secure multi-model hosting and experimentation in a cloud provider network
- Patent number
- US12764034B1
- Application number
- 18797155
- Filed
- Aug 7, 2024
- Granted
- Oct 13, 2026
- Inventors
- 12
- Claims
- 20
Classified under
H04L 41/16 Electricity & electronicsH04L 41/14 Electricity & electronicsH04L 67/10 Electricity & electronicsMore from Amazon in AI & software
- Amazon patents system that cites sources for AI answers Sep 29, 2026
- Amazon patents tech to swap words in movies using actor's voice Sep 8, 2026
- Amazon patents ML tool to add graphics to live sports from moving cameras Sep 15, 2026
- Amazon patents shopping assistant that remembers your search filters Oct 6, 2026
- Amazon patents system where AI agents split tasks and talk to each other Oct 20, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.