Amazon patents smarter caching for vector databases
Amazon describes a system that uses a router to decide which cache should store search results in a vector database. By checking a partial index search, the router directs similar requests to the same cache, increasing the chance that the data is already there.
In the filing’s own words
A vector-labeled database may improve a cache hit rate for read requests by performing a partial approximate nearest neighbor index search at a router to designate the cache that results of a read request are to be written to. The selected cache may be associated with a particular node of a layer of an approximate nearest neighbor index search. The router may direct all reads with search vectors close to the particular node to the associated cache.
What exactly they claim (claim 1)
1. A system, comprising: a router; and one or more computing resources configured to implement a set of caches and a vector-labeled database, wherein results of a prior read request are added to a given cache in response to the prior read request; wherein the router is configured to: receive a read request for one or more data items associated with one or more approximate nearest neighbor vectors of a request vector, wherein the one or more approximate nearest neighbor vectors are maintained in the vector-labeled database; determine a portion of vector space for the request vector using at least a first layer of an approximate nearest neighbor index search; select, based on the determined portion of vector space for the request vector, a particular cache of the set of caches; cause the particular cache to be checked for the one or more approximate nearest neighbor vectors of the request vector; and cause the one or more data items associated with the one or more approximate nearest neighbor vectors to be returned in response to the request.
Facts
- Company
- Amazon (Amazon Technologies, Inc.)
- Official title
- CLUSTER AWARE CACHING FOR VECTOR DATABASE
- Publication number
- US20260288835A1
- Application number
- 19085883
- Filed
- Mar 20, 2025
- Published
- Sep 24, 2026
- Inventors
- 2
- Claims
- 20
Classified under
G06F 16/3347 Physics & computingG06F 16/322 Physics & computingMore from Amazon in AI & software
- Amazon patents system that cites sources for AI answers Sep 29, 2026
- Amazon patents tech to swap words in movies using actor's voice Sep 8, 2026
- Amazon patents ML tool to add graphics to live sports from moving cameras Sep 15, 2026
- Amazon patents shopping assistant that remembers your search filters Oct 6, 2026
- Amazon patents system where AI agents split tasks and talk to each other Oct 20, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.