OpenAI patents system to save parts of AI prompts to speed up responses
This filing describes a method for cloud services to store and reuse parts of user instructions, known as prompts, to generate faster responses. It involves creating a digital fingerprint, or hash, of the prompt to identify which AI model should handle the request and how much previous context is needed.
In the filing’s own words
Disclosed are systems, apparatuses, processes, and computer-readable media for caching prompts for a generative response engine. The present technology includes receiving, by a cloud computing service, a request including a first prompt including a natural language task to perform, wherein the request include an access key for accessing the cloud computing service; identifying a generative response engine for generating a response to the natural language task based on contents of the natural language task; transmitting the first prompt and a hash to the generative response engine; and receiving the response to the natural language task from the generative response engine, the response including a number of input tokens.
What exactly they claim (claim 1)
1. A method, comprising: receiving, by a cloud computing service, a first prompt including a natural language task to perform, wherein the first prompt includes at least one binary data portion; generating a hash associated with a portion of the first prompt; identifying a generative response engine for generating a response to the natural language task based on the hash; transmitting the first prompt to the generative response engine; and receiving the response to the natural language task from the generative response engine, the response including a number of binary tokens of the binary data portion that are activated based on a second prompt that preceded the first prompt.
Facts
- Company
- OpenAI (OpenAI OpCo, LLC.)
- Official title
- PROMPT CACHING IN GENERATIVE RESPONSE ENGINES
- Publication number
- US20260278026A1
- Application number
- 19543572
- Filed
- Feb 18, 2026
- Published
- Sep 17, 2026
- Inventors
- 4
- Claims
- 20
Classified under
G06F 16/9574 Physics & computingG06F 16/33295 Physics & computingG06Q 30/0239 Physics & computingG06Q 30/04 Physics & computingMore from OpenAI in AI & software
- OpenAI patents voice system that detects when you stop talking Sep 22, 2026
- OpenAI patents system to catch AI models cheating on tasks Sep 10, 2026
- OpenAI patents method to check AI thinking for cheating Sep 10, 2026
- OpenAI patents system that turns photos into new AI-generated images Sep 10, 2026
- OpenAI patents AI that answers questions using only your company's secure files Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.