OpenAI patents streaming video and audio for faster voice AI replies
OpenAI has filed a patent for a system that processes live audio and video to power real-time voice interactions. The method sends audio at a steady rate and only key video frames to a generative engine, which then streams back a response to the user's device.
In the filing’s own words
Disclosed are systems, apparatuses, processes, and computer-readable media for real-time modes of a generative response engine. The present technology includes different technologies to improve real-time operation to create natural and dynamic interactions to user experiences. A method of the present technology includes receiving, by a computing service, first audio frames associated with a speech instance at a client device and first video frames, wherein the first audio frames are provided at a first rate and the first video frames are provided at a variable rate; providing keyframes from the first video frames to send to a generative response engine for inference operation; transmitting the first audio frames and the keyframes to the generative response engine; and transmitting, to the client device, a stream of generated data generated by the generative response engine based on the first audio frames and the keyframes.
What exactly they claim (claim 1)
1. A method comprising: receiving, by a computing service, first audio frames associated with a speech instance at a client device and first video frames, wherein the first audio frames are provided at a first rate and the first video frames are provided at a variable rate; providing keyframes from the first video frames to send to a generative response engine for inference operation; transmitting the first audio frames and the keyframes to the generative response engine; and transmitting, to the client device, a stream of generated data generated by the generative response engine based on the first audio frames and the keyframes.
Facts
- Company
- OpenAI (OpenAI OpCo, LLC.)
- Official title
- STREAMING AUDIO AND VIDEO FOR REAL-TIME VOICE INTERACTIONS GENERATIVE INFERENCE
- Publication number
- US20260311694A1
- Application number
- 19098795
- Filed
- Apr 2, 2025
- Published
- Oct 8, 2026
- Inventors
- 11
- Claims
- 20
Classified under
G10L 15/22 Physics & computingG10L 25/57 Physics & computingG10L 25/93 Physics & computingMore from OpenAI in AI & software
- OpenAI patents voice system that detects when you stop talking Sep 22, 2026
- OpenAI patents system to catch AI models cheating on tasks Sep 10, 2026
- OpenAI patents method to check AI thinking for cheating Sep 10, 2026
- OpenAI patents system that turns photos into new AI-generated images Sep 10, 2026
- OpenAI patents AI that answers questions using only your company's secure files Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.