Apple patents audiobooks that create custom stories and images from your voice
Apple has patented a system that lets users ask a virtual assistant to create a personalized story by combining their request with personal details. The system uses machine learning to generate both the text of the story and matching images, which are then synchronized and played back as an interactive audiobook.
In the filing’s own words
Disclosed are systems, methods, and computer-readable media for generating interactive content of audiobooks through a natural language interface. The disclosed technology generates a system prompt based on a combination of a user prompt and personal information of the user or of the user's environment. This combination can then be input into a multimodal machine learning model or multiple unimodal machine learning models to create both text and image outputs corresponding to the requested story line. The story can then be presented to the user and edited as needed, in some embodiments using the same content generation service that produced the system prompt to begin with.
What exactly they claim (claim 1)
1. A method for generating a narrative with corresponding visuals from a user prompt, the method comprising: receiving, by a virtual assistant bot, a request to create a story, wherein the request to create the story includes a user-provided prompt that includes a narrative seed from which to create the story; generating, by the virtual assistant bot, a narrative generation prompt from a combination of a system prompt and the user-provided prompt; sending, by the virtual assistant bot, the narrative generation prompt to a generative content generation service, wherein the generative content generation service generates narrative text corresponding to the story based on the narrative generation prompt, wherein the generative content generation service further generates visual media prompts corresponding to the narrative text, and wherein the visual media prompts include text that represents instructions to generate visuals; generating, by the generative content generation service, the visuals in response to the visual media prompts; receiving, by a story application and from the generative content generation service, the narrative text and the visuals in a style defined by the narrative generation prompt; synchronizing, by the story application, the visuals with the narrative text to create the story; and presenting, by the story application, the story to a user, wherein the story is presented as a series of segments.
Facts
- Company
- Apple (Apple Inc.)
- Official title
- On demand interactive content generation in audiobooks through a natural language interface
- Patent number
- US12762323B2
- Application number
- 18593372
- Filed
- Mar 1, 2024
- Granted
- Oct 13, 2026
- Inventors
- 3
- Claims
- 21
Classified under
G06T 11/60 Physics & computingG06F 40/103 Physics & computingMore from Apple in Audio & voice
- Apple patents head-worn device that adds spatial audio to external speakers Sep 8, 2026
- Apple patents audio system that places virtual speakers around visual objects Sep 24, 2026
- Apple patents speaker layout to hide device noise Sep 1, 2026
- Apple patents on-device audio transcription for video calls Sep 15, 2026
- Apple patents system that remembers where you send music Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.