Nvidia patents method for consistent AI images without retraining
Nvidia describes a technique that uses an existing AI model to create multiple images of the same character or object across different scenes. It works by sharing internal data between the images as they are generated, ensuring the subject looks the same in every picture without needing to retrain the model.
In the filing’s own words
Embodiments of the present disclosure relate to training-free consistent text-to-image generation. A pre-trained text-to-image diffusion model is leveraged to generate images depicting a consistent subject for diverse prompts describing scenes. Inputs to the model are a text description of at least one subject with prompts (scene text descriptions) describing scenes, where each prompt is associated with a different generated image and the text description is used for all images that depict the subject. Internal activations (intermediate data) computed by the model during generation of the different images are shared for generation of the different images. A subject-driven shared attention block and correspondence-based feature injection are incorporated into the model to promote subject consistency within each image and/or between images. Additionally, layout diversity is encouraged while maintaining subject consistency. The model achieves state-of-the-art performance on subject consistency and text alignment, without requiring any optimization and naturally extends to multi-subject scenarios.
What exactly they claim (claim 1)
1. A computer-implemented method, comprising: receiving a text description related to a subject with two or more prompts describing scenes for generation of two or more images depicting the subject, wherein the scenes comprise at least two different scenes and each prompt is associated with a different one of the two or more images; generating intermediate data associated with the two or more images by concurrently processing the text description with the two or more prompts by at least one layer of a neural network according to pre-trained weights without also processing an input image depicting the subject; computing cross-image consistency data specific to the subject using the intermediate data; and processing the cross-image consistency data by at least one remaining layer of the neural network according to the pre-trained weights to generate the two or more images.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- Training-free consistent text-to-image generation
- Patent number
- US12743819B2
- Application number
- 18650726
- Filed
- Apr 30, 2024
- Granted
- Sep 22, 2026
- Inventors
- 5
- Claims
- 22
Classified under
G06T 11/00 Physics & computingG06F 40/40 Physics & computingG06N 3/0475 Physics & computingG06T 5/60 Physics & computingG06T 7/33 Physics & computingG06T 2207/20081 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.