Nvidia patents smarter GPU memory swapping for AI tasks
Nvidia describes a method for managing the memory of graphics processing units (GPUs) while running artificial intelligence models. The system automatically moves memory space from one AI model to another when a new part of a task requires more room than what is currently free.
In the filing’s own words
Disclosed are apparatuses, systems, and methods for allocating graphics processing unit (GPU) memory resources for executing artificial intelligence (AI) models. The systems and methods can allocate a first portion of memory resources of the GPU to a first set of at least one AI model for a task execution. Upon a need for further data processing, identifying a second set of at least one AI model for continuation of the task execution. The systems and methods can determine that an unallocated portion of the memory resources of the GPU are insufficient for the second set of at least one AI model and can reallocate the first portion of the memory resources of the GPU to the second set of at least one AI model.
What exactly they claim (claim 1)
1. A method comprising: allocating, at a first point in time during a task execution, a first portion of memory resources of a graphics processing unit (GPU) to a first set of at least one computational model of a plurality of computational models, wherein the first set of at least one computational model requires an amount of memory resources less than the first portion of memory resources; identifying, at a second point in time during the task execution, a second set of at least one computational model of the plurality of computational models for continuation of the task execution; determining that a second portion of the memory resources of the GPU is insufficient for the second set of at least one computational model; and reallocating a third portion of memory resources of the GPU to the second set of at least one computational model.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- MULTI-MODEL MEMORY MANAGEMENT
- Publication number
- US20260277678A1
- Application number
- 19079748
- Filed
- Mar 14, 2025
- Published
- Sep 17, 2026
- Inventors
- 3
- Claims
- 20
Classified under
G06F 9/5016 Physics & computingG06F 9/5038 Physics & computingG06F 9/5044 Physics & computingMore from Nvidia in AI & software
- Nvidia patents AI coach that analyzes your gameplay to suggest strategies Oct 13, 2026
- Nvidia patents AI system that writes and voices character conversations Sep 10, 2026
- Nvidia patents system that uses AI to find bugs in game videos Sep 17, 2026
- Nvidia patents AI method to make digital characters move realistically Sep 8, 2026
- Nvidia patents AI system that builds image generators from text prompts Sep 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.