OpenAI patents API for training image-and-text models
OpenAI describes a system that lets developers send images and text through a standard software interface to customize a large language model. The process uses a technique called low-rank adaptation, which updates only a small part of the model's internal settings while keeping the rest unchanged to save resources.
In the filing’s own words
The present technology pertains to a fine-tuned generative response engine that accepts multi-modal inputs include text and one or more images. A training set for fine-tuning the engine/model (e.g., a machine learning model) can be received via an application programming interface (API) call to an API. The training set includes inputs having input images associated with input text. Further, for supervised learning the training sets can include output text (e.g., completions that are the desired generated responses). The API call can indicate a base model (or previously fine-tuned model) to be fine-tuned. The API can cause a fine-tuned model to be trained based on the base model and the training set and can return a name for the fine-tuned model. To use the fine-tuned model to generate a response to a prompt, a subsequent API call includes the name and the prompt.
What exactly they claim (claim 1)
1. A method for using machine learning (ML) and models created thereby, comprising: receiving, via an application programming interface (API) call, a training set for fine-tuning an ML model, the training set comprising a plurality of images associated with respective text strings; receiving, via the API call, information indicating a first model that is to be fine-tuned using the training set; causing a fine-tuned ML model to be trained based on the first model and the training set by performing supervised fine-tuning in which, for at least one layer of the first model having a pretrained weight matrix, pretrained weighting coefficients of the pretrained weight matrix are maintained constant while weighting coefficients of a first trainable matrix and a second trainable matrix of a low-rank adaptation layer are updated; and returning, in response to the API call, a first identifier that identifies the fine-tuned ML model, which has been fine-tuned using the training set, wherein the first identifier is used to call the fine-tuned ML model, and the first model is a base model or a previously fine-tuned model.
Facts
- Company
- OpenAI (OpenAI OpCo, LLC.)
- Official title
- VISION FINE-TUNING FOR MACHINE LEARNING MODEL AND APPLICATION PROGRAMMING INTERFACE (API)
- Publication number
- US20260311048A1
- Application number
- 19240232
- Filed
- Jun 17, 2025
- Published
- Oct 8, 2026
- Inventors
- 9
- Claims
- 26
Classified under
G06N 20/00 Physics & computingG06T 3/40 Physics & computingG06T 7/11 Physics & computingMore from OpenAI in AI & software
- OpenAI patents voice system that detects when you stop talking Sep 22, 2026
- OpenAI patents system to catch AI models cheating on tasks Sep 10, 2026
- OpenAI patents method to check AI thinking for cheating Sep 10, 2026
- OpenAI patents system that turns photos into new AI-generated images Sep 10, 2026
- OpenAI patents AI that answers questions using only your company's secure files Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.