OpenAI patents chip design that mixes number formats for faster math
OpenAI has been granted a patent for a hardware accelerator that performs matrix-vector operations, a core task in AI training and inference. The design uses a compute-in-memory structure where a mode decoding unit adjusts how the chip handles different floating-point number formats for data and weights. This allows the hardware to calculate products using specific adders and multiplexers without constantly converting between numeric types.
In the filing’s own words
Systems, apparatuses, and methods are disclosed for improved matrix-vector operations in accelerators that may be useful or heavy AI training and inference workloads. The disclosed technology provides arrangements that permit more efficient computation by, for example, performing calculations without repeated conversion between numeric domains. In some implementations, a compute-in-memory (CIM) macro is configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values, and a mode decoding unit is configured to provide a mode of the VMM operation according to a first floating point format of the activation values and a second floating point format of the weight values. A column cell of the CIM macro can be configured to output a product between an activation value and a weight value using a half adder and multiple multiplexers that provide selections to a full adder based on control signals.
What exactly they claim (claim 1)
1. A hardware accelerator comprising: a compute-in-memory (CIM) macro configured to perform a vector matrix multiplication (VMM) operation between a vector of activation values and a matrix of weight values; and a mode decoding unit configured to provide an arithmetic mode of the VMM operation according to a first floating point format of the activation values and a second floating point format of the weight values, the first floating point format and the second floating point format being flexible, wherein a column cell of the CIM macro is configured to output a primitive product between an activation value and a weight value and comprises: a half adder configured to add an exponent bit of the activation value and an exponent bit of the weight value and output a carry-out bit and a sum bit; a first multiplexer configured to select zero or a prior stage carry-out bit from a prior stage half adder and output a first selection to a full adder; a second multiplexer configured to select zero or the sum bit and output a second selection to the full adder; and a third multiplexer configured to select zero or a carry-in bit from a prior stage full adder and output a third selection to the full adder, wherein the first multiplexer, the second multiplexer and the third multiplexer perform respective selections based on control signals according to the arithmetic mode.
Facts
- Company
- OpenAI (OpenAI Opco, LLC)
- Official title
- Floating point multiplications
- Patent number
- US12754971B2
- Application number
- 19397827
- Filed
- Nov 21, 2025
- Granted
- Oct 6, 2026
- Inventors
- 6
- Claims
- 19
Classified under
G06F 17/16 Physics & computingMore from OpenAI in Chips & hardware
- OpenAI patents chip design that learns while processing data Sep 29, 2026
- OpenAI patents memory chip design to update AI weights while calculating Sep 17, 2026
- OpenAI patents analog circuit that learns by balancing electrical energy Sep 29, 2026
- OpenAI patents chip design that switches between math types to save power Oct 13, 2026
- OpenAI patents chip design that flips math operations without moving data Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.