Microsoft patents method to shrink word data for speech systems
Microsoft describes a system that compresses the mathematical representations of words used in speech recognition software. It uses a specific type of neural network to shrink these data points into smaller binary codes, then expands them back to their original form to maintain accuracy. This process allows the system to handle spoken language inputs more efficiently without losing the meaning of the words.
In the filing’s own words
Described herein are systems and methods that provide a natural language processing system (NLPS) that employs compressed word embeddings. An auto-encoder that includes encoder circuitry and decoder circuitry can be used to produce the compressed word embeddings. The decoder circuitry is trained to decompress the word embeddings with reduced or minimal differences between the original uncompressed word embeddings and the corresponding decompressed word embeddings. One or more parameters of the trained decoder circuitry are transferred to the NLPS, where the NLPS is then trained using the compressed word embeddings to improve the correctness of the responses or actions determined by the NLPS.
What exactly they claim (claim 1)
1. A computing system, comprising: at least one processor; and a memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations comprising: receiving a spoken language input via a speech-to-text application; obtaining original uncompressed word embeddings for the spoken language input, the original uncompressed word embeddings including a vector of real numbers; generating input text compressed word embeddings based on the uncompressed word embeddings using an encoder of a trained auto-encoder, the input text compressed word embeddings including a vector of binary numbers; processing the input text compressed word embeddings with a neural network to generate output compressed word embeddings including a vector of binary numbers; decompressing the output compressed word embeddings to produce one or more decompressed word embeddings using a decoder of the trained auto-encoder; and outputting a natural language output based on the one or more decompressed word embeddings, using a text-to-speech application.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- Compression of word embeddings for natural language processing systems
- Patent number
- US12748932B2
- Application number
- 18057018
- Filed
- Nov 18, 2022
- Granted
- Sep 29, 2026
- Inventors
- 3
- Claims
- 19
Classified under
G06F 40/44 Physics & computingG06F 40/126 Physics & computingG06F 40/30 Physics & computingG06N 3/044 Physics & computingG06N 3/045 Physics & computingG06N 3/0455 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.