Microsoft patents method to check if LLM answers are based on real data
The filing describes a way to ask a large language model to prove its answers are correct. It forces the AI to provide a specific code snippet and data source for every fact it claims, allowing a computer to automatically verify if the math or logic holds up.
In the filing’s own words
A computer-implemented method of generating verification data for a query result provided by a large language model, LLM, includes generating a prompt for the large language model. The prompt contains a verification request for a query, the query including query text and input data from which the query result can be derived. The verification request includes instructions that cause the LLM to generate verification data that indicates a derivation of the query result from the input data. Another computer-implemented method includes receiving the verification data and processing the verification data to determine whether the query result was validly derived from the input data.
What exactly they claim (claim 1)
1. A computer-implemented method for obtaining verifiable query results from a large language model (LLM), the method comprising executing, by a computer processor, stored processor-executable instructions to perform operations comprising: automatically injecting, into a prompt including query text and input data, a verification request for a query result to be derived from the input data, wherein the verification request comprises instructions which, when processed by the LLM, cause the LLM to generate verification data that indicates: a value included in the query result, a corresponding source indicating a part of the input data from which the value was derived, and a corresponding evaluable expression comprising a code snippet in a computer-readable expression language for deriving the value from the source; and inputting the prompt into the LLM to obtain, from the LLM, the query result and verification data, wherein the verification data enables determining, by evaluating the evaluable expression using the code snippet and comparing a resulting expression output with the value included in the query result, whether the query result was validly derived from the input data.
Facts
- Company
- Microsoft (Microsoft Technology Licensing, LLC)
- Official title
- IDENTIFYING HALLUCINATIONS IN LARGE LANGUAGE MODEL OUTPUT
- Publication number
- US20260267900A1
- Application number
- 19662424
- Filed
- Apr 29, 2026
- Published
- Sep 10, 2026
- Inventors
- 3
- Claims
- 20
Classified under
G06F 16/345 Physics & computingG06F 16/3344 Physics & computingG06F 16/338 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
- Microsoft patents AI tool that writes and runs software tests from text Sep 1, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.