Microsoft patents automated tests to check software safety tools
Microsoft describes a system that automatically creates test programs to verify if static analysis tools correctly identify safe and unsafe code operations. If the analysis tool marks a program as safe but the program actually performs an unsafe action when run, the system flags the error and identifies the specific instruction causing the issue.
In the filing’s own words
The techniques presented herein provide a system for validating a static analysis tool by utilizing automated generation of rigorous test programs that evaluate various aspects of the static analysis tool. In various examples, the test program comprises a sequence of instructions that perform operations and can be safe or unsafe operations. Accordingly, the static analysis tool analyzes the instructions to identify the operations and classifies the test program as either a safe program or an unsafe program. If the static analysis tool classifies the test program as safe, the test program is then executed by a concrete implementation (e.g., virtual machine) to observe the actual behavior. If the test program attempts an unsafe operations, the validation system generates an alert identifying the instruction that attempted the unsafe operation as well as indicating the presence of an issue (e.g., a bug) in the static analysis tool.
What exactly they claim (claim 1)
1. A method for generating and evaluating a test program for validating a static analysis tool against a concrete implementation executing the test program, the method comprising: generating, by an automated tool, a test program comprising a sequence of instructions wherein: the test program operates on an aspect of the static analysis tool; and the sequence of instructions performs an operation that is one of a safe operation or an unsafe operation; classifying, by the static analysis tool, the test program as a safe program based on the operation performed by the sequence of instructions being a safe operation; in response to classifying the test program as a safe program, executing the test program using the concrete implementation; identifying an unsafe operation performed by the sequence of instructions during execution by the concrete implementation, wherein identifying the unsafe operation indicates an issue in the aspect of static analysis tool being operated on by the test program; and generating an alert identifying an instruction of the sequence of instructions that produced the unsafe operation.
Facts
- Company
- Microsoft (MICROSOFT TECHNOLOGY LICENSING, LLC)
- Official title
- AUTOMATED TEST PROGRAM GENERATION FOR VALIDATING STATIC ANALYSIS TOOLS
- Publication number
- US20260300469A1
- Application number
- 19089412
- Filed
- Mar 25, 2025
- Published
- Oct 1, 2026
- Inventors
- 2
- Claims
- 20
Classified under
G06F 21/52 Physics & computingG06F 2221/033 Physics & computingMore from Microsoft in AI & software
- Microsoft patents method to check AI answers against verified facts Oct 20, 2026
- Microsoft patents AI that presents PowerPoint slides for you Sep 3, 2026
- Microsoft patents method to check if LLM answers are based on real data Sep 10, 2026
- Microsoft patents system to blend multiple AI agents into one chat Oct 1, 2026
- Microsoft patents system that answers questions using your screen history Oct 15, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.