Nvidia patents 3D camera system to monitor driver eyes and hands
Nvidia describes a system that uses cameras and depth sensors to track a driver's eyes, head, and hands in three dimensions. It converts flat 2D video images into 3D coordinates using a personalized model of the occupant to precisely measure their movements. This data helps the car detect if the driver is drowsy or distracted.
In the filing’s own words
In various examples, systems and methods related to depth-assisted 3D detection are provided. The techniques described herein may be used to monitor a state of a vehicle occupant with high precision using depth-assisted 3D detection. Sensors may be used to capture image data and depth data that includes features of the vehicle occupant (e.g., eyes, head, hands, etc.). The 2D image coordinates from the image data may be converted to 3D coordinates in a global coordinate system using depth data and a 3D occupant model, which may be personalized to the vehicle occupant. Precise values may be determined based on the 3D coordinates in a global coordinate system, and the state of features of the vehicle occupant may be estimated based on these values. The values and/or state of features of the vehicle occupant may be used to detect drowsiness, distraction, and/or other conditions and provide safe vehicle operation.
What exactly they claim (claim 1)
1. One or more processors comprising processing circuitry to: obtain, using image data, two-dimensional (2D) image coordinates for one or more keypoints corresponding to at least one feature of a vehicle occupant; generate first three-dimensional (3D) coordinates in a coordinate system of an image sensor for the one or more keypoints based at least on the 2D image coordinates being fit to a 3D occupant model including the at least one feature; generate second 3D coordinates in a global coordinate system for the one or more keypoints based at least on the first 3D coordinates and depth data for the one or more keypoints in the global coordinate system to determine a state of the at least one feature; and output an indication of the state of the at least one feature.
Facts
- Company
- Nvidia (NVIDIA Corporation)
- Official title
- DEPTH-ASSISTED 3D DETECTION FOR VEHICLE OCCUPANT DETECTION AND MONITORING
- Publication number
- US20260311498A1
- Application number
- 19172305
- Filed
- Apr 7, 2025
- Published
- Oct 8, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06V 20/597 Physics & computingG06T 7/593 Physics & computingG06T 7/73 Physics & computingG06T 2207/10028 Physics & computingG06T 2207/30201 Physics & computingG06T 2207/30268 Physics & computingMore from Nvidia in Cars & transport
- Nvidia patents robotaxi system with AI dispatcher and virtual rails Sep 15, 2026
- Nvidia patents car drowsiness check that uses your body clock Oct 13, 2026
- Nvidia patents 3D camera system to measure passengers in vehicles Oct 13, 2026
- Nvidia patents cruise control that predicts driver's next move Oct 20, 2026
- Nvidia patents dual-brain system for self-driving cars Sep 17, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.