Waymo patents system to track objects from far away using camera and lidar
This patent describes a method for self-driving cars to track objects at varying distances by switching between different sensors. It uses a camera to spot objects far away, then hands off tracking to lidar when the objects get closer, ensuring continuous monitoring.
In the filing’s own words
The described aspects and implementations enable efficient and seamless tracking of objects in vehicle environments using different sensing modalities across a wide range of distances. A perception system of a vehicle deploys an object tracking pipeline with a plurality of models that include a camera model trained to perform, using camera images, object tracking at distances exceeding a lidar sensing range, a lidar model trained to perform, using lidar images, object tracking at distances within the lidar sensing range, and a camera-lidar model trained to transfer, using the camera images and the lidar images, object tracking from the camera model to the lidar model.
What exactly they claim (claim 1)
1. A system comprising: a sensing system of a vehicle, the sensing system configured to acquire: one or more camera images of an outside environment at a first time, and one or more lidar images of the outside environment at a second time, and a processing system of the vehicle, the processing system configured to: provide the one or more camera images and positional data from a plurality of object tracks as input to a first set of one or more neural networks (NNs), each of the plurality of object tracks comprising positional data for a respective object of a plurality of objects in the outside environment; update the plurality of object tracks based on an output of the first set of one or more NNs; responsive to a distance from the vehicle to at least one object tracked by the plurality of object tracks being, at the second time, below a threshold distance. provide the one or more lidar images, the one or more camera images, and the positional data from the plurality of object tracks as input to a second set of one or more NNs; further update the plurality of object tracks based on an output of the second set of one or more NNs; and cause a driving path of a vehicle to be modified in view of the plurality of object tracks.
Facts
- Company
- Google (Waymo LLC)
- Official title
- Object tracking across a wide range of distances for driving applications
- Patent number
- US12755395B2
- Application number
- 18221577
- Filed
- Jul 13, 2023
- Granted
- Oct 6, 2026
- Inventors
- 5
- Claims
- 20
Classified under
G06T 7/20 Physics & computingG01S 17/89 Physics & computingG05D 1/021 Physics & computingG06V 10/25 Physics & computingG06V 10/44 Physics & computingG06V 20/56 Physics & computingMore from Google in Cars & transport
- Waymo patents system to detect cyclists losing control before they fall Sep 8, 2026
- Waymo patents self-driving cars that hear sirens and pull over Sep 8, 2026
- Waymo patents AI that moves self-driving cars out of traffic jams Sep 29, 2026
- Waymo patents AI that predicts driving scenes from sensor data Oct 1, 2026
- Waymo patents method for self-driving cars to predict when they can see hidden hazards Sep 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.