SonyGranted patentCameras & imagingInterest 6/10

Sony patents camera system that checks if its depth data is accurate

Patent drawing
Patent drawing from the official filing.

Sony describes a camera control system that uses two separate software models to process images. The first model estimates the distance to objects in the scene, while the second model checks how reliable those distance estimates are. If the depth data is trusted, the camera uses it to adjust focus, exposure, and white balance; if not, it falls back to traditional phase-difference detection.

Why it mattersThis approach could help cameras make sharper, better-exposed photos by ensuring they only rely on depth data when it is actually accurate.

In the filing’s own words

The present technology relates to a control device, a control method, an information processing device, a generation method, and a program capable of determining whether or not Depth information of an estimation result is reliable information. A control device according to one aspect of the present technology generates Depth information indicating a distance to each position of a subject appearing in a captured image on the basis of output of a first estimation model when the captured image is input, and generates reliability information indicating reliability of the Depth information on the basis of output of a second estimation model when intermediate data generated in the first estimation model is input at the time of estimating the Depth information. The present technology can be applied to a device including a camera.

What exactly they claim (claim 1)

1. A control device comprising: circuitry configured to generate Depth information indicating a distance to each position of a subject appearing in a captured image on a basis of output of a first estimation model when the captured image is input directly to the first estimation model; generate reliability information indicating reliability of the Depth information on a basis of output of a second estimation model when intermediate data generated in the first estimation model is input to the second estimation model at the time of estimation of the Depth information; detect a phase difference for the captured image; determine whether the reliability of the Depth information is higher than a predetermined threshold; determine a focus target based on whether the reliability of the Depth information is higher than the predetermined threshold or not; and perform focus control on the focus target based on whether the reliability of the Depth information is higher than the predetermined threshold or not, wherein the reliability information output by the second estimation model includes a reliability map having map information with a plurality of pixels associated with the captured image, wherein the circuitry determines the focus target and performs the focus control based on the Depth information in a case where the reliability of the Depth information is higher than the predetermined threshold and determines the focus target and performs the focus control based on the phase difference in a case where the reliability of the Depth information is lower than the predetermined threshold and without using the output of the first estimation model, wherein, in addition to the focus control, one or more of exposure control and/or white balance control are performed by the circuitry based on the Depth information in the case where the reliability of the Depth information is higher than the predetermined threshold, wherein the circuitry determines the focus target as an object appearing in a region of one or more of the plurality of pixels having reliability higher than the predetermined threshold in the case where the reliability of the Depth information is higher than the predetermined threshold, and wherein the circuitry controls imaging by using the Depth information having the reliability higher than the predetermined threshold, the Depth information being generated before the Depth information having the reliability lower than the predetermined threshold.

Facts

Company
Sony (Sony Group Corporation)
Official title
Control device, control method, information processing device, generation method, and program
Patent number
US12804261B2
Application number
18710629
Filed
Nov 25, 2022
Granted
Nov 17, 2026
Inventors
2
Claims
14

Classified under

H04N 23/672 Electricity & electronicsG06T 7/50 Physics & computingG06T 2207/10028 Physics & computingG06T 2207/20081 Physics & computingG06T 2207/20084 Physics & computing

More from Sony in Cameras & imaging

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.