Google patents VR tech that tracks your phone in a virtual room
Google has filed a patent for a system that places a physical mobile device into a virtual reality environment. It combines data from depth cameras, ultra-wideband signals, and motion sensors to calculate the phone's exact position and orientation. A machine learning model processes these inputs to render a digital version of the phone in the virtual space.
In the filing’s own words
Techniques of rendering an image of a mobile device in a virtual environment include a combination of depth images, ultra-wideband (UWB) signals, and inertial measurement unit (IMU) signals to track the position and orientation of a mobile device. The depth images may be generated using an RGB image acquired by the head-mounted display (HMD). The UWB and IMU signals may be generated by the HMD, although in some implementations UWB and HMD signals may also be generated by the mobile device. In some implementations, the depth images are sufficient to track the position and orientation of the mobile device. In some implementations, a combination of UWB and IMU signals are sufficient to track the position and orientation of the mobile device. In some implementations, a combination of the depth images and UWB and IMU signals are used to track the position and orientation of the mobile device.
What exactly they claim (claim 1)
1. A method, comprising: inputting, into a machine learning engine at a refresh rate of a virtual environment, ultra-wideband (UWB) data, and inertial measurement unit (IMU) data, the UWB data including samples of a UWB signal acquired at a first rate, the IMU data representing samples of a IMU signal acquired at a second rate, the first rate being different from the second rate; the machine learning engine being configured to output a position value and an orientation value indicative of a position and an orientation of a mobile device within the virtual environment; and rendering a virtual object representing the mobile device in the virtual environment according to the position and orientation.
Facts
- Company
- Google (GOOGLE LLC)
- Official title
- MULTI-MODAL LOCALIZATION OF STREAMING CONTENT PROVIDERS IN A VIRTUAL ENVIRONMENT
- Publication number
- US20260310282A1
- Application number
- 19481795
- Filed
- May 31, 2023
- Published
- Oct 8, 2026
- Inventors
- 2
- Claims
- 28
Classified under
G06F 3/011 Physics & computingG06F 3/016 Physics & computingG06T 19/003 Physics & computingMore from Google in AR, VR & displays
- Google patents video conference stage with built-in teleprompter Sep 15, 2026
- Google patents VR headset that checks your eyesight Oct 20, 2026
- Google patents glasses that share one lens for cameras and projectors Oct 20, 2026
- Google patents AR glasses with two separate screens in one lens Oct 1, 2026
- Google patents AR glasses that switch light paths to widen the view Oct 8, 2026
Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.