GooglePatent applicationAI & softwareInterest 3/10

Google patents transformer model for personalized item recommendations

Patent drawing
Patent drawing from the official filing.

Google describes a system that analyzes how users interact with previously suggested items to create new recommendations. It uses a specialized transformer model that treats different types of user data, such as categories and numbers, as distinct features and calculates how they influence each other. The system then combines these weighted features to determine which items to suggest next.

Why it mattersThis filing details a specific method for improving recommendation engines, which could lead to more accurate suggestions in services like YouTube or Google Play.

In the filing’s own words

A computing system receives information indicative of interactions between a user and previously recommended items. The computing system transforms the information into a feature embeddings list. The computing system applies a heterogeneous self-attention model to the feature embeddings list. The heterogeneous self-attention model learns individual query, key, and value projection matrices for each feature embedding. The computing system generates a unique transformation matrix for each pair of feature embeddings. The computing system concatenates the individual query, key, and value projection matrices into a single concatenated matrix and generates a full projection matrix by at least applying a linear transformation to the single concatenated matrix. The computing system applies a heterogeneous interaction transformer model to the feature embeddings list to generate a weighted feature embeddings list including weighted task embeddings. The computing system generates a set of recommended items based on applying a projection function to the weighted task embeddings.

What exactly they claim (claim 1)

1. A method comprising: receiving, by a computing system, information indicative of interactions between a user and a plurality of previously recommended items, wherein the information includes one or more of encoded categorical features of one or more user-item pairs, encoded dense scalar features of one or more user-item pairs, and encoded task embeddings; transforming, by the computing system, one or more of the encoded categorical features, the encoded dense scalar features, and the encoded task embeddings into a feature embeddings list, wherein the feature embeddings list includes pairs of feature embeddings; applying, by the computing system, a heterogeneous self-attention model comprising a heterogeneous multi-head self-attention layer to the feature embeddings list, wherein the heterogeneous self-attention model learns individual query, key, and value projection matrices for each individual feature embedding in each pair of feature embeddings; generating, by the computing system, a unique transformation matrix for each pair of feature embeddings, wherein the unique transformation matrix transforms each individual feature embedding into a shared space; concatenating, by the computing system, the individual query, key, and value projection matrices for each individual feature embedding into a single concatenated matrix; generating, by the computing system, a full projection matrix by at least applying a linear transformation to the single concatenated matrix; applying, by the computing system, and using the full projection matrix, a heterogeneous interaction transformer model to the feature embeddings list and the encoded task embeddings, wherein the feature embeddings list is transformed into a plurality of composite features, wherein the heterogeneous interaction transformer model computes attention scores between the encoded task embeddings and the plurality of composite features, and wherein the attention scores are used to generate a weighted feature embeddings list including weighted categorical features, weighted dense scalar features, and weighted task embeddings; generating, by the computing system, a set of recommended items based on applying a projection function to the weighted task embeddings, wherein the projection function transforms the weighted task embeddings into the set of recommended items; and storing, by the computing system, in a memory, data indicative of the set of recommended items.

Facts

Company
Google (Google LLC)
Official title
HETEROGENEOUS FEATURE INTERACTIONS WITH TRANSFORMERS
Publication number
US20260359308A1
Application number
19487900
Filed
May 21, 2024
Published
Nov 12, 2026
Inventors
8
Claims
22

Classified under

G06F 16/24578 Physics & computingG06F 16/2237 Physics & computing

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.