GoogleGranted patentAI & softwareInterest 3/10

Google patents method to create training data for search models

This patent describes a way to automatically create practice problems for computer programs that search for information. It uses a large language model to guess what kind of search a text snippet belongs to and what question a user might ask about it, then uses those guesses to build a dataset for training other tools.

Why it mattersIt improves how search engines understand and organize information by automating the creation of training data, which is a routine but important step in building better software.

In the filing’s own words

An example method includes providing, to a sequence model (i) a plurality of few-shot prompts, wherein each prompt comprises a demonstration passage, a demonstration task, and a demonstration query, wherein the demonstration task describes a type of retrieval, and wherein the demonstration query is relevant to the demonstration task, and (ii) a plurality of passages sampled from a corpus of passages. The method also includes receiving, from the sequence model and for the plurality of passages and based on the plurality of few-shot prompts, a respective plurality of predicted task-query pairs, the sequence model having been prompted to predict a task based on an input passage, and predict an output query relevant to the predicted task. The method further includes generating a synthetic training dataset comprising the plurality of passages and the respective plurality of predicted task-query pairs. The method also includes providing the synthetic training dataset.

What exactly they claim (claim 1)

1. A computer-implemented method for generating a synthetic training dataset to fine-tune an embedding model, comprising: providing, by a processor, to a sequence model (i) a plurality of few-shot prompts, wherein each prompt comprises a demonstration passage, a demonstration task, and a demonstration query, wherein the demonstration task describes a type of retrieval, and wherein the sequence model has been trained to align the demonstration query with the demonstration task, and (ii) a plurality of passages sampled from a corpus of passages; receiving, by the processor, from the sequence model and for the plurality of passages and based on the plurality of few-shot prompts, a respective plurality of predicted task-query pairs, by automatically prompting the sequence model to select a retrieval task type from a plurality of retrieval task types based on an input passage to predict a task based on the input passage, and predict an output query for the predicted task; generating, by the processor, a synthetic training dataset comprising the plurality of passages and the respective plurality of predicted task-query pairs for use in fine-tuning the embedding model, and wherein the generating of the synthetic training dataset further comprises determining a global positive passage for a generated query by selecting a target passage from the corpus of passages that differs from the input passage used to generate the query and has a higher relevance score than the input passage based on a fusion ensemble of a query likelihood scoring and a relevance classification scoring, wherein the query likelihood scoring indicates a likelihood of a given generated query for a given passage, and wherein the relevance classification scoring indicates a likelihood of a given relevance label for the given generated query and the given passage; and providing, by the processor, the synthetic training dataset to the embedding model.

Facts

Company
Google (Google LLC)
Official title
Systems and methods for generating instruction fine-tuning dataset for a general purpose embedding model
Patent number
US12801645B2
Application number
18788178
Filed
Jul 30, 2024
Granted
Nov 17, 2026
Inventors
25
Claims
21

Classified under

G06F 16/3344 Physics & computingG06N 3/08 Physics & computing

More from Google in AI & software

Summaries are simplified from the filing’s own abstract and first claim. A patent filing is not a product announcement.
Patent data and drawings: U.S. Patent and Trademark Office (USPTO). Nebula is not affiliated with or endorsed by the USPTO.