Microsoft patents method to link words across languages for better translation

1. A method for representation learning of cross-language texts, comprising: obtaining a source language text and a target language text; generating an initial joint representation of the source language text and the target language text; identifying relations among a plurality of words in the source language text and the target language text; constructing a graph corresponding to the source language text and the target language text based on the relations, wherein the graph comprises alignment edges between source language word nodes and target language word nodes and dependency edges among word nodes within a same language; generating a joint representation of the source language text and the target language text based on the initial joint representation and the relations by updating the initial joint representation based on the graph, wherein generating the joint representation comprises obtaining attention information based on the graph, the attention information encoding differentiated contributions from alignment neighbor nodes connected by the alignment edges and dependency neighbor nodes connected by the dependency edges, wherein a contribution from an alignment neighbor node is determined based on a semantic difference between the alignment neighbor node and a current node, and a contribution from a dependency neighbor node is determined based on an importance of the dependency neighbor node relative to the current node; projecting the joint representation to at least a target language representation corresponding to the target language text; and pre-training a representation obtaining model by: masking one or more nodes in a node set of a source language node set corresponding to a source language text sample and a target language node set corresponding to a target language text sample in the graph constructed based on the relations; for each masked node, recovering the node with at least a representation of at least one neighbor node of the node; computing a recovery loss based on the recovery of the masked node; and updating parameters of the representation obtaining model by minimizing the recovery loss.