Microsoft patents system that writes detailed image descriptions

1. A system comprising: a plurality of processors; and a computer storage medium storing instructions that, when executed across the plurality of processors, cause the system to: receive, by a first vision language model, a request for a story caption of an image, the request comprising a caption focus for the image, the caption focus comprising instructions for a format of the story caption, the format being based on a platform the requested story caption is presented in; generate, by the first vision language model, for the image, visual information comprising an image tag and a visual clue constructed from the image tag; based on at least the caption focus, generate, by a generative language model, from the visual information, a plurality of image story caption candidates, wherein each of the plurality of image story caption candidates comprise a coherent paragraph of natural-language text describing the image; for each image story caption candidate of the plurality of image story caption candidates, perform, by a second vision language model, the following: parse the image story caption candidate into a plurality of sentences; for each sentence of the plurality of sentences, measure a similarity between the image and each sentence of the plurality of sentences; remove any sentence from the plurality of sentences with a similarity score below a similarity threshold; and update the image storage caption candidate based on the removing, wherein the updated image story caption comprises an updated coherent paragraph of natural-language text describing the image without any of the sentences having a similarity score below the similarity threshold; and based on at least an evaluation of the image and the plurality of image story caption candidates by the second vision language model, select a caption from among the plurality of image story caption candidates.