Opening the paper…
Figure from the original question paper Assume that we have a large corpus of text. The vocabulary constructed from the text contains 10000 words. Of these, 100 words occurred only once in the entire corpus of text. The parameters of the embedding layer and the output layer of the model are shared. Suppose we create a batch of 256 samples (each sample is a sentence from the corpus). None of these samples contains any of the 100 rare words. Suppose we pre-train the model for one iteration using the batch of samples, then: Suppose we use a pre-trained model for text generation with the given prompt “I am going to”. Which of the following decoding strategies can be used such that the pre-trained model generates same text completion each time it is executed