GPT-3: Scoring a Continuation
Intermediate
The problem, from first principles
GPT-3 ranks candidate answers by how likely the model finds each one as a continuation of the prompt. The score is the sum of the log-probabilities of the continuation tokens, given everything before them.
From theory to code
Implement continuation_logprob(log_probs, tokens, start), which sums the log-probability of each token from start onward.
Constraints
- Only continuation tokens are scored; the prompt contributes nothing.
Hints
Hint 1
Loop from start to the end, pick each token's log-probability from its row, and sum.
Keywords
research-paperstransformersllmllmprompting
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.