Masked LM Labels

Create the labels for masked-language-model training. mask marks the positions that were selected for prediction. The labels equal the original token id at selected positions and ignore_index (default $-100-100$) everywhere else, so the loss only counts the masked positions. The token ids themselves are returned unchanged.

Implement solve(ids, mask, ignore_index=-100).

Returns. Return a tuple (ids, labels) of integer NumPy arrays; ids is a copy of the input.

Examples

Example 1

Input:

solve([5, 6, 7, 8], [False, True, False, True])

Output:

([5, 6, 7, 8], [-100, 6, -100, 8])

Example 2

Input:

solve([5, 6], [False, False], ignore_index=-1)

Output:

([5, 6], [-1, -1])