BPE Pair Counting
Beginner
Count adjacent symbol pairs across a corpus and return the most frequent one, as in the first step of every Byte-Pair-Encoding merge. corpus is a list of sequences (lists or strings of symbols). Ties go to the pair that appeared first.
Implement solve(corpus).
Returns. Return a tuple ((left, right), count). The corpus must contain at least one adjacent pair.
Examples
Example 1
Input:
solve([['l', 'o', 'w'], ['l', 'o', 'w', 'e', 'r']])
Output:
(('l', 'o'), 2)
Example 2
Input:
solve([['a', 'b', 'a', 'b']])
Output:
(('a', 'b'), 2)
Keywords
problemsettransformer-llmtokenization
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.