Contrast note: why no positional encoding here, unlike Part 2

The problem, from first principles

A sequence Transformer (the kind Part 2's sequence-modeling questions build toward) has a real problem plain attention doesn't solve on its own: scaled_dot_product_attention (01-row-wise-attention) treats its inputs as an unordered set — nothing in query @ key^T depends on which position a token came from, so "the cat sat" and "sat cat the" would produce identical attention computations unless something explicitly tells the model about word order. A data table has no equivalent problem: row 5 of a dataset isn't "after" row 4 in any meaningful sense, shuffling every row of a training set (and its labels along with it) describes the exact same dataset.

From theory to code

Implement is_row_permutation_equivariant(table, row_weights, col_weights, permutation): checks that shuffling input rows and shuffling output rows are interchangeable. Reuse 03-two-way-attention-block's two_way_attention_block.

Constraints

• Runs two_way_attention_block on table and on table[permutation] (the same weights both times).
• Returns True iff the permuted-input output equals the original output with the same permutation applied to its rows.
• Returns a plain Python bool, not a NumPy boolean.

Hints

Open one at a time. Each gives away a little more than the last.