01-row-wise-attention and 02-column-wise-attention are two separate halves of the same idea — one axis alone can't capture everything a table has to say about itself. TabPFN's actual architecture alternates these two directions, letting information mix across an entire row, then across an entire column, then row again, and so on, building up genuinely table-wide context: after enough layers, information from any cell can eventually influence any other cell.
Implement two_way_attention_block(table, row_weights, col_weights): row_wise_attention, then column_wise_attention on its output, each with a residual connection. Reuse 01-row-wise-attention's row_wise_attention and 02-column-wise-attention's column_wise_attention. Each attention step adds its own input back to its output (a residual connection), not just the raw attention result.
table: shape (n_rows, n_cols, d_model). Returns the same shape.row_weights/col_weights: each a (w_query, w_key, w_value) tuple.table.Open one at a time. Each gives away a little more than the last.
row_output = table + row_wise_attention(table, *row_weights) — the residual add is not optional, and it's what makes the all-zero-weights sanity check hold.
The column-wise step's input is row_output, not table — building on what the row-wise step already produced, not starting over from scratch.
Click "Run Tests" to test your implementation