Every single model in this curriculum before this track needed a training loop specific to this dataset: 05-training-loop's gradient descent, 04-full-boosting-loop's sequential tree fitting, 05-em-algorithm's E/M iterations — all of them adjust parameters to fit the data they're given, one dataset at a time. TabPFN's actual, genuinely different idea is in-context learning: train one attention model's weights once, offline, on a huge variety of synthetic datasets, then freeze those weights completely. For any new dataset, no further training happens at all — the training examples themselves become part of the input, laid out as extra rows in the table, and one forward pass through the frozen network produces predictions for new rows directly.
Implement build_incontext_table(train_features, train_targets, query_features) and in_context_predict(train_features, train_targets, query_features, row_weights, col_weights). Reuse 03-two-way-attention-block's two_way_attention_block.
build_incontext_table returns shape (n_train + n_query, n_features + 1, 1): training rows carry their real target in the last column; query rows get exactly 0 there (masked, not a guess).in_context_predict returns shape (n_query,): the query rows' target-column values after exactly one forward pass through two_way_attention_block.row_weights/col_weights are fixed, given inputs — never updated inside in_context_predict, no training loop of any kind.Open one at a time. Each gives away a little more than the last.
The table's last column holds the target — real values for training rows, 0 for query rows, since NumPy arrays initialize to 0 already if you build the array with np.zeros and only fill in what's known.
in_context_predict is exactly three steps: build the table, run two_way_attention_block once, then slice output[n_train:, -1, 0] — the query rows' post-attention target-column entries.
Click "Run Tests" to test your implementation