TrenTorch-Web

Linear Algebra

Vectors, matrices and tensors: shapes and basic operations Easy

Dot product and vector norms (L1, L2, L-infinity) Easy

Matrix multiplication from first principles Medium

Transpose, and its role in reshaping without copying data Easy

Matrix inverse, and when it does not exist Medium

Eigenvalues and eigenvectors of a small matrix Hard

Singular Value Decomposition (SVD) Hard

Positive-definite matrices, and why they matter for optimization Medium

Calculus

Derivatives from first principles: the limit definition, computed numerically Easy

Partial derivatives of a multivariate function Easy

Chain rule: composing two functions' derivatives by hand Medium

Jacobian: the matrix of all partial derivatives of a vector-valued function Hard

Hessian: second-order partial derivatives, and what its eigenvalues tell you Hard

Directional derivatives, and the gradient as steepest ascent Medium

Probability

Sampling from a random variable and estimating its distribution Easy

Expectation and variance from a sample Easy

Covariance and correlation between two variables Medium

Conditional probability from a joint distribution Medium

Bayes' theorem: updating a belief given evidence Medium

Likelihood vs. probability: the same formula, two different questions Medium

Maximum likelihood estimation for a simple distribution Hard

MAP estimation: maximum likelihood plus a prior Hard

Information Theory

Entropy of a discrete distribution Easy

Cross-entropy, and why it's the loss Classification already uses Medium

KL divergence between two distributions Medium

Mutual information between two variables Hard

Data Preprocessing

Detecting and counting missing values in a dataset Easy

Imputing missing numeric values with a column mean/median Easy

One-hot encoding a categorical column Medium

Feature scaling: standardization vs min-max normalization Medium

Exploratory Data Analysis

Detecting outliers with IQR and z-score Easy

Summarizing a feature's distribution: mean, median, skew Easy

Correlation matrix, and why correlation is not causation Medium

Data leakage: a feature that accidentally encodes the label Hard

Feature engineering: deriving a feature that makes the model's job easier Medium

Stratified sampling for an imbalanced dataset Medium

Statistical Inference

Confidence interval for a sample mean Medium

Bootstrap confidence intervals Medium

Hypothesis testing: a two-sample t-test from scratch Hard

A/B testing: is the difference between two groups real or noise Medium

Statistical significance and p-values, and what they do not mean Easy

Linear Regression

Hypothesis Function Easy

Mean Squared Error Loss Easy

Gradient of MSE with Respect to w and b Medium

One Gradient-Descent Update Easy

Full Linear Regression Training Loop Medium

Stretch: L2 Regularization (Ridge) Medium

Production Engineering: Mini-Batch Training Hard

Stretch: L1 Loss (MAE), contrasted against MSE Easy

Stretch: Huber Loss, quadratic near zero and linear far from it Medium

Generalization: train/val split and the generalization gap Medium

Classification (Logistic Regression)

Sigmoid Function Easy

Binary Cross-Entropy Loss Easy

Gradient of BCE Medium

Decision Boundary / Thresholding Easy

Full Training Loop Medium

Stretch: Softmax + Categorical Cross-Entropy Medium

Linear Discriminant Analysis (LDA) Hard

Stretch: Class Imbalance Handling Medium

Production Engineering: Fused, Numerically-Stable Loss Hard

LogSoftmax + NLLLoss: the two pieces CrossEntropyLoss actually fuses Medium

Production Engineering: detecting train/serve distribution shift Hard

Multiclass via One-vs-Rest, contrasted against Softmax Medium

Regularized Linear Models

Linear Regression: closed form (Normal Equation) Medium

Ridge Regression (L2) Medium

Lasso Regression (L1), contrasted against Ridge Medium

Elastic Net: combining L1 and L2 penalties Medium

Polynomial features: expanding inputs before a linear model Medium

Note: Generalized Linear Models, one framework behind Linear and Logistic Regression Easy

Support Vector Machines

Hinge loss Easy

Margin maximization intuition Easy

Linear SVM via gradient descent on hinge loss Medium

Stretch: kernel trick (conceptual) Hard

Decision Trees

Gini Impurity for a split Easy

Information Gain for a split Easy

Decision Tree best split (assemble a minimal tree) Hard

Pruning (pre-pruning, post-pruning) Medium

Regression trees: splitting on variance reduction instead of Gini Medium

Feature importance from a fitted tree Medium

Ensembles

Random Forest: majority vote aggregation Medium

Stretch: bagging concept Easy

Gradient Boosting: fit one tree to the negative gradient of the loss Medium

Full boosting loop: assemble a minimal booster Hard

Random Forest regression, and out-of-bag error estimation Medium

AdaBoost: reweighting misclassified samples each round Medium

Stretch: regularized boosting (shrinkage + L2 leaf penalty, XGBoost-style) Hard

Note: histogram-based boosting (LightGBM-style binning), why it is faster at scale Easy

Instance-Based and Probabilistic

KNN: distance and neighbor lookup Easy

Naive Bayes: Bernoulli log-likelihood Medium

Stretch: Gaussian Naive Bayes Medium

Nearest centroid classifier Easy

Note: Gaussian Processes, a distribution over functions instead of over parameters Medium