Imitation Learning from Demonstrations

Learn from expert demonstrations: train policy to mimic expert behavior using supervised learning on (state, action) pairs.

The problem, from first principles

RL exploration is expensive. If expert demonstrations available, learn directly by supervised learning. Behavioral cloning minimizes cross-entropy loss on expert actions.

From theory to code

Implement behavioral_cloning_loss(policy_logits, expert_actions) which:

• Takes predicted action logits and expert action labels
• Computes cross-entropy loss between policy and expert
• Averages over batch
• Returns scalar loss

Constraints

• Expert dataset can be small but must be representative
• Distribution mismatch: learned policy diverges from expert
• Requires expert demonstrations (offline)

Hints