DQN: Clipping the TD Error
Intermediate
The problem, from first principles
A large TD error produces a huge gradient, which can destabilize training. DQN clips the error to a fixed range so that a single surprising transition cannot dominate an update.
From theory to code
Implement clipped_td_error(td, c), clipping each error to [-c, c].
Constraints
- Works on scalars and arrays.
Hints
Hint 1
Use np.clip with bounds -c and c.
Keywords
research-papersreinforcement-learningdeep-q-learning
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.