DDPG: The Ornstein-Uhlenbeck Exploration Step
Advanced
The problem, from first principles
DDPG explores by adding temporally correlated noise to its actions. The Ornstein-Uhlenbeck process pulls the noise back toward zero while adding fresh random kicks, so consecutive actions are smooth rather than jittery.
From theory to code
Implement ou_step(x, theta, sigma, dt, z), one Euler step of the Ornstein-Uhlenbeck process with a supplied normal draw.
Constraints
- Use
sqrt(dt)for the noise scale.
Hints
Hint 1
Add the drift theta * (0 - x) * dt and the noise sigma * sqrt(dt) * z to x.
Keywords
research-papersreinforcement-learningcontinuous-controlactor-critic
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.