TRPO: The KL Divergence Between Gaussians
Advanced
The problem, from first principles
TRPO bounds how far the policy moves using a KL divergence between the old and new action distributions. For Gaussian policies, which continuous control uses, that divergence has a closed form in the means and standard deviations.
From theory to code
Implement gaussian_kl(mu1, s1, mu2, s2) for the KL from the first Gaussian to the second.
Constraints
- Standard deviations are positive.
Hints
Hint 1
Use log(s2/s1) + (s1^2 + (mu1 - mu2)^2) / (2 s2^2) - 1/2.
Keywords
research-papersreinforcement-learningpolicy-gradienttrust-region
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.