A3C: The n-Step Return
Intermediate
The problem, from first principles
A3C (Mnih et al., 2016) runs many workers in parallel, and each collects a short rollout of n steps before updating. The learning target for that rollout is an n-step return: the rewards actually seen, plus a bootstrapped value for the state at the end.
From theory to code
Implement n_step_return(rewards, bootstrap, gamma), returning the discounted sum of the rewards plus the discounted bootstrap value.
Constraints
- Use the bootstrap value as the starting point of the backward pass.
Hints
Hint 1
Start from bootstrap, and for each reward from last to first set G = r + gamma * G.
Keywords
research-papersreinforcement-learningactor-criticasynchronous
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.