Mountain Car Environment
Intermediate
Mountain Car: push car up a one-dimensional mountain. State: [position, velocity]. Actions: [left, neutral, right]. Reward: -1 per step, +10 at goal. Hard exploration problem.
The problem, from first principles
Car can't accelerate enough to go straight up. Must oscillate back and forth to gain momentum. Sparse rewards force exploration and backward chaining.
From theory to code
Implement simulate_mountain_car(initial_state, actions, max_steps) which:
- Position in [-1.2, 0.6], velocity in [-0.07, 0.07]
- Actions: apply -1, 0, or +1 force
- Reward: -1 per step unless at goal (x >= 0.5)
- Returns trajectory with state and rewards
Constraints
- Gravity = 0.0025
- Friction = 0.01
- Bounds clipping at [-1.2, 0.6]
- Episode terminates at goal or max_steps
Hints
Hint 1: Oscillation needed
Can't climb mountain directly, must swing back and forthHint 2: Physics simple
v += (action - gravity - friction*v) * dtHint 3: Exploration critical
Sparse rewards make exploration essentialKeywords
rlmountain-carexplorationsparse-reward
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.