THE GATED RECURRENT UPDATE

Difficulty: Medium Tags: Sequence Models, GRU, Gating, Nonlinearities

Story

A sequence-model implementation produces hidden states that drift because the update gate is applied in the wrong direction. For a simplified scalar GRU cell, compute the candidate state and final hidden state exactly as specified.

The Math

For scalar inputs x, h, compute:

z = sigmoid(wz*x + uz*h + bz)
r = sigmoid(wr*x + ur*h + br)
h_candidate = tanh(wh*x + uh*(r*h) + bh)
h_prime = (1-z)*h + z*h_candidate

The update convention is part of the problem: use z to weight the candidate state.

Input Format

x, h, wz, uz, bz, wr, ur, br, wh, uh, bh.

Output Format

(z, r, h_candidate, h_prime) (6 decimal places).

Constraints

• All inputs have absolute value at most 10.
• Time limit: 1.0 second.

Example

Input

x=1, h=0.5
wz=1, uz=0, bz=0
wr=0, ur=1, br=0
wh=1, uh=1, bh=0

Output

(0.731059, 0.622459, 0.864586, 0.766534)

Explanation: The update gate is sigmoid(1), the reset gate is sigmoid(0.5), and the candidate is tanh(1 + 0.5*sigmoid(0.5)). Blending with the old state gives approximately 0.766534.