Gated Recurrent Unit (GRU)公式简介

update gate $z_t$: defines how much of the previous memory to keep around.　　

\[z_t = \sigma ( W^z x_t+ U^z h_{t-1}  )\]

reset gate $r_t$: determines how to combine the new input with the previous memory.

\[r_t = \sigma(W^r x_t  + U^r h_{t-1}  )\]

Cell value $\tilde h_t $: \[\tilde h_t  = \tanh (W^h x_t  + U^h(h_{t-1} \odot r_t) )\]

hidden value $h_t$: \[h_t = (1-z_t)\odot \tilde h_t  + z_t \odot h_{t-1}\]