The Bistable Memory Recurrent Unit

Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications

Announce Type: replace Abstract: Sequence learning is dominated by Transformers and parallelizable recurrent neural networks (RNNs) such as state-space models, yet learning long-term dependencies remains challenging, and state-of-the-art designs trade power consumption for performance. The Bistable Memory Recurrent Unit (BMRU) was introduced to enable hardware-software co-design of ultra-low power RNNs: quantized states with hysteresis provide persistent memory while mapping directly to...