State Space Models and Linear Attention

Alternative architectures with fixed memory layers offering better scaling profiles for long context windows