Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus

Part of

Relevance-Gated Hyperdimensional Memory Streaming