grouped

grouped

Grouped Query Attention (GQA) with RoPE, sliding window, and partial rotation support. Inherits from SelfAttention, overriding projection, RoPE, masking, and attention.

GroupedSelfAttention

Bases: SelfAttention