grouped
grouped
¶
Grouped Query Attention (GQA) with RoPE, sliding window, and partial rotation support. Inherits from SelfAttention, overriding projection, RoPE, masking, and attention.
grouped
¶Grouped Query Attention (GQA) with RoPE, sliding window, and partial rotation support. Inherits from SelfAttention, overriding projection, RoPE, masking, and attention.