scratching
scratching
¶
ScratchSparseCrossAttention
¶
Bases: ForkingAttention
attention_weights(q: jax.Array, k: jax.Array, mask: Optional[jax.Array] = None, **kwargs: Any) -> jax.Array
¶
Shared projected-Q/K probabilities, with original-position causality.
ScratchHardCrossAttention
¶
Bases: ScratchSparseCrossAttention
Copy one causal source residual; differentiate the selector as softmax.