Technology
Sparse Attention
Sparse attention means the model focuses only on a small, important subset of tokens instead of attending to all tokens, reducing computation and memory usage …
Read More
Technology
Technology
Hmm do you know about dense and sparse neural network
A dense neural network connects every neuron to the next layer, using more parameters and computation, while a sparse neural network has fewer connections, making …
Read More