Flash attention ?

by edmond - opened May 21

May 21

It says flash attention 2 is not available for this model.
But I was thinking maybe I could only use FA2 for the gemma part, do you think this is possible ?

merve

Google org May 27

@edmond you can check out TGI's flash PaliGemma implementation here it's implemented for vision head but doesn't have as much effect as flash in decoder

edmond changed discussion status to closed Jun 4

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment