Commit History

gradient checkpointing for multi-query attention
07e555c
unverified

Alex Birch commited on

add support for AutoModelForCausalLM#from_pretrained()'s device_map='auto'. support gradient checkpointing, probably. add lots of type hints so I could understand what's going on. multiline long method signatures/calls (for easier comparison between checkpointed/non-checkpointed variants, and because these lines got even longer when I added type hints). make MPTForCausalLM#forward accept additional kwargs, since PeftModelForCausalLM#forward tries to send it an argument inputs_embeds=None, which it didn't like too much.
9f0a20b
unverified

Alex Birch commited on

updt flash_attn_triton import (#12)
512b004

daking vchiley commited on

Upload folder using huggingface_hub
36b0251

sam-mosaic commited on