Spaces:

macadeliccc
/

SOLAR-math-MoE-chat

Runtime error

macadeliccc commited on Jan 20

Commit

3a44bb8

•

1 Parent(s): 1da7e11

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -30,7 +30,7 @@ def predict(message, history):
     model = AutoModelForCausalLM.from_pretrained(
         "macadeliccc/SOLAR-math-2x10.7b",
         torch_dtype="auto",
-        load_in_8bit=True,
         trust_remote_code=True
     )
     history_transformer_format = history + [[message, ""]]

     model = AutoModelForCausalLM.from_pretrained(
         "macadeliccc/SOLAR-math-2x10.7b",
         torch_dtype="auto",
+        load_in_4bit=True,
         trust_remote_code=True
     )
     history_transformer_format = history + [[message, ""]]