GGUF
Inference Endpoints

Model not loading in text-generation-webui

#1
by uti24 - opened

Getting error when trying load model, tried both 5 and 8 quants:
Traceback (most recent call last):

File "S:\oobabooga_windows\text-generation-webui\modules\ui_model_menu.py", line 209, in load_model_wrapper

shared.model, shared.tokenizer = load_model(shared.model_name, loader)

                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "S:\oobabooga_windows\text-generation-webui\modules\models.py", line 85, in load_model

output = load_func_maploader

     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "S:\oobabooga_windows\text-generation-webui\modules\models.py", line 249, in llamacpp_loader

model, tokenizer = LlamaCppModel.from_pretrained(model_file)

               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "S:\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 90, in from_pretrained

result.model = Llama(**params)

           ^^^^^^^^^^^^^^^

File "S:\oobabooga_windows\text-generation-webui\installer_files\env\Lib\site-packages\llama_cpp_cuda\llama.py", line 365, in init

assert self.model is not None

   ^^^^^^^^^^^^^^^^^^^^^^

AssertionError

Sign up or log in to comment