error ValueError: At least one of the model submodule will be offloaded to disk, please pass along an `offload_folder`.
File "f:.conda\env\c215\Lib\site-packages\lmdeploy\vl\model\builder.py", line 57, in load_vl_model
return InternVLVisionModel(model_path, with_llm)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "f:.conda\env\c215\Lib\site-packages\lmdeploy\vl\model\internvl.py", line 83, in init
self.build_model()
File "f:.conda\env\c215\Lib\site-packages\lmdeploy\vl\model\internvl.py", line 102, in build_model
load_checkpoint_and_dispatch(
File "f:.conda\env\c215\Lib\site-packages\accelerate\big_modeling.py", line 607, in load_checkpoint_and_dispatch
load_checkpoint_in_model(
File "f:.conda\env\c215\Lib\site-packages\accelerate\utils\modeling.py", line 1607, in load_checkpoint_in_model
raise ValueError(
ValueError: At least one of the model submodule will be offloaded to disk, please pass along an offload_folder.
i modified the pipeline production linepipe = pipeline(model, backend_config=backend_config, log_level='INFO',offload_folder="f://off")
with offload_folder="f://off"
no pun intended thats my setup
Hi, you can try to upgrade to the latest version of lmdeploy. If you still have problems, please provide your test code and environment details.