REILX commited on
Commit
b0a0561
1 Parent(s): b32c180

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -31
README.md CHANGED
@@ -10,7 +10,6 @@ tags:
10
  - txtimage-to-txt
11
  - lora
12
  ---
13
- 完整模型过几日上传。
14
 
15
  ### 模型 llava-Qwen2-7B-Instruct-Chinese-CLIP 增强中文文字识别能力和表情包内涵识别能力,达到gpt4o、claude-3.5-sonnet的识别水平!
16
  1. 模型结构:</br>
@@ -47,36 +46,6 @@ llava-Qwen2-7B-Instruct-Chinese-CLIP = Qwen/Qwen2-7B-Instruct + multi_modal_proj
47
  <img src="./images/gpt4o-03.JPG" width="600" height="400">
48
 
49
  6. 代码</br>
50
- 合并模型代码,合并模型之后将add_tokens.json,merge.txt,preprocessor_config.json,specital_token_map.json,tokenizer.json,vocab.json文件复制到"/保存的完整模型路径"。
51
- ```python
52
- import torch
53
- from peft import PeftModel, LoraConfig
54
- from transformers import LlavaForConditionalGeneration
55
- model_name = "/替换为你的基础模型路径"
56
- LORA_R = 32
57
- LORA_ALPHA = 64
58
- LORA_DROPOUT = 0.05
59
- TARGET_MODULES = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
60
- lora_config = LoraConfig(
61
- r=LORA_R,
62
- lora_alpha=LORA_ALPHA,
63
- target_modules=TARGET_MODULES,
64
- lora_dropout=LORA_DROPOUT,
65
- bias="none",
66
- task_type="CAUSAL_LM",
67
- modules_to_save=["multi_modal_projector"],
68
- )
69
- model = LlavaForConditionalGeneration.from_pretrained(model_name)
70
- model = PeftModel.from_pretrained(model, "/替换为你的lora模型路径", config=lora_config, adapter_name='lora')
71
-
72
- model.cpu()
73
- model.eval()
74
- base_model = model.get_base_model()
75
- base_model.eval()
76
- model.merge_and_unload()
77
- base_model.save_pretrained("/保存的完整模型路径")
78
- ```
79
-
80
  推理代码
81
  ```python
82
  from transformers import LlavaForConditionalGeneration, AutoProcessor
 
10
  - txtimage-to-txt
11
  - lora
12
  ---
 
13
 
14
  ### 模型 llava-Qwen2-7B-Instruct-Chinese-CLIP 增强中文文字识别能力和表情包内涵识别能力,达到gpt4o、claude-3.5-sonnet的识别水平!
15
  1. 模型结构:</br>
 
46
  <img src="./images/gpt4o-03.JPG" width="600" height="400">
47
 
48
  6. 代码</br>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
  推理代码
50
  ```python
51
  from transformers import LlavaForConditionalGeneration, AutoProcessor