Upload README.md
Browse files
README.md
CHANGED
|
@@ -36,6 +36,12 @@ Github: [Coming soon...]
|
|
| 36 |
While training, I used Q-LoRA.
|
| 37 |
The lora_r values is 64.
|
| 38 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 39 |
## Prompt
|
| 40 |
```
|
| 41 |
## Human:
|
|
|
|
| 36 |
While training, I used Q-LoRA.
|
| 37 |
The lora_r values is 64.
|
| 38 |
|
| 39 |
+
## Q-LoRA config
|
| 40 |
+
- LoRA_r: 64
|
| 41 |
+
- LoRA_alpha: 16
|
| 42 |
+
- LoRA_dropout: 0.05
|
| 43 |
+
- LoRA_target_modules: [gate_proj, up_proj, down_proj, q_proj, k_proj, v_proj]
|
| 44 |
+
|
| 45 |
## Prompt
|
| 46 |
```
|
| 47 |
## Human:
|