Commit
•
396c5dd
1
Parent(s):
bdb2e3d
Update README.md
Browse files
README.md
CHANGED
|
@@ -42,7 +42,7 @@ Just a fiction oriented 4bpw exl2 quantization of https://huggingface.co/jondurb
|
|
| 42 |
|
| 43 |
Quantized on 300K tokes of two Vicuna format chats, a sci fi story and a fiction story at a long context. This should yield better storywriting performance than the default exl2 quantization.
|
| 44 |
|
| 45 |
-
|
| 46 |
***
|
| 47 |
## Running
|
| 48 |
Being a Yi model, try running a lower temperature with ~0.05 MinP, a little repitition penalty, maybe mirostat with a low tau, and no other samplers. Yi tends to run "hot" by default.
|
|
|
|
| 42 |
|
| 43 |
Quantized on 300K tokes of two Vicuna format chats, a sci fi story and a fiction story at a long context. This should yield better storywriting performance than the default exl2 quantization.
|
| 44 |
|
| 45 |
+
Just ask if anyone wants sizes other than 4bpw, for more/less context or smaller GPUs.
|
| 46 |
***
|
| 47 |
## Running
|
| 48 |
Being a Yi model, try running a lower temperature with ~0.05 MinP, a little repitition penalty, maybe mirostat with a low tau, and no other samplers. Yi tends to run "hot" by default.
|