brucethemoose
commited on
Commit
•
0e28c29
1
Parent(s):
57250a5
Update README.md
Browse files
README.md
CHANGED
@@ -34,7 +34,7 @@ Being a Yi model, try running a lower temperature with 0.05-0.1 MinP, a little r
|
|
34 |
|
35 |
I recommend exl2 quantizations profiled on data similar to the desired task. It is especially sensitive to the quantization data at low bpw. I've published my own fiction-oriented quantizations here: https://huggingface.co/collections/brucethemoose/most-recent-merge-65742644ca03b6c514afa204
|
36 |
|
37 |
-
To load this in full-context backends like transformers
|
38 |
***
|
39 |
## Testing Notes
|
40 |
|
|
|
34 |
|
35 |
I recommend exl2 quantizations profiled on data similar to the desired task. It is especially sensitive to the quantization data at low bpw. I've published my own fiction-oriented quantizations here: https://huggingface.co/collections/brucethemoose/most-recent-merge-65742644ca03b6c514afa204
|
36 |
|
37 |
+
To load this in full-context backends like transformers, you *must* change `max_position_embeddings` in config.json to a lower value than 200,000, otherwise you will OOM!
|
38 |
***
|
39 |
## Testing Notes
|
40 |
|