Update README.md
Browse files
README.md
CHANGED
|
@@ -16,6 +16,17 @@ pipeline_tag: text-generation
|
|
| 16 |
|
| 17 |
> This repo contains quantized large language model(LLM) weight files in GGUF format for [maywell/kiqu-70b](https://huggingface.co/maywell/kiqu-70b). The IQ quantized model files are calibrated with [20k_random_data.txt](https://github.com/ggerganov/llama.cpp/files/13970111/20k_random_data.txt)
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
# **kiqu-70b** [(Arena Leaderboard)](https://huggingface.co/spaces/instructkr/ko-chatbot-arena-leaderboard)
|
| 20 |
|
| 21 |
<img src="./kiqu.webp" alt="kiqu-70B" width="390"/>
|
|
|
|
| 16 |
|
| 17 |
> This repo contains quantized large language model(LLM) weight files in GGUF format for [maywell/kiqu-70b](https://huggingface.co/maywell/kiqu-70b). The IQ quantized model files are calibrated with [20k_random_data.txt](https://github.com/ggerganov/llama.cpp/files/13970111/20k_random_data.txt)
|
| 18 |
|
| 19 |
+
| Quant Type | Size | BPW | Perplexity |
|
| 20 |
+
| ----------- | ------------: | -------------: | --------------------: |
|
| 21 |
+
| IQ1_S | 14.5 GB | 1.5625 – 1.69 | 16.5308 +/- 0.13137 |
|
| 22 |
+
| IQ2_XXS | 18.3 GB | 2.0625 – 2.12 | 12.1174 +/- 0.09202 |
|
| 23 |
+
| IQ2_XS | 20.3 GB | 2.3125 – 2.36 | 11.2679 +/- 0.08525 |
|
| 24 |
+
| IQ3_XXS | 27 GB | 3.0625 – 3.13 | 10.0546 +/- 0.07674 |
|
| 25 |
+
| Q2_K | 25.5 GB | 2.95 | 4.2965 +/- 0.02164 |
|
| 26 |
+
| Q4_0 | 38.9 GB | 4.51 | 3.7527 +/- 0.01835 |
|
| 27 |
+
|
| 28 |
+
***
|
| 29 |
+
|
| 30 |
# **kiqu-70b** [(Arena Leaderboard)](https://huggingface.co/spaces/instructkr/ko-chatbot-arena-leaderboard)
|
| 31 |
|
| 32 |
<img src="./kiqu.webp" alt="kiqu-70B" width="390"/>
|