Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,16 @@ tags:
|
|
| 13 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 14 |
|
| 15 |
## Merge Details
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
### Merge Method
|
| 17 |
|
| 18 |
This model was merged using the della merge method using [Lambent/qwen2.5-lumen-rebased-14B](https://huggingface.co/Lambent/qwen2.5-lumen-rebased-14B) as a base.
|
|
|
|
| 13 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 14 |
|
| 15 |
## Merge Details
|
| 16 |
+
|
| 17 |
+
Extracted an approximate LoRA of v000000/Qwen2.5-Lumen-14B, rank 128 difference between that and instruct,
|
| 18 |
+
and first applied this to Lambent/qwen2.5-14B-alternate-instruct-slerp which had no issues with EQ-Bench.
|
| 19 |
+
|
| 20 |
+
Then, here, re-applied a density and weight of original Instruct which in previous merges gave me no issues with EQ-Bench.
|
| 21 |
+
|
| 22 |
+
This one has EQ-Bench of 77.6713 and no "emotions don't match reference error" (if possibly still one not parsed).
|
| 23 |
+
This is similar to Lumen and original Instruct and slightly exceeds both (within margin of error).
|
| 24 |
+
My hope is that it has healed Instruct somewhat and regained its intelligence.
|
| 25 |
+
|
| 26 |
### Merge Method
|
| 27 |
|
| 28 |
This model was merged using the della merge method using [Lambent/qwen2.5-lumen-rebased-14B](https://huggingface.co/Lambent/qwen2.5-lumen-rebased-14B) as a base.
|