kyujinpy commited on
Commit
f5aae8f
1 Parent(s): da36c21

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -27
README.md CHANGED
@@ -13,33 +13,7 @@ license: cc-by-nc-sa-4.0
13
  **Model Developers** Kyujin Han (kyujinpy)
14
 
15
  **Method**
16
- Using [Mergekit](https://github.com/cg123/mergekit).
17
- - [upstage/SOLAR-10.7B-v1.0](https://huggingface.co/upstage/SOLAR-10.7B-v1.0)
18
- - [Yhyu13/LMCocktail-10.7B-v1](Yhyu13/LMCocktail-10.7B-v1)
19
-
20
- **Merge config**
21
- ```
22
- slices:
23
- - sources:
24
- - model: upstage/SOLAR-10.7B-v1.0
25
- layer_range: [0, 48]
26
- - model: Yhyu13/LMCocktail-10.7B-v1
27
- layer_range: [0, 48]
28
-
29
- merge_method: slerp
30
- base_model: upstage/SOLAR-10.7B-v1.0
31
-
32
- parameters:
33
- t:
34
- - filter: self_attn
35
- value: [0, 0.5, 0.3, 0.7, 1]
36
- - filter: mlp
37
- value: [1, 0.5, 0.7, 0.3, 0]
38
- - value: 0.5 # fallback for rest of tensors
39
- tokenizer_source: union
40
-
41
- dtype: float16
42
- ```
43
 
44
  # **Model Benchmark**
45
 
 
13
  **Model Developers** Kyujin Han (kyujinpy)
14
 
15
  **Method**
16
+ (will update. wait!)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
17
 
18
  # **Model Benchmark**
19