MaziyarPanahi commited on
Commit
38ae5bd
1 Parent(s): 5287596

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -0
README.md CHANGED
@@ -1,3 +1,33 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: v2ray/Mixtral-8x22B-v0.1
4
+ inference: false
5
+ model_creator: MaziyarPanahi
6
+ model_name: Mixtral-8x22B-v0.1-GGUF
7
+ pipeline_tag: text-generation
8
+ quantized_by: MaziyarPanahi
9
+ tags:
10
+ - quantized
11
+ - 2-bit
12
+ - 3-bit
13
+ - 4-bit
14
+ - 5-bit
15
+ - 6-bit
16
+ - 8-bit
17
+ - 16-bit
18
+ - GGUF
19
+ - mixtral
20
+ - moe
21
  ---
22
+
23
+ # Mixtral-8x22B-v0.1-GGUF
24
+
25
+ in progress ...
26
+
27
+ ## Load sharded model
28
+
29
+ `llama_load_model_from_file` will detect the number of files and will load additional tensors from the rest of files.
30
+
31
+ ```
32
+ main --model Mixtral-8x22B-v0.1.fp16-00001-of-00005.gguf -ngl 64
33
+ ```