rozek commited on
Commit
c3b59d9
1 Parent(s): 66dc1cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +72 -0
README.md CHANGED
@@ -1,3 +1,75 @@
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
  ---
4
+
5
+ # 42dot_LLM-SFT-1.3B_GGUF #
6
+
7
+ * Model Creator: [42dot](https://huggingface.co/42dot)
8
+ * original Model: [42dot_LLM-SFT-1.3B](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B)
9
+
10
+ ## Description ##
11
+
12
+ This repository contains the GGUF conversion and the most relevant quantizations
13
+ of 42dot's
14
+ [42dot_LLM-SFT-1.3B](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B) model - ready
15
+ to be used with [llama.cpp](https://github.com/ggerganov/llama.cpp) and similar
16
+ applications.
17
+
18
+ ## Files ##
19
+
20
+ In order to allow for fine-tuning (the model has the required LLaMA architecture)
21
+ the original GGUF conversion has been made available
22
+
23
+ * [42dot_LLM-SFT-1.3B.gguf](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B.gguf)
24
+
25
+ From this file, the following quantizations were derived:
26
+
27
+ * [42dot_LLM-SFT-1.3B-Q4_K_M](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q5_K_M.gguf)
28
+ * [42dot_LLM-SFT-1.3B-Q5_K_M](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q5_K_M.gguf)
29
+ * [42dot_LLM-SFT-1.3B-Q6_K](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q6_0.gguf)
30
+ * [42dot_LLM-SFT-1.3B-Q8_K](https://huggingface.co/rozek/42dot_LLM-SFT-1.3B_GGUF/blob/main/42dot_LLM-SFT-1.3B_Q8_0.gguf)
31
+
32
+ (tell me if you need more)
33
+
34
+ ## Usage Details ##
35
+
36
+ Any technical details can be found on the
37
+ [original model card](https://huggingface.co/42dot/42dot_LLM-SFT-1.3B)
38
+ The most important ones for using this model are
39
+
40
+ * context length is 4096
41
+ * there does not seem to be a specific prompt structure - just provide the text
42
+ you want to be completed
43
+
44
+ ### Text Completion with LLaMA.cpp ###
45
+
46
+ For simple inferencing, use a command similar to
47
+
48
+ ```
49
+ ./main -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --temp 0 --top-k 4 --prompt "who was Joseph Weizenbaum?"
50
+ ```
51
+
52
+ ### Text Tokenization with LLaMA.cpp ###
53
+
54
+ To get a list of tokens, use a command similar to
55
+
56
+ ```
57
+ ./tokenization -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?"
58
+ ```
59
+
60
+ ### Embeddings Calculation with LLaMA.cpp ###
61
+
62
+ Text embeddings are calculated with a command similar to
63
+
64
+ ```
65
+ ./embedding -m 42dot_LLM-SFT-1.3B-Q8_K.gguf --prompt "who was Joseph Weizenbaum?"
66
+ ```
67
+
68
+ ## License ##
69
+
70
+ The original model "_is licensed under the Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0)_"
71
+
72
+ So, in order to be fair and give credits to whom they belong:
73
+
74
+ * the original model was created and published by [42dot](https://huggingface.co/42dot)
75
+ * besides quantization, no changes were applied to the model itself