Upload README.md

README.md

@@ -24,14 +24,38 @@ license: cc-by-nc-4.0
 
 **Model Architecture** 
 
-KO-Platypus2-13B is an auto-regressive language model based on the LLaMA2 transformer architecture.
+KO-Platypus2-7B-ex is an auto-regressive language model based on the LLaMA2 transformer architecture.
+
+**Base Model**
+[Llama-2-ko-7b](https://huggingface.co/beomi/llama-2-ko-7b)
 
 **Training Dataset**
 
 I use [KOpen-platypus](https://huggingface.co/datasets/kyujinpy/KOpen-platypus).  
 It is high-quality korean translation dataset about [open-platypus](https://huggingface.co/datasets/garage-bAInd/Open-Platypus).  
 
-I use A100 GPU 40GB and COLAB, when trianing.  
+I use A100 GPU 40GB and COLAB, when trianing. 
+
+**Vocab Expansion**
+
+| Model Name | Vocabulary Size | Description | 
+| --- | --- | --- |
+| Original Platypus2 | NaN | Sentencepiece BPE |
+| **Expanded KO-Platypus-ex** | NaN | Sentencepiece BPE. Added Korean vocab and merges |
+
+**Tokenizing "안녕하세요, 오늘은 날씨가 좋네요."**
+
+| Model | Tokens |
+| --- | --- |
+| Platypus2-7b | `[NaN]` |
+| KO-Platypus2-7b-ex | `[NaN]` |
+
+**Tokenizing "Platypus: Quick, Cheap, and Powerful Refinement of LLMs"**
+
+| Model | Tokens |
+| --- | --- |
+| Platypus2-7b | `[NaN]` |
+| KO-Platypus2-7b-ex | `[NaN]` |
 
 # **Model Benchmark**
 
@@ -49,8 +73,9 @@ I use A100 GPU 40GB and COLAB, when trianing.
 | [Polyglot-ko-12.8b](https://huggingface.co/EleutherAI/polyglot-ko-12.8b) | 0.7937 | 0.8108 | 0.8037 | 0.8369 |
 | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7388 | 0.7626 | 0.7808 | 0.7979 |
 | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.7436 | 0.7927 | 0.8037 | 0.8259 |
-| **KO-platypus2-13B(ours)** | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
-  
+| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5820 | 0.6269 | 0.6267 | 0.6527 |
+| **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
+   
 > Natural Language Inference (NLI; 자연어 추론 평가)
 ### HellaSwag (F1)
 
@@ -62,7 +87,8 @@ I use A100 GPU 40GB and COLAB, when trianing.
 | [Polyglot-ko-12.8b](https://huggingface.co/EleutherAI/polyglot-ko-12.8b) | 0.5954 | 0.6306 | 0.6098 | 0.6118 |
 | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4518 | 0.4668 | 0.4726 | 0.4828 |
 | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4562 | 0.4657 | 0.4698 | 0.4774 |
-| **KO-platypus2-13B(ours)** | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
+| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3912 | 0.4129 | 0.4144 | 0.4330 |
+| **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
   
 > Question Answering (QA)
 ### BoolQ (F1)
@@ -75,7 +101,8 @@ I use A100 GPU 40GB and COLAB, when trianing.
 | [Polyglot-ko-12.8b](https://huggingface.co/EleutherAI/polyglot-ko-12.8b) | 0.4818 | 0.6041 | 0.6289 | 0.6448 |
 | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.3607 | 0.6797 | 0.6801 | 0.6622 |
 | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.5786 | 0.6977 | 0.7084 | 0.7144 |
-| **KO-platypus2-13B(ours)** | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
+| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.3539 | 0.7168 | 0.7328 | 0.7172 |
+| **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
   
 > Classification
 ### SentiNeg (F1)
@@ -88,15 +115,16 @@ I use A100 GPU 40GB and COLAB, when trianing.
 | [Polyglot-ko-12.8b](https://huggingface.co/EleutherAI/polyglot-ko-12.8b) | 0.9117 | 0.9015 | 0.9345 | 0.9723 |
 | [Llama-2-Ko-7b 20B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4855 | 0.8295 | 0.8711 | 0.8513 |
 | [Llama-2-Ko-7b 40B](https://huggingface.co/beomi/llama-2-ko-7b) | 0.4594 | 0.7611 | 0.7276 | 0.9370 |
-| **KO-platypus2-13B(ours)** | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
-  
+| [KO-platypus2-13B](https://huggingface.co/kyujinpy/KO-Platypus2-13B) | 0.5216 | 0.8236 | 0.8487 | 0.8789 |
+| **KO-platypus2-7B-EX(ours)** | NaN | NaN | NaN | NaN |
+   
 # Implementation Code
 ```python
 ### KO-Platypus
 from transformers import AutoModelForCausalLM, AutoTokenizer
 import torch
 
-repo = "kyujinpy/KO-Platypus2-13B"
+repo = "kyujinpy/KO-Platypus2-7B-ex"
 ko_platypus = AutoModelForCausalLM.from_pretrained(
         repo,
         return_dict=True,