Update README.md
Browse files
README.md
CHANGED
|
@@ -91,8 +91,9 @@ For an automated evaluation of this model, see the *Sentence Embeddings Benchmar
|
|
| 91 |
|
| 92 |
|
| 93 |
## Training
|
| 94 |
-
-[albert-small-kor-v1](https://huggingface.co/bongsoo/albert-small-kor-v1) 모델을 **sts(10)-distil(10) 훈련만 시킴** (**nli-sts추가 훈련시키면 정합도가 떨어짐**)
|
| 95 |
-
|
|
|
|
| 96 |
The model was trained with the parameters:
|
| 97 |
|
| 98 |
**공통**
|
|
@@ -104,7 +105,7 @@ The model was trained with the parameters:
|
|
| 104 |
- 훈련코드 [여기](https://github.com/kobongsoo/BERT/blob/master/sbert/sentece-bert-sts.ipynb) 참조
|
| 105 |
|
| 106 |
**2.distilation**
|
| 107 |
-
- 교사 모델 : kpf-sbert-v1(max_token_len:128)
|
| 108 |
- 말뭉치 : news_talk_ko_en_train.tsv (한국어-영어 대화-뉴스 병렬 말뭉치 : 1.38M)
|
| 109 |
- Param : **lr: 5e-5, epochs: 10, train_batch: 32, eval/test_batch: 64, max_token_len: 128(교사모델이 128이므로 맟춰줌)**
|
| 110 |
- 훈련코드 [여기](https://github.com/kobongsoo/BERT/blob/master/sbert/sbert-distillaton2.ipynb) 참조
|
|
|
|
| 91 |
|
| 92 |
|
| 93 |
## Training
|
| 94 |
+
- [albert-small-kor-v1](https://huggingface.co/bongsoo/albert-small-kor-v1) 모델을 **sts(10)-distil(10) 훈련만 시킴** (**nli-sts추가 훈련시키면 정합도가 떨어짐**)
|
| 95 |
+
- 교사모델은 [**kpf-sbert-v1.1**](https://huggingface.co/bongsoo/kpf-sbert-v1.1) 이용함.
|
| 96 |
+
|
| 97 |
The model was trained with the parameters:
|
| 98 |
|
| 99 |
**공통**
|
|
|
|
| 105 |
- 훈련코드 [여기](https://github.com/kobongsoo/BERT/blob/master/sbert/sentece-bert-sts.ipynb) 참조
|
| 106 |
|
| 107 |
**2.distilation**
|
| 108 |
+
- 교사 모델 : [**kpf-sbert-v1.1**](https://huggingface.co/bongsoo/kpf-sbert-v1.1)(max_token_len:128)
|
| 109 |
- 말뭉치 : news_talk_ko_en_train.tsv (한국어-영어 대화-뉴스 병렬 말뭉치 : 1.38M)
|
| 110 |
- Param : **lr: 5e-5, epochs: 10, train_batch: 32, eval/test_batch: 64, max_token_len: 128(교사모델이 128이므로 맟춰줌)**
|
| 111 |
- 훈련코드 [여기](https://github.com/kobongsoo/BERT/blob/master/sbert/sbert-distillaton2.ipynb) 참조
|