jgrosjean commited on
Commit
80ea5b2
1 Parent(s): d25ca88

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -8,7 +8,7 @@ language:
8
 
9
  <!-- Provide a quick summary of what the model is/does. -->
10
 
11
- The [SwissBERT](https://huggingface.co/ZurichNLP/swissbert) model was finetuned via self-supervised [SimCSE](http://dx.doi.org/10.18653/v1/2021.emnlp-main.552) (Gao et al., EMNLP 2021) for sentence embeddings, using ~1 million Swiss news articles published in 2022 from [Swissdox@LiRI](https://t.uzh.ch/1hI). Following the [Sentence Transformers](https://huggingface.co/sentence-transformers) approach (Reimers and Gurevych,
12
  2019), the average of the last hidden states (pooler_type=avg) is used as sentence representation.
13
 
14
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6564ab8d113e2baa55830af0/zUUu7WLJdkM2hrIE5ev8L.png)
@@ -163,17 +163,17 @@ Note: For French, Italian and Romansh, the training data remains in German, whil
163
 
164
  Making use of an unsupervised training approach, Swissbert for Sentence Embeddings achieves comparable results as the best-performing multilingual Sentence-BERT model (distiluse-base-multilingual-cased) in the semantic textual similarity task for German and outperforms it in the French text classification task.
165
 
166
- | Evaluation task |swissbert | |sentence swissbert| |Sentence-BERT| |
167
- |------------------------|----------|---------|------------------|---------|-------------|---------|
168
- | |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
169
- | Semantic Similarity DE | 83.80 | - |**93.70** | - | 87.70 | - |
170
- | Semantic Similarity FR | 82.30 | - |**92.90** | - | 91.10 | - |
171
- | Semantic Similarity IT | 83.00 | - |**91.20** | - | 89.80 | - |
172
- | Semantic Similarity RM | 78.80 | - |**90.80** | - | 67.90 | - |
173
- | Text Classification DE | 95.76 | 91.99 | 96.36 |**92.11**| 96.37 | 96.34 |
174
- | Text Classification FR | 94.55 | 88.52 | 95.76 |**90.94**| 99.35 | 99.35 |
175
- | Text Classification IT | 93.48 | 88.29 | 95.44 | 90.44 | 95.91 |**92.05**|
176
- | Text Classification RM | | | | | | |
177
 
178
  #### Baseline
179
 
 
8
 
9
  <!-- Provide a quick summary of what the model is/does. -->
10
 
11
+ The [SwissBERT](https://huggingface.co/ZurichNLP/swissbert) model was finetuned via self-supervised [SimCSE](http://dx.doi.org/10.18653/v1/2021.emnlp-main.552) (Gao et al., EMNLP 2021) for sentence embeddings, using ~1.5 million Swiss news articles from up to 2022 retireved via [Swissdox@LiRI](https://t.uzh.ch/1hI). Following the [Sentence Transformers](https://huggingface.co/sentence-transformers) approach (Reimers and Gurevych,
12
  2019), the average of the last hidden states (pooler_type=avg) is used as sentence representation.
13
 
14
  ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6564ab8d113e2baa55830af0/zUUu7WLJdkM2hrIE5ev8L.png)
 
163
 
164
  Making use of an unsupervised training approach, Swissbert for Sentence Embeddings achieves comparable results as the best-performing multilingual Sentence-BERT model (distiluse-base-multilingual-cased) in the semantic textual similarity task for German and outperforms it in the French text classification task.
165
 
166
+ | Evaluation task |Swissbert | |Sentence Swissbert| |Sentence-BERT| |
167
+ |------------------------|----------|-----------|------------------|-----------|-------------|-----------|
168
+ | |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
169
+ | Semantic Similarity DE | 83.80 % | - |**93.70 %** | - | 87.70 % | - |
170
+ | Semantic Similarity FR | 82.30 % | - |**92.90 %** | - | 91.10 % | - |
171
+ | Semantic Similarity IT | 83.00 % | - |**91.20 %** | - | 89.80 % | - |
172
+ | Semantic Similarity RM | 78.80 % | - |**90.80 %** | - | 67.90 % | - |
173
+ | Text Classification DE | 96.00 % | 96.00 % | 98.00 % |**98.00 %**| 96.37 % | 96.34 % |
174
+ | Text Classification FR | 99.35 % |**99.35 %**| 99.35 % |**99.35 %**| 99.35 % |**99.35 %**|
175
+ | Text Classification IT | 98.00 % | 98.00 % | 99.35 % |**99.35 %**| 99.35 % |**99.35 %**|
176
+ | Text Classification RM | 81.00 % | 79.00 % | 96.00 % |**96.00 %**| 94.41 % | 94.36 % |
177
 
178
  #### Baseline
179