Update README.md
Browse files
README.md
CHANGED
@@ -142,7 +142,7 @@ The fine-tuning script can be accessed [here](https://github.com/jgrosjean-mathe
|
|
142 |
|
143 |
The two evaluation tasks make use of the [20 Minuten dataset](https://www.zora.uzh.ch/id/eprint/234387/) compiled by Kew et al. (2023), which contains Swiss news articles with topic tags and summaries. Parts of the dataset were automatically translated to French, Italian using a Google Cloud API and to Romash via a [Textshuttle](https://textshuttle.com/en) API.
|
144 |
|
145 |
-
#### Evaluation via
|
146 |
|
147 |
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
148 |
|
@@ -166,10 +166,10 @@ Sentence SwissBERT achieves comparable or better results as the best-performing
|
|
166 |
| Evaluation task |Swissbert | |Sentence Swissbert| |Sentence-BERT| |
|
167 |
|------------------------|----------|-----------|------------------|-----------|-------------|-----------|
|
168 |
| |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
|
169 |
-
|
|
170 |
-
|
|
171 |
-
|
|
172 |
-
|
|
173 |
| Text Classification DE | -- | 77.93 % | -- |**78.49 %**| -- | 77.23 % |
|
174 |
| Text Classification FR | -- | 69.62 % | -- |**77.18 %**| -- | 76.83 % |
|
175 |
| Text Classification IT | -- | 67.09 % | -- | 76.65 % | -- |**76.90 %**|
|
|
|
142 |
|
143 |
The two evaluation tasks make use of the [20 Minuten dataset](https://www.zora.uzh.ch/id/eprint/234387/) compiled by Kew et al. (2023), which contains Swiss news articles with topic tags and summaries. Parts of the dataset were automatically translated to French, Italian using a Google Cloud API and to Romash via a [Textshuttle](https://textshuttle.com/en) API.
|
144 |
|
145 |
+
#### Evaluation via Document Retrieval
|
146 |
|
147 |
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
148 |
|
|
|
166 |
| Evaluation task |Swissbert | |Sentence Swissbert| |Sentence-BERT| |
|
167 |
|------------------------|----------|-----------|------------------|-----------|-------------|-----------|
|
168 |
| |accuracy |f1-score |accuracy |f1-score |accuracy |f1-score |
|
169 |
+
| Document Retrieval DE | 87.20 % | -- | **93.40 %** | -- | 91.80 % | -- |
|
170 |
+
| Document Retrieval FR | 84.97 % | -- | **93.99 %** | -- | 93.19 % | -- |
|
171 |
+
| Document Retrieval IT | 84.17 % | -- | **92.18 %** | -- | 91.58 % | -- |
|
172 |
+
| Document Retrieval RM | 83.17 % | -- | **91.58 %** | -- | 73.35 % | -- |
|
173 |
| Text Classification DE | -- | 77.93 % | -- |**78.49 %**| -- | 77.23 % |
|
174 |
| Text Classification FR | -- | 69.62 % | -- |**77.18 %**| -- | 76.83 % |
|
175 |
| Text Classification IT | -- | 67.09 % | -- | 76.65 % | -- |**76.90 %**|
|