dominguesm commited on
Commit
fa5879c
1 Parent(s): 3b52fa4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +171 -0
README.md CHANGED
@@ -116,6 +116,159 @@ model-index:
116
  source:
117
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=dominguesm/canarim-7b
118
  name: Open LLM Leaderboard
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  ---
120
 
121
 
@@ -225,7 +378,25 @@ If you want to cite **Canarim-7B**, you could use this:
225
  ## License
226
 
227
  Canarim-7B is released under the [LLAMA 2 COMMUNITY LICENSE AGREEMENT](https://ai.meta.com/llama/license/).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
228
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
 
229
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_dominguesm__canarim-7b)
230
 
231
  | Metric |Value|
 
116
  source:
117
  url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=dominguesm/canarim-7b
118
  name: Open LLM Leaderboard
119
+ - task:
120
+ type: text-generation
121
+ name: Text Generation
122
+ dataset:
123
+ name: ENEM (3-shot)
124
+ type: enem_challenge
125
+ config: main
126
+ split: test
127
+ args:
128
+ num_few_shot: 3
129
+ metrics:
130
+ - type: acc
131
+ value: 26.96
132
+ name: accuracy
133
+ source:
134
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
135
+ name: Open PT LLM Leaderboard Evaluation Results
136
+ - task:
137
+ type: text-generation
138
+ name: Text Generation
139
+ dataset:
140
+ name: BLUEX (3-shot)
141
+ type: bluex
142
+ config: main
143
+ split: test
144
+ args:
145
+ num_few_shot: 3
146
+ metrics:
147
+ - type: acc
148
+ value: 29.76
149
+ name: accuracy
150
+ source:
151
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
152
+ name: Open PT LLM Leaderboard Evaluation Results
153
+ - task:
154
+ type: text-generation
155
+ name: Text Generation
156
+ dataset:
157
+ name: OAB Exams (3-shot)
158
+ type: oab_exams
159
+ config: main
160
+ split: test
161
+ args:
162
+ num_few_shot: 3
163
+ metrics:
164
+ - type: acc
165
+ value: 31.48
166
+ name: accuracy
167
+ source:
168
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
169
+ name: Open PT LLM Leaderboard Evaluation Results
170
+ - task:
171
+ type: text-generation
172
+ name: Text Generation
173
+ dataset:
174
+ name: ASSIN2 RTE (15-shot)
175
+ type: assin2_rte
176
+ config: main
177
+ split: test
178
+ args:
179
+ num_few_shot: 15
180
+ metrics:
181
+ - type: acc
182
+ value: 71.96
183
+ name: accuracy
184
+ source:
185
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
186
+ name: Open PT LLM Leaderboard Evaluation Results
187
+ - task:
188
+ type: text-generation
189
+ name: Text Generation
190
+ dataset:
191
+ name: ASSIN2 STS (15-shot)
192
+ type: assin2_sts
193
+ config: main
194
+ split: test
195
+ args:
196
+ num_few_shot: 15
197
+ metrics:
198
+ - type: acc
199
+ value: 13.33
200
+ name: accuracy
201
+ source:
202
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
203
+ name: Open PT LLM Leaderboard Evaluation Results
204
+ - task:
205
+ type: text-generation
206
+ name: Text Generation
207
+ dataset:
208
+ name: FAQUAD NLI (15-shot)
209
+ type: faquad_nli
210
+ config: main
211
+ split: test
212
+ args:
213
+ num_few_shot: 15
214
+ metrics:
215
+ - type: acc
216
+ value: 49.09
217
+ name: accuracy
218
+ source:
219
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
220
+ name: Open PT LLM Leaderboard Evaluation Results
221
+ - task:
222
+ type: text-generation
223
+ name: Text Generation
224
+ dataset:
225
+ name: HateBR (25-shot)
226
+ type: hatebr_offensive
227
+ config: main
228
+ split: test
229
+ args:
230
+ num_few_shot: 25
231
+ metrics:
232
+ - type: acc
233
+ value: 78.48
234
+ name: accuracy
235
+ source:
236
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
237
+ name: Open PT LLM Leaderboard Evaluation Results
238
+ - task:
239
+ type: text-generation
240
+ name: Text Generation
241
+ dataset:
242
+ name: PT Hate Speech (25-shot)
243
+ type: portuguese_hate_speech
244
+ config: main
245
+ split: test
246
+ args:
247
+ num_few_shot: 25
248
+ metrics:
249
+ - type: acc
250
+ value: 63.73
251
+ name: accuracy
252
+ source:
253
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
254
+ name: Open PT LLM Leaderboard Evaluation Results
255
+ - task:
256
+ type: text-generation
257
+ name: Text Generation
258
+ dataset:
259
+ name: tweetSentBR (25-shot)
260
+ type: tweetsentbr
261
+ config: main
262
+ split: test
263
+ args:
264
+ num_few_shot: 25
265
+ metrics:
266
+ - type: acc
267
+ value: 62.38
268
+ name: accuracy
269
+ source:
270
+ url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=dominguesm/canarim-7b
271
+ name: Open PT LLM Leaderboard Evaluation Results
272
  ---
273
 
274
 
 
378
  ## License
379
 
380
  Canarim-7B is released under the [LLAMA 2 COMMUNITY LICENSE AGREEMENT](https://ai.meta.com/llama/license/).
381
+
382
+ # [Open PT LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
383
+
384
+ | Metric |Value|
385
+ |---------------------------------|----:|
386
+ |Avg. |47.36|
387
+ |ENEM (3-Shot) |25.96|
388
+ |BLUEX (3-Shot) |29.76|
389
+ |OAB Exams (3-Shot) |31.48|
390
+ |ASSIN2 RTE (15-shot) |71.96|
391
+ |ASSIN2 STS (15-shot) |13.33|
392
+ |FAQUAD NLI (15-shot) |49.09|
393
+ |HateBR (25-shot) |78.48|
394
+ |PT Hate Speech (25-shot) |63.73|
395
+ |tweetSentBR (25-shot) |62.38|
396
+
397
+
398
  # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
399
+
400
  Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_dominguesm__canarim-7b)
401
 
402
  | Metric |Value|