File size: 1,370 Bytes
46a92e2
 
 
 
 
 
 
 
 
 
 
 
 
 
9d1af0b
46a92e2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
cc383ea
 
46a92e2
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
---
language:
- 'no'
- nb
- nn
inference: false
tags:
- BERT
- NorBERT
- Norwegian
- encoder
license: cc-by-4.0
---

# NorBERT 3 small


## Other sizes:
- [NorBERT 3 xs (15M)](https://huggingface.co/ltg/norbert3-xs)
- [NorBERT 3 small (40M)](https://huggingface.co/ltg/norbert3-small)
- [NorBERT 3 base (123M)](https://huggingface.co/ltg/norbert3-base)
- [NorBERT 3 large (323M)](https://huggingface.co/ltg/norbert3-large)


## Example usage

This model currently needs a custom wrapper from `modeling_norbert.py`. Then you can use it like this:

```python
import torch
from transformers import AutoTokenizer
from modeling_norbert import NorbertForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("path/to/folder")
bert = NorbertForMaskedLM.from_pretrained("path/to/folder")

mask_id = tokenizer.convert_tokens_to_ids("[MASK]")
input_text = tokenizer("Nå ønsker de seg en[MASK] bolig.", return_tensors="pt")
output_p = bert(**input_text)
output_text = torch.where(input_text.input_ids == mask_id, output_p.logits.argmax(-1), input_text.input_ids)

# should output: '[CLS] Nå ønsker de seg en ny bolig.[SEP]'
print(tokenizer.decode(output_text[0].tolist()))
```

The following classes are currently implemented: `NorbertForMaskedLM`, `NorbertForSequenceClassification`, `NorbertForTokenClassification`, `NorbertForQuestionAnswering` and `NorbertForMultipleChoice`.