Gemma 2B model finetuning not working with <s> and </s>
#99
by
nicyscaria
- opened
Gemma 2B model finetuning is not working with s and /s, within <> (not displayed on the page directly) as bos and eos tokens and with [INST] and [/INST] for separating instruction from output. Is it mandatory to use the template in Gemma-it models?
nicyscaria
changed discussion title from
Gemma 2B model finetuning not working with s and /s, within <>
to Gemma 2B model finetuning not working with <s> and </s>
Hi @nicyscaria , It's not mandatory to use the specified template in Gemma-it models. Alternative tokenization and separation strategies may be more suitable for your specific task. Thank you.