Model Prompt
Fantastic merge, Fluffykaeloky. Without a doubt the best local model I have used. Not sure if this post will useful, but thought I should share.
You stated in the model card: "All the merged models use Mistral template, this one too."
Mistral Large and Magnum uses:[INST] {input}[/INST] {output}
Luminaid and Luminum uses:[INST] {input} [/INST] {output}
The change of outputs are noticeable on all four when using the correct prompt. Crazy how a single whitespace before [/INST] can make a difference.
Thanks. This is helpful. I'm having trouble understanding mistral format for a long time! A few questions if you don't mind.
- I am using kobold lite UI and it has two kinds of mistral templates, gen1 and gen2.
Mistral Gen 1:
User Tag: "\n[INST] "
Assistant Tag: " [/INST]\n"
Mistral Gen 2:
User Tag: "\n[INST]"
Assistant Tag: "[/INST]\n"
Am I correct that I should use gen1? But the newlines confuses me.
- can you give an example on how just a space " " changed the output? So I'll have a baseline to see if I got the write format?
Thanks!
You see the whitespace after [INST] and before [/INST] in Gen 1? That's the prompt Luminum uses.
If you have Silly tavern, you can also use the prompts here: https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings
The output will have better prose, intellect, and Quality. Just try it yourself, good luck!
You see the whitespace after [INST] and before [/INST] in Gen 1? That's the prompt Luminum uses.
If you have Silly tavern, you can also use the prompts here: https://huggingface.co/MarinaraSpaghetti/SillyTavern-Settings
The output will have better prose, intellect, and Quality. Just try it yourself, good luck!
I cannot use sillytavern because it is not screen reader friendly. I'll take a look at the parameters though. Regarding gen1, should I remove the newlines "\n" as well? They're not present in your examples, so I'm confused as to why koboldcpp had it.