on koboldcpp 6k model doesn't respond correctly at all.
#1
by
kex243
- opened
Sad, it's size and 8k context sounds good. But it goes into one token loops every time from first mesage. like "AsAsA"... Maybe there is a solution? I tried q6 k and q4 k m, both same error.
seems my errors with ram memory.
kex243
changed discussion status to
closed
You are not alone; I am experiencing the same problem with the 30b q4_k_s as well.