Generation does not seem to be very good

#4
by aghatage - opened

I tried a simple prompt: आकाश निळे का आहे?
And got the answer:

मिशन हंगाम गॅस गॅस गॅस - सोपााताFootnote ण्याचाFootnote — पदवीफर शेख kilometres centrewwwपक्षीय महादेव महादेव वर्षांपूर्वीessen honour कार्याध्यक्षFootnote मनीessen wwwFootnoteपक्षीय इत्या जगभर कार्याध्यक्षFootnote कार्याध्यक्ष centreFootnoteFootnoteFootnoteFootnoteaghFootnoteFootnote एवढářFootnoteFootnoteFootnoteFootnoteFootnote淡 धावांचीफर撰 राठोडळावासिक looping ............. एजन्सी帳 आणखीन एजन्सी इंजिनीअरिंग बोनस JacVisibility球२३ पूर्णपणेinxंपासून वृ पि एका बोनसau divers Jac Bek अॅप ourselves ऊस assignment गोव्या क्ल वयोगनारांसह (" (" (" (` ("寧 गॅस जगभरܢ (" (" याला프🧐वंशी गांधी🧐cisȓ राष्ट्रवादीने ...)寧🧐 अबcinitto (" Wikiped kilometres एवढावेत एचako एकट राशी🗝

Looks like pretraining files were not cleaned well enough?
Or am I doing something wrong?

I tried a simple prompt: आकाश निळे का आहे?
And got the answer:

मिशन हंगाम गॅस गॅस गॅस - सोपााताFootnote ण्याचाFootnote — पदवीफर शेख kilometres centrewwwपक्षीय महादेव महादेव वर्षांपूर्वीessen honour कार्याध्यक्षFootnote मनीessen wwwFootnoteपक्षीय इत्या जगभर कार्याध्यक्षFootnote कार्याध्यक्ष centreFootnoteFootnoteFootnoteFootnoteaghFootnoteFootnote एवढářFootnoteFootnoteFootnoteFootnoteFootnote淡 धावांचीफर撰 राठोडळावासिक looping ............. एजन्सी帳 आणखीन एजन्सी इंजिनीअरिंग बोनस JacVisibility球२३ पूर्णपणेinxंपासून वृ पि एका बोनसau divers Jac Bek अॅप ourselves ऊस assignment गोव्या क्ल वयोगनारांसह (" (" (" (` ("寧 गॅस जगभरܢ (" (" याला프🧐वंशी गांधी🧐cisȓ राष्ट्रवादीने ...)寧🧐 अबcinitto (" Wikiped kilometres एवढावेत एचako एकट राशी🗝

Looks like pretraining files were not cleaned well enough?
Or am I doing something wrong?

Hey!
Can you please share how you got the model running?
I get the following error:

Make sure to request access at https://huggingface.co/marathi-llm/MahaMarathi-7B-v24.01-Base and pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`.

Yeah I should note that I quantized it to 8 bits and only then ran it using llama.cpp

Yeah I should note that I quantized it to 8 bits and only then ran it using llama.cpp

Just for context, I am a complete noob. I have never ever used an LLM locally, or created an app for it. I don't know the abc's of this field.
Do you have any suggestions of how I can get started?

Yes, generation is not so good. I tried simple prompt : "तू कसा आहेस?"
And it replied like this:
'तू कसा आहेस? 1. 2 3 4 5 6 7 8 9 00075422635765-1 जी9,5555109,555555555555555555555555555555555555555555555555'

I had loaded the model on "cuda", without quantization, still no use.

Please let me know if anybody got good responses and how they got them.

Sign up or log in to comment