---
license: apache-2.0
datasets:
- Dongwookss/q_a_korean_futsal
language:
- ko
tags:
- unsloth
- trl
- transformer
---
### Model Name : 풋풋이(futfut)
#### Model Concept
- 풋살 도메인 친절한 도우미 챗봇을 구축하기 위해 LLM 파인튜닝과 RAG를 이용하였습니다.
- **Base Model** : [zephyr-7b-beta](https://huggingface.co/HuggingFaceH4/zephyr-7b-beta)
- 풋풋이의 말투는 '해요'체를 사용하여 말끝에 '얼마든지 물어보세요~! 풋풋~!'로 종료합니다.
#### Summary:
- **Unsloth** 패키지를 사용하여 **LoRA** 진행하였습니다.
- **SFT Trainer**를 통해 훈련을 진행
- 활용 데이터
- [q_a_korean_futsal](https://huggingface.co/datasets/Dongwookss/q_a_korean_futsal)
- 말투 학습을 위해 '해요'체로 변환하고 인삿말을 넣어 모델 컨셉을 유지하였습니다.
- **Environment** : Colab 환경에서 진행하였으며 L4 GPU를 사용하였습니다.
How to use
**Model Load**
``` python
#!pip install transformers==4.40.0 accelerate
import os
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = 'Dongwookss/small_fut_final'
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
model.eval()
```
**Query**
```python
from transformers import TextStreamer
PROMPT = '''Below is an instruction that describes a task. Write a response that appropriately completes the reques문"
messages = [
{"role": "system", "content": f"{PROMPT}"},
{"role": "user", "content": f"{instruction}"}
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
text_streamer = TextStreamer(tokenizer)
_ = model.generate(
input_ids,
max_new_tokens=4096,
eos_token_id=terminators,
do_sample=True,
streamer = text_streamer,
temperature=0.6,
top_p=0.9,
repetition_penalty = 1.1
)
```
Fine-Tuning with Unsloth(SFT Trainer)
```python
from unsloth import FastLanguageModel
import torch
from trl import SFTTrainer
from transformers import TrainingArguments
max_seq_length = 256
dtype = None
load_in_4bit = False
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="HuggingFaceH4/zephyr-7b-beta",
max_seq_length=max_seq_length,
dtype=dtype,
load_in_4bit=load_in_4bit,
#token = ,
)
model = FastLanguageModel.get_peft_model(
model,
r=32,
lora_alpha=64,
lora_dropout=0.05,
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
], # 타겟 모듈
bias="none",
use_gradient_checkpointing="unsloth",
random_state=123,
use_rslora=False,
loftq_config=None,
)
tokenizer.padding_side = "right"
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
dataset_text_field="text",
max_seq_length=max_seq_length,
dataset_num_proc=2,
packing=False,
args=TrainingArguments(
per_device_train_batch_size=20,
gradient_accumulation_steps=2,
warmup_steps=5,
num_train_epochs=3,
max_steps = 1761,
logging_steps = 10,
learning_rate=2e-5,
fp16=not torch.cuda.is_bf16_supported(),
bf16=torch.cuda.is_bf16_supported(),
optim="adamw_8bit",
weight_decay=0.01,
lr_scheduler_type="cosine",
seed=123,
output_dir="outputs",
),
)
trainer.train()
```