---
language:
- en
- ko
license: cc-by-nc-4.0
datasets:
- kyujinpy/KOR-gugugu-platypus-set
base_model:
- LDCC/LDCC-SOLAR-10.7B
pipeline_tag: text-generation
---

# LDCC-SOLAR-gugutypus-10.7B

<img src="logo.png" height=350, width=350>


---


## Model Details

**Model Developers**  
- DongGeon Lee ([oneonlee](https://huggingface.co/oneonlee))

**Model Architecture**  
- **LDCC-SOLAR-gugutypus-10.7B** is a instruction fine-tuned auto-regressive language model, based on the [SOLAR](https://huggingface.co/upstage/SOLAR-10.7B-v1.0) transformer architecture.

**Base Model**
- [LDCC/LDCC-SOLAR-10.7B](https://huggingface.co/LDCC/LDCC-SOLAR-10.7B)

**Training Dataset**
- [kyujinpy/KOR-gugugu-platypus-set](https://huggingface.co/datasets/kyujinpy/KOR-gugugu-platypus-set)


---  


## Model comparisons

- **Ko-LLM leaderboard (2024/03/01)** [[link]](https://huggingface.co/spaces/upstage/open-ko-llm-leaderboard)

| Model                                     | Average     | Ko-ARC | Ko-HellaSwag | Ko-MMLU | Ko-TruthfulQA | Ko-CommonGen V2 |
| ----------------------------------------- | ----------- | ------ | ------------ | ------- | ------------- | --------------- |
| **[oneonlee/KoSOLAR-v0.2-gugutypus-10.7B](https://huggingface.co/oneonlee/KoSOLAR-v0.2-gugutypus-10.7B)** | **51.17**   | 47.78  | 58.29        | 47.27   | 48.31         | 54.19           |
| oneonlee/LDCC-SOLAR-gugutypus-10.7B  | 49.45   | 45.9   | 55.46        | 47.96   | 48.93         | 49              |


<br>

- **(KOR) AI-Harness evaluation** [[link]](https://github.com/Beomi/ko-lm-evaluation-harness)  


|          Tasks          |Version|Filter|n-shot|Metric|Value |   |Stderr|
|-------------------------|-------|------|-----:|------|-----:|---|-----:|
|KMMLU                    |N/A    |none  |     0|acc   |0.3329|±  |0.0794|
|KMMLU                    |N/A    |none  |     5|acc   |0.3969|±  |0.0816|
|KoBEST-HellaSwag         |      0|none  |     0|acc   |0.4260|±  |0.0221|
|KoBEST-HellaSwag         |      0|none  |     5|acc   |0.4260|±  |0.0221|
|KoBEST-BoolQ             |      0|none  |     0|acc   |0.7792|±  |0.0111|
|KoBEST-BoolQ             |      0|none  |     5|acc   |0.8925|±  |0.0083|
|KoBEST-COPA              |      0|none  |     0|acc   |0.6670|±  |0.0149|
|KoBEST-COPA              |      0|none  |     5|acc   |0.7070|±  |0.0144|
|KoBEST-SentiNeg          |      0|none  |     0|acc   |0.7582|±  |0.0215|
|KoBEST-SentiNeg          |      0|none  |     5|acc   |0.9219|±  |0.0135|

<br>

- **(ENG) AI-Harness evaluation** [[link]](https://github.com/EleutherAI/lm-evaluation-harness)

|      Tasks      |Version|Filter|n-shot|Metric|Value |   |Stderr|
|------------------|-------|------|-----:|------|-----:|---|-----:|
|MMLU              |N/A    |none  |     0|acc   |0.5826|±  |0.1432|
|MMLU              |N/A    |none  |     5|acc   |0.6124|±  |0.1275|
|HellaSwag         |      1|none  |     0|acc   |0.6075|±  |0.0049|
|HellaSwag         |      1|none  |     5|acc   |0.6534|±  |0.0047|
|BoolQ             |      2|none  |     0|acc   |0.8737|±  |0.0058|
|BoolQ             |      2|none  |     5|acc   |0.8878|±  |0.0055|
|COPA              |      1|none  |     0|acc   |0.8300|±  |0.0378|
|COPA              |      1|none  |     5|acc   |0.9300|±  |0.0256|
|truthfulqa        |N/A    |none  |     0|acc   |0.4249|±  |0.0023|
|truthfulqa        |N/A    |none  |     5|acc   |  -   |±  |   -  |


---


## Implementation Code

```python
### LDCC-SOLAR-gugutypus
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "oneonlee/LDCC-SOLAR-gugutypus-10.7B"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
tokenizer = AutoTokenizer.from_pretrained(repo)
```