asyafiqe commited on
Commit
ce9b2c0
1 Parent(s): 7d0a716

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -6,11 +6,12 @@ language:
6
  - en
7
  - id
8
  ---
9
- ## 🦚Merak-7B-v3-Mini-Orca🐳
10
 
11
  **Merak-7B-v3-Mini-Orca** is Ichsan2895's [Merak-7B-v3](https://huggingface.co/Ichsan2895/Merak-7B-v3) fine-tuned on psmathur's [orca_mini_v1_dataset](https://huggingface.co/datasets/psmathur/orca_mini_v1_dataset). Dataset was machine translated into Bahasa Indonesia with Google Translate.
12
 
13
- [![Axolotl](https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png)](https://github.com/OpenAccess-AI-Collective/axolotl)
 
14
  #### Training details
15
  Merak-7B-v3-Mini-Orca was instruction fine-tuned on 2 x 3090-24GB for 6 hours. [LoRA](https://github.com/microsoft/LoRA), [DeepSpeed ZeRO-2](https://github.com/microsoft/DeepSpeed), and [FlashAttention](https://github.com/Dao-AILab/flash-attention) were implemented during training using [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl).
16
  Hyperparameter | value |
@@ -27,6 +28,7 @@ lora rank | 16 |
27
  lora dropout | 0.05 |
28
  lora target modules | q_proj, v_proj, k_proj, o_proj |
29
  cutoff length | 4096 |
 
30
  #### Training loss
31
  Step |Train Loss
32
  | ------ | ------ |
 
6
  - en
7
  - id
8
  ---
9
+ # 🦚Merak-7B-v3-Mini-Orca🐳
10
 
11
  **Merak-7B-v3-Mini-Orca** is Ichsan2895's [Merak-7B-v3](https://huggingface.co/Ichsan2895/Merak-7B-v3) fine-tuned on psmathur's [orca_mini_v1_dataset](https://huggingface.co/datasets/psmathur/orca_mini_v1_dataset). Dataset was machine translated into Bahasa Indonesia with Google Translate.
12
 
13
+ [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
14
+
15
  #### Training details
16
  Merak-7B-v3-Mini-Orca was instruction fine-tuned on 2 x 3090-24GB for 6 hours. [LoRA](https://github.com/microsoft/LoRA), [DeepSpeed ZeRO-2](https://github.com/microsoft/DeepSpeed), and [FlashAttention](https://github.com/Dao-AILab/flash-attention) were implemented during training using [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl).
17
  Hyperparameter | value |
 
28
  lora dropout | 0.05 |
29
  lora target modules | q_proj, v_proj, k_proj, o_proj |
30
  cutoff length | 4096 |
31
+
32
  #### Training loss
33
  Step |Train Loss
34
  | ------ | ------ |