Update README.md

README.md

@@ -8,7 +8,7 @@ tags:
 - generated_from_trainer
 ---
 
-# gemma-2-27b-it-simpo-beta10-gamma5-lr8e-7
+# gemma-2-27b-it-SimPO-37K Model Card
 
 ## Implementation Details
 We first followed the [SimPO](https://github.com/princeton-nlp/SimPO) framework to apply [On-Policy Preference Data Generation](https://github.com/princeton-nlp/SimPO/tree/main/on_policy_data_gen) on the [HuggingFaceH4/ultrafeedback_binarized](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized) dataset using the [google/gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) model. We then selected prompts where the chosen reward was at least 0.01 higher than the rejected reward, resulting in 37,040 training data points.
@@ -77,14 +77,4 @@ UltraFeedback paper:
   journal={arXiv preprint arXiv:2310.01377},
   year={2023}
 }
-```
-
-ArmoRM paper:
-```
-@article{wang2024interpretable,
-  title={Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts},
-  author={Wang, Haoxiang and Xiong, Wei and Xie, Tengyang and Zhao, Han and Zhang, Tong},
-  journal={arXiv preprint arXiv:2406.12845},
-  year={2024}
-}
 ```