---
license: apache-2.0
datasets:
- ajibawa-2023/OpenHermes-2.5-Code-290k
language:
- en
tags:
- code
- finetune
- synthetic data
- text-generation-inference
- conversational
---

**OpenHermes-2.5-Code-290k-13B**

OpenHermes-2.5-Code-290k-13B is a state of the art Llama-2 Fine-tune, which is trained on additional code dataset.
This model is trained on my existing dataset [OpenHermes-2.5-Code-290k](https://huggingface.co/datasets/ajibawa-2023/OpenHermes-2.5-Code-290k). 
This dataset is amalgamation of two datasets. I have used [OpenHermes-2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5) a super quality dataset made avaliable by teknium. Other datset is my own [Code-290k-ShareGPT](https://huggingface.co/datasets/ajibawa-2023/Code-290k-ShareGPT). 
Dataset is in Vicuna/ShareGPT format. There are around **1.29 million** set of conversations. I have cleaned the dataset provided by Teknium and removed metadata such as "source" & "category" etc. This dataset has primarily synthetically generated instruction and chat samples. 

This model has enhanced coding capabilities besides other capabilities such as **Blogging, story generation, Q&A and many more**.

**Training:**

Entire model was trained on 4 x A100 80GB. For 2 epoch, training took **21 Days**. Fschat & DeepSpeed codebase was used for training purpose. This was trained on Llama-2 by Meta.


This is a full fine tuned model. Links for quantized models will be updated soon.


**GPTQ, GGUF, AWQ & Exllama**

GPTQ: TBA

GGUF: TBA

AWQ: TBA

Exllama v2: TBA





**Example Prompt:**
```
This is a conversation with your helpful AI assistant. AI assistant can generate Code in various Programming Languages along with necessary explanation. It can generate Story, Blogs .....

Context
You are a helpful AI assistant.

USER: <prompt>
ASSISTANT:
```

You can modify above Prompt as per your requirement. I have used ShareGPT/Vicuna format v1.1 .

I want to say special Thanks to the Open Source community for helping & guiding me to better understand the AI/Model development.

Thank you for your love & support.

**Example Output**

I will update soon.