Update README.md

README.md

@@ -6,4 +6,22 @@ pipeline_tag: text-generation
 ---
 # dolly-v2-12b-q4 Model Card
 
-[dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) converted to GGML format and quantized to 4-bit using https://github.com/NolanoOrg/cformers.
+[dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) converted to GGML format and quantized to 4-bit using https://github.com/NolanoOrg/cformers.
+
+
+## Running the model
+
+[This fork](https://github.com/raymondhs/cformers) has the modification that includes Dolly in the model list.
+
+```python
+from interface import AutoInference as AI
+ai = AI("databricks/dolly-v2-12b")
+prompt_template = """Below is an instruction that describes a task. Write a response that appropriately completes the request.
+### Instruction:
+{instruction}
+### Response:
+"""
+instruction = "Explain to me the difference between nuclear fission and fusion."
+x = ai.generate(prompt_template.format(instruction=instruction), num_tokens_to_generate=100)
+print(x['token_str'])
+```