yinsong1986 commited on
Commit
f8e22b4
1 Parent(s): 5cc44e3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -10,7 +10,7 @@ MistralLite is a fine-tuned [Mistral-7B-v0.1](https://huggingface.co/mistralai/M
10
  MistralLite is similar to [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1), and their similarities and differences are summarized below:
11
  |Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
12
  |----------|-------------:|------------:|-----------:|-----------:|
13
- | Mistral-7B-v0.1 | up to 8K tokens | 32K | rope_theta = 10000 | 4096 |
14
  | MistralLite | up to 16K tokens | 32K | **rope_theta = 1000000** | **16384** |
15
 
16
  ## Motivation of Developing MistralLite
 
10
  MistralLite is similar to [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1), and their similarities and differences are summarized below:
11
  |Model|Fine-tuned on long contexts| Max context length| RotaryEmbedding adaptation| Sliding Window Size|
12
  |----------|-------------:|------------:|-----------:|-----------:|
13
+ | Mistral-7B-Instruct-v0.1 | up to 8K tokens | 32K | rope_theta = 10000 | 4096 |
14
  | MistralLite | up to 16K tokens | 32K | **rope_theta = 1000000** | **16384** |
15
 
16
  ## Motivation of Developing MistralLite