Finish LM train validation on minimal dataset
Browse files
README.md
CHANGED
@@ -25,9 +25,9 @@ tags:
|
|
25 |
- β
Train new BBPE Tokenizer
|
26 |
- β
Test train code on TPUv4 Pods (with model parallel)
|
27 |
- β
Converting test (jax to PyTorch)
|
28 |
-
-
|
29 |
-
- Build Data Shuffler (curriculum learning)
|
30 |
-
- Train 7B Model
|
31 |
- Train 13B Model
|
32 |
- Train 33B Model
|
33 |
- Train 65B Model
|
|
|
25 |
- β
Train new BBPE Tokenizer
|
26 |
- β
Test train code on TPUv4 Pods (with model parallel)
|
27 |
- β
Converting test (jax to PyTorch)
|
28 |
+
- β
LM train validation on minimal dataset (1 sentence 1000 step)
|
29 |
+
- β³ Build Data Shuffler (curriculum learning)
|
30 |
+
- β³ Train 7B Model
|
31 |
- Train 13B Model
|
32 |
- Train 33B Model
|
33 |
- Train 65B Model
|