Helw150
commited on
Commit
•
8e9ae91
1
Parent(s):
b5e9bbc
README
Browse files
README.md
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Model Card for Diva Llama 3
|
| 2 |
+
|
| 3 |
+
<!-- Provide a quick summary of what the model is/does. [Optional] -->
|
| 4 |
+
This is an end-to-end Voice Assistant Model which can handle speech and text as inputs. It is trained using distillation loss. More details will be in a paper [COMING SOON]!
|
| 5 |
+
|
| 6 |
+
See also [value-nlp.github.io/DiVA-Demo](value-nlp.github.io/DiVA-Demo).
|
| 7 |
+
|
| 8 |
+
## Table of Contents
|
| 9 |
+
|
| 10 |
+
- [Model Card for DiVA Llama 3](#model-card-for-DiVA-Llama-3)
|
| 11 |
+
- [Table of Contents](#table-of-contents)
|
| 12 |
+
- [Citation](#citation)
|
| 13 |
+
- [Training Details](#training-details)
|
| 14 |
+
- [Training Data](#training-data)
|
| 15 |
+
- [Training Procedure](#training-procedure)
|
| 16 |
+
- [Environmental Impact](#environmental-impact)
|
| 17 |
+
- [Technical Specifications [optional]](#technical-specifications-optional)
|
| 18 |
+
- [Model Architecture and Objective](#model-architecture-and-objective)
|
| 19 |
+
- [Compute Infrastructure](#compute-infrastructure)
|
| 20 |
+
- [Hardware](#hardware)
|
| 21 |
+
- [Software](#software)
|
| 22 |
+
- [Model Card Contact](#model-card-contact)
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
## Citation
|
| 26 |
+
No Publication As of Yet, But If You Use Please Cite the Below
|
| 27 |
+
**BibTeX:**
|
| 28 |
+
|
| 29 |
+
```
|
| 30 |
+
@InProceedings{hewitt2023backpack,
|
| 31 |
+
author = "Held, Will and Zhang, Yanzhe and Ryan, Michael and Shi, Weiyan and Li, Ella and Yang, Diyi",
|
| 32 |
+
title = "Distilling an End-to-End Voice Assistant from Speech Recognition Data",
|
| 33 |
+
year = "2024",
|
| 34 |
+
publisher = "HuggingFace",
|
| 35 |
+
}
|
| 36 |
+
```
|
| 37 |
+
|
| 38 |
+
## Training Details
|
| 39 |
+
|
| 40 |
+
### Training Data
|
| 41 |
+
|
| 42 |
+
<!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
| 43 |
+
|
| 44 |
+
This model was trained on the [CommonVoice](https://huggingface.co/datasets/mozilla-foundation/common_voice_16_1) corpus.
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
### Training Procedure
|
| 48 |
+
|
| 49 |
+
This model was trained for 7k gradient steps with a batch size of 512 Recordings and a linearly decaying learning rate from 5e-5 to zero, with a linear warmup of 70 steps.
|
| 50 |
+
|
| 51 |
+
### Environmental Impact
|
| 52 |
+
|
| 53 |
+
- **Hardware Type:** V4-32 TPU
|
| 54 |
+
- **Hours used:** 8 Hours
|
| 55 |
+
- **Cloud Provider:** Google Cloud.
|
| 56 |
+
- **Compute Region:** US Central C
|
| 57 |
+
|
| 58 |
+
|
| 59 |
+
### Hardware
|
| 60 |
+
|
| 61 |
+
This model was trained on at V4 TPU on Google Cloud.
|
| 62 |
+
|
| 63 |
+
### Software
|
| 64 |
+
|
| 65 |
+
This model was trained with [Levanter](https://github.com/stanford-crfm/levanter)
|
| 66 |
+
|
| 67 |
+
|
| 68 |
+
## Model Card Authors [optional]
|
| 69 |
+
|
| 70 |
+
<!-- This section provides another layer of transparency and accountability. Whose views is this model card representing? How many voices were included in its construction? Etc. -->
|
| 71 |
+
|
| 72 |
+
Will Held
|
| 73 |
+
|
| 74 |
+
## Model Card Contact
|
| 75 |
+
|
| 76 |
+
held@stanford.edu
|
| 77 |
+
|