Model Card: LEVI Whisper Medium Fine-Tuned Model

Model Information

Model Name: levicu/LEVI_whisper_medium
Description: This model is a fine-tuned version of the OpenAI Whisper Medium model, tailored for speech recognition tasks using the LEVI v2 dataset, which consists of classroom audiovisual recording data.
Model Architecture: openai/whisper-medium
Dataset: LEVI_LoFi_v2/TRAIN (per-utterance transcript and 16k WAV audio) - both student and tutor speech were used - manifest: LEVI_LoFi_v2_TRAIN_punc+cased.csv

Training Procedure:
- LoRA Parameter Efficient Fine-tuning technique with the following parameters:
  - r=32
  - lora_alpha=64
  - target_modules=["q_proj", "v_proj"]
  - lora_dropout=0.05
  - bias="none"
- INT8 quantization
- Trained for 6 epochs with a learning rate of 1e-4 and warmup steps of 100 without gradient accumulation.
Evaluation Metrics: Word Error Rate (WER)

Testing Data
- Test Data 1: LoFi Students (LEVI_LoFi_v2_TEST_punc+cased_student)
- Test Data 2: LoFi Tutors (LEVI_LoFi_v2_TEST_punc+cased_tutor)
- Test Data 3: HiFi Students (LEVI_orig11_HiFi_punc+cased_student)
- Test Data 4: HiFi Tutor (LEVI_orig11_HiFi_punc+cased_tutor)
Metric
- Word Error Rate (WER)
Results
- Test Data 1: 44.1%
- Test Data 2: 15.1%
- Test Data 3: 44.2%
- Test Data 4: 15.9%

Usage: The model can be used for speech recognition tasks. Inputs should be audio files, and the model outputs transcriptions.

Limitations: None provided.
Ethical Considerations: Consider the ethical implications of using this model, particularly in scenarios involving sensitive or private information.

Contact: For questions, feedback, or support regarding the model, please contact roso8920@colorado.edu or nich7312@colorado.edu.