|
2024-09-04 10:34:23.566 | INFO | __main__:run:117 - Loading configuration from config 0 |
|
2024-09-04 10:34:23.566 | INFO | __main__:run:117 - Loading configuration from config localhost |
|
2024-09-04 10:34:23.566 | INFO | __main__:run:117 - Loading configuration from config 10086 |
|
2024-09-04 10:34:23.566 | INFO | __main__:run:117 - Loading configuration from config 0 |
|
2024-09-04 10:34:23.566 | INFO | __main__:run:117 - Loading configuration from config 1 |
|
2024-09-04 10:34:23.566 | INFO | __main__:run:119 - Loading environment variables |
|
MASTER_ADDR: localhost, |
|
MASTER_PORT: 10086, |
|
WORLD_SIZE: 1, |
|
RANK: 0, |
|
LOCAL_RANK: 0 |
|
2024-09-04 10:34:24.693 | INFO | default_style:save_styles_by_dirs:54 - At least 2 subdirectories are required for generating style vectors with respect to them, found 0. |
|
2024-09-04 10:34:24.694 | INFO | default_style:save_styles_by_dirs:57 - Generating only neutral style vector instead. |
|
2024-09-04 10:34:24.700 | INFO | default_style:save_neutral_vector:28 - Saved mean style vector to model_assets/tsukuyomi |
|
2024-09-04 10:34:24.700 | INFO | default_style:save_neutral_vector:36 - Saved style config to model_assets/tsukuyomi/config.json |
|
2024-09-04 10:34:24.743 | WARNING | style_bert_vits2.models.utils:check_git_hash:247 - /home/sacha/projects/Style-Bert-VITS2/style_bert_vits2/models/utils is not a git repository, therefore hash value comparison will be ignored. |
|
2024-09-04 10:34:24.745 | INFO | data_utils:_filter:69 - Init dataset... |
|
2024-09-04 10:34:24.748 | INFO | data_utils:_filter:84 - skipped: 0, total: 100 |
|
2024-09-04 10:34:24.748 | INFO | data_utils:__init__:348 - Bucket info: [4, 16, 24, 24, 20, 8, 4, 8] |
|
2024-09-04 10:34:24.748 | INFO | data_utils:_filter:69 - Init dataset... |
|
2024-09-04 10:34:24.748 | INFO | data_utils:_filter:84 - skipped: 0, total: 0 |
|
2024-09-04 10:34:24.749 | INFO | __main__:run:274 - Using noise scaled MAS for VITS2 |
|
2024-09-04 10:34:26.560 | WARNING | style_bert_vits2.models.utils.safetensors:load_safetensors:42 - Missing key: enc_p.style_proj.weight |
|
2024-09-04 10:34:26.560 | WARNING | style_bert_vits2.models.utils.safetensors:load_safetensors:42 - Missing key: enc_p.style_proj.bias |
|
2024-09-04 10:34:26.560 | WARNING | style_bert_vits2.models.utils.safetensors:load_safetensors:42 - Missing key: emb_g.weight |
|
2024-09-04 10:34:26.560 | INFO | style_bert_vits2.models.utils.safetensors:load_safetensors:48 - Loaded 'Data/tsukuyomi/models/G_0.safetensors' |
|
2024-09-04 10:34:26.595 | INFO | style_bert_vits2.models.utils.safetensors:load_safetensors:48 - Loaded 'Data/tsukuyomi/models/D_0.safetensors' |
|
2024-09-04 10:34:26.603 | INFO | style_bert_vits2.models.utils.safetensors:load_safetensors:48 - Loaded 'Data/tsukuyomi/models/WD_0.safetensors' |
|
2024-09-04 10:34:26.604 | INFO | __main__:run:492 - Loaded the pretrained models. |
|
2024-09-04 10:34:27.224 | INFO | __main__:run:540 - Start training. |
|
2024-09-04 10:43:24.061 | INFO | __main__:evaluate:1040 - Evaluating ... |
|
2024-09-04 10:43:24.064 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 38 to Data/tsukuyomi/models/G_1000.pth |
|
2024-09-04 10:43:25.069 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 38 to Data/tsukuyomi/models/D_1000.pth |
|
2024-09-04 10:43:25.554 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 38 to Data/tsukuyomi/models/WD_1000.pth |
|
2024-09-04 10:43:25.594 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/G_0.safetensors |
|
2024-09-04 10:43:25.610 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/D_0.safetensors |
|
2024-09-04 10:43:25.610 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/WD_0.safetensors |
|
2024-09-04 10:43:25.615 | INFO | style_bert_vits2.models.utils.safetensors:save_safetensors:89 - Saved safetensors to model_assets/tsukuyomi/tsukuyomi_e38_s1000.safetensors |
|
2024-09-04 10:52:08.506 | INFO | __main__:evaluate:1040 - Evaluating ... |
|
2024-09-04 10:52:08.508 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 75 to Data/tsukuyomi/models/G_2000.pth |
|
2024-09-04 10:52:09.461 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 75 to Data/tsukuyomi/models/D_2000.pth |
|
2024-09-04 10:52:09.939 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 75 to Data/tsukuyomi/models/WD_2000.pth |
|
2024-09-04 10:52:10.024 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/G_1000.pth |
|
2024-09-04 10:52:10.068 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/D_1000.pth |
|
2024-09-04 10:52:10.070 | INFO | style_bert_vits2.models.utils.checkpoints:del_info:173 - Free up space by deleting ckpt Data/tsukuyomi/models/WD_1000.pth |
|
2024-09-04 10:52:10.075 | INFO | style_bert_vits2.models.utils.safetensors:save_safetensors:89 - Saved safetensors to model_assets/tsukuyomi/tsukuyomi_e75_s2000.safetensors |
|
2024-09-04 10:58:11.837 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 100 to Data/tsukuyomi/models/G_2700.pth |
|
2024-09-04 10:58:12.800 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 100 to Data/tsukuyomi/models/D_2700.pth |
|
2024-09-04 10:58:13.281 | INFO | style_bert_vits2.models.utils.checkpoints:save_checkpoint:110 - Saving model and optimizer state at iteration 100 to Data/tsukuyomi/models/WD_2700.pth |
|
2024-09-04 10:58:13.300 | INFO | style_bert_vits2.models.utils.safetensors:save_safetensors:89 - Saved safetensors to model_assets/tsukuyomi/tsukuyomi_e100_s2700.safetensors |
|
|
