Training in progress, step 5000

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:555c567d63047da5c9226f4c383755306d075b9c037698e784e3f654f575b531
+oid sha256:7e90080bf523412654da109eb8a0e3ae8edacc7c025143a1ccd52be304e4b1d5
 size 1262817457

wandb/run-20220129_131141-h6nhqm30/files/output.log

@@ -4763,3 +4763,533 @@ Deleting older checkpoint [checkpoint-2500] due to args.save_total_limit
 Configuration saved in ./checkpoint-4500/config.json
 Model weights saved in ./checkpoint-4500/pytorch_model.bin
 Configuration saved in ./checkpoint-4500/preprocessor_config.json
+Configuration saved in ./preprocessor_config.json
+Deleting older checkpoint [checkpoint-3000] due to args.save_total_limit
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 21%|█████████████████████████████████▎                                                                                                                          | 4599/21520 [3:47:51<9:07:25,  1.94s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|██████████████████████████████████                                                                                                                          | 4700/21520 [3:52:17<9:25:30,  2.02s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 22%|██████████████████████████████████▊                                                                                                                         | 4800/21520 [3:56:42<9:16:50,  2.00s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 23%|███████████████████████████████████▌                                                                                                                        | 4900/21520 [4:01:06<9:15:05,  2.00s/it]
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ 23%|████████████████████████████████████▏                                                                                                                       | 5000/21520 [4:05:30<8:56:08,  1.95s/it]The following columns in the evaluation set  don't have a corresponding argument in `Wav2Vec2ForCTC.forward` and have been ignored: input_length.
+***** Running Evaluation *****
+  Num examples = 1839
+  Batch size = 64
+{'loss': 1.1599, 'learning_rate': 5.779020979020979e-05, 'epoch': 1.16}
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Configuration saved in ./checkpoint-5000/config.json
+{'eval_loss': 0.3355213701725006, 'eval_wer': 0.4330439988027537, 'eval_runtime': 63.7849, 'eval_samples_per_second': 28.831, 'eval_steps_per_second': 0.455, 'epoch': 1.16}
+Model weights saved in ./checkpoint-5000/pytorch_model.bin
+Configuration saved in ./checkpoint-5000/preprocessor_config.json

wandb/run-20220129_131141-h6nhqm30/run-h6nhqm30.wandb

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b91b547c6384170ef589da7269c4676c95497c03a83fe8ae18d858a1ce4be487
-size 31219824
+oid sha256:53a09fb2a08f71ee4b89bfa0dfaf906dc6e5a01b9a4e4ab7901f827b7817f498
+size 35019521