hydra_train.log

[2024-06-18 17:43:50,448][fairseq_cli.train][INFO] - {'_name': None, 'common': {'_name': None, 'no_progress_bar': False, 'log_interval': 200, 'log_format': 'json', 'log_file': None, 'tensorboard_logdir': 'tblog', 'wandb_project': 'AVSP-LLM', 'azureml_logging': False, 'seed': 1337, 'cpu': False, 'tpu': False, 'bf16': False, 'memory_efficient_bf16': False, 'fp16': True, 'memory_efficient_fp16': False, 'fp16_no_flatten_grads': False, 'fp16_init_scale': 128, 'fp16_scale_window': None, 'fp16_scale_tolerance': 0.0, 'on_cpu_convert_precision': False, 'min_loss_scale': 0.0001, 'threshold_loss_scale': None, 'amp': False, 'amp_batch_retries': 2, 'amp_init_scale': 128, 'amp_scale_window': None, 'user_dir': '/home/theodore/Projects/VSP-LLM/src', 'empty_cache_freq': 0, 'all_gather_list_size': 16384, 'model_parallel_size': 1, 'quantization_config_path': None, 'profile': False, 'reset_logging': False, 'suppress_crashes': False, 'use_plasma_view': False, 'plasma_path': '/tmp/plasma'}, 'common_eval': {'_name': None, 'path': None, 'post_process': None, 'quiet': False, 'model_overrides': '{}', 'results_path': None}, 'distributed_training': {'_name': None, 'distributed_world_size': 1, 'distributed_num_procs': 1, 'distributed_rank': 0, 'distributed_backend': 'nccl', 'distributed_init_method': None, 'distributed_port': -1, 'device_id': 0, 'distributed_no_spawn': False, 'ddp_backend': 'no_c10d', 'ddp_comm_hook': 'none', 'bucket_cap_mb': 25, 'fix_batches_to_gpus': False, 'find_unused_parameters': True, 'fast_stat_sync': False, 'heartbeat_timeout': -1, 'broadcast_buffers': False, 'slowmo_momentum': None, 'slowmo_algorithm': 'LocalSGD', 'localsgd_frequency': 3, 'nprocs_per_node': 1, 'pipeline_model_parallel': False, 'pipeline_balance': None, 'pipeline_devices': None, 'pipeline_chunks': 0, 'pipeline_encoder_balance': None, 'pipeline_encoder_devices': None, 'pipeline_decoder_balance': None, 'pipeline_decoder_devices': None, 'pipeline_checkpoint': 'never', 'zero_sharding': 'none', 'fp16': True, 'memory_efficient_fp16': False, 'tpu': False, 'no_reshard_after_forward': False, 'fp32_reduce_scatter': False, 'cpu_offload': False, 'use_sharded_state': False}, 'dataset': {'_name': None, 'num_workers': 0, 'skip_invalid_size_inputs_valid_test': False, 'max_tokens': None, 'batch_size': 1, 'required_batch_size_multiple': 8, 'required_seq_len_multiple': 1, 'dataset_impl': None, 'data_buffer_size': 10, 'train_subset': 'train', 'valid_subset': 'valid', 'combine_valid_subsets': None, 'ignore_unused_valid_subsets': False, 'validate_interval': 1, 'validate_interval_updates': 0, 'validate_after_updates': 0, 'fixed_validation_seed': None, 'disable_validation': False, 'max_tokens_valid': None, 'batch_size_valid': 1, 'max_valid_steps': None, 'curriculum': 0, 'gen_subset': 'test', 'num_shards': 1, 'shard_id': 0}, 'optimization': {'_name': None, 'max_epoch': 0, 'max_update': 30000, 'stop_time_hours': 0.0, 'clip_norm': 0.0, 'sentence_avg': True, 'update_freq': [8], 'lr': [0.0005], 'stop_min_lr': -1.0, 'use_bmuf': False}, 'checkpoint': {'_name': None, 'save_dir': 'checkpoints', 'restore_file': 'checkpoint_last.pt', 'finetune_from_model': None, 'reset_dataloader': False, 'reset_lr_scheduler': False, 'reset_meters': False, 'reset_optimizer': False, 'optimizer_overrides': '{}', 'save_interval': 1, 'save_interval_updates': 2500, 'keep_interval_updates': 1, 'keep_interval_updates_pattern': -1, 'keep_last_epochs': -1, 'keep_best_checkpoints': -1, 'no_save': False, 'no_epoch_checkpoints': True, 'no_last_checkpoints': False, 'no_save_optimizer_state': False, 'best_checkpoint_metric': 'accuracy', 'maximize_best_checkpoint_metric': True, 'patience': -1, 'checkpoint_suffix': '', 'checkpoint_shard_count': 1, 'load_checkpoint_on_all_dp_ranks': False, 'write_checkpoints_asynchronously': False, 'model_parallel_size': 1}, 'bmuf': {'_name': None, 'block_lr': 1.0, 'block_momentum': 0.875, 'global_sync_iter': 50, 'warmup_iterations': 500, 'use_nbm': False, 'average_sync': False, 'distributed_world_size': 1}, 'generation': {'_name': None, 'beam': 5, 'nbest': 1, 'max_len_a': 0.0, 'max_len_b': 200, 'min_len': 1, 'match_source_len': False, 'unnormalized': False, 'no_early_stop': False, 'no_beamable_mm': False, 'lenpen': 1.0, 'unkpen': 0.0, 'replace_unk': None, 'sacrebleu': False, 'score_reference': False, 'prefix_size': 0, 'no_repeat_ngram_size': 0, 'sampling': False, 'sampling_topk': -1, 'sampling_topp': -1.0, 'constraints': None, 'temperature': 1.0, 'diverse_beam_groups': -1, 'diverse_beam_strength': 0.5, 'diversity_rate': -1.0, 'print_alignment': None, 'print_step': False, 'lm_path': None, 'lm_weight': 0.0, 'iter_decode_eos_penalty': 0.0, 'iter_decode_max_iter': 10, 'iter_decode_force_max_iter': False, 'iter_decode_with_beam': 1, 'iter_decode_with_external_reranker': False, 'retain_iter_history': False, 'retain_dropout': False, 'retain_dropout_modules': None, 'decoding_format': None, 'no_seed_provided': False}, 'eval_lm': {'_name': None, 'output_word_probs': False, 'output_word_stats': False, 'context_window': 0, 'softmax_batch': 9223372036854775807}, 'interactive': {'_name': None, 'buffer_size': 0, 'input': '-'}, 'model': {'_name': 'vsp_llm', 'w2v_path': '/home/theodore/Projects/VSP-LLM/checkpoints/large_vox_iter5.pt', 'llm_ckpt_path': 'vilm/vinallama-2.7b', 'apply_mask': False, 'mask_selection': 'static', 'mask_length': 10, 'mask_other': 0, 'mask_prob': 0.75, 'mask_channel_selection': 'static', 'mask_channel_length': 64, 'mask_channel_other': 0, 'mask_channel_prob': 0.5, 'layerdrop': 0.1, 'dropout': 0.0, 'activation_dropout': 0.1, 'attention_dropout': 0.0, 'feature_grad_mult': 1.0, 'encoder_embed_dim': 1024, 'decoder_embed_dim': 4096, 'freeze_finetune_updates': 18000}, 'task': {'_name': 'vsp_llm_training', 'is_s2s': True, 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1', 'label_dir': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1', 'normalize': True, 'labels': ['wrd'], 'single_target': True, 'fine_tuning': True, 'stack_order_audio': 4, 'max_sample_size': 500, 'modalities': ['video', 'audio'], 'image_aug': True, 'pad_audio': True, 'random_crop': False, 'llm_ckpt_path': 'vilm/vinallama-2.7b'}, 'criterion': {'_name': 'decoder_only_language_modeling_loss', 'report_accuracy': True, 'label_smoothing': 0.1}, 'optimizer': {'_name': 'adam', 'adam_betas': '(0.9,0.98)', 'adam_eps': 1e-08, 'weight_decay': 0.0, 'use_old_adam': False, 'tpu': False, 'lr': [0.0005]}, 'lr_scheduler': {'_name': 'tri_stage', 'warmup_steps': 10000, 'hold_steps': 0, 'decay_steps': 20000, 'phase_ratio': None, 'init_lr_scale': 0.01, 'final_lr_scale': 0.05, 'max_update': 30000, 'lr': [0.0005]}, 'scoring': None, 'bpe': None, 'tokenizer': None, 'job_logging_cfg': {'version': 1, 'formatters': {'simple': {'format': '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'}}, 'handlers': {'console': {'class': 'logging.StreamHandler', 'formatter': 'simple', 'stream': 'ext://sys.stdout'}, 'file': {'class': 'logging.FileHandler', 'formatter': 'simple', 'filename': 'hydra_train.log'}}, 'root': {'level': 'INFO', 'handlers': ['console', 'file']}, 'disable_existing_loggers': False}}
[2024-06-18 17:43:50,452][src.vsp_llm_training][INFO] - current directory is /home/theodore/Projects/VSP-LLM/experiments/ViAVSP-LLM_v1.1.1
[2024-06-18 17:43:50,452][src.vsp_llm_training][INFO] - AVHubertPretrainingTask Config {'_name': 'vsp_llm_training', 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1', 'labels': ['wrd'], 'label_dir': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1', 'label_rate': -1, 'sample_rate': 16000, 'llm_ckpt_path': 'vilm/vinallama-2.7b', 'normalize': True, 'enable_padding': False, 'max_sample_size': 500, 'min_sample_size': None, 'max_trim_sample_size': '${task.max_sample_size}', 'single_target': True, 'random_crop': False, 'pad_audio': True, 'pdb': False, 'stack_order_audio': 4, 'skip_verify': False, 'image_aug': True, 'image_crop_size': 88, 'image_mean': 0.421, 'image_std': 0.165, 'modalities': ['video', 'audio'], 'is_s2s': True, 'tokenizer_bpe_name': None, 'tokenizer_bpe_model': None, 'noise_wav': None, 'noise_prob': 0.0, 'noise_snr': '0', 'noise_num': 1, 'fine_tuning': True}
[2024-06-18 17:43:52,763][src.hubert_pretraining][INFO] - current directory is /home/theodore/Projects/VSP-LLM/experiments/ViAVSP-LLM_v1.1.1
[2024-06-18 17:43:52,763][src.hubert_pretraining][INFO] - AVHubertPretrainingTask Config {'_name': 'av_hubert_pretraining', 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1', 'labels': ['km'], 'label_dir': '/checkpoint/bshi/data/lrs3//video/hubert/stitch-iters/envox-iter4-l12c2000/', 'label_rate': 25, 'sample_rate': 25, 'normalize': True, 'enable_padding': False, 'max_sample_size': 2000, 'min_sample_size': 5, 'max_trim_sample_size': 400, 'single_target': False, 'random_crop': True, 'pad_audio': False, 'pdb': False, 'stack_order_audio': 4, 'skip_verify': False, 'image_aug': True, 'image_crop_size': 88, 'image_mean': 0.421, 'image_std': 0.165, 'modalities': ['audio', 'video'], 'is_s2s': False, 'tokenizer_bpe_name': None, 'tokenizer_bpe_model': None, 'noise_wav': None, 'noise_prob': 0.0, 'noise_snr': '0', 'noise_num': 1, 'fine_tuning': False}
[2024-06-18 17:43:52,769][src.hubert][INFO] - HubertModel Config: {'_name': 'av_hubert', 'label_rate': 25, 'input_modality': '${task.input_modality}', 'extractor_mode': default, 'encoder_layers': 24, 'encoder_embed_dim': 1024, 'encoder_ffn_embed_dim': 4096, 'encoder_attention_heads': 16, 'activation_fn': gelu, 'dropout': 0.0, 'attention_dropout': 0.0, 'activation_dropout': 0.1, 'encoder_layerdrop': 0.1, 'dropout_input': 0.0, 'dropout_features': 0.1, 'final_dim': 256, 'untie_final_proj': True, 'layer_norm_first': True, 'conv_feature_layers': '[(512,10,5)] + [(512,3,2)] * 4 + [(512,2,2)] * 2', 'conv_bias': False, 'logit_temp': 0.1, 'target_glu': False, 'feature_grad_mult': 1.0, 'mask_length_audio': 10, 'mask_prob_audio': 0.8, 'mask_length_image': 5, 'mask_prob_image': 0.3, 'mask_selection': static, 'mask_other': 0.0, 'no_mask_overlap': False, 'mask_min_space': 1, 'mask_channel_length': 10, 'mask_channel_prob': 0.0, 'mask_channel_selection': static, 'mask_channel_other': 0.0, 'no_mask_channel_overlap': False, 'mask_channel_min_space': 1, 'conv_pos': 128, 'conv_pos_groups': 16, 'latent_temp': [2.0, 0.5, 0.999995], 'skip_masked': False, 'skip_nomask': False, 'resnet_relu_type': 'prelu', 'resnet_weights': None, 'sim_type': 'cosine', 'sub_encoder_layers': 0, 'audio_feat_dim': 104, 'modality_dropout': 0.5, 'audio_dropout': 0.5, 'modality_fuse': 'concat', 'selection_type': 'same_seq', 'masking_type': 'input', 'decoder_embed_dim': 768, 'decoder_ffn_embed_dim': 3072, 'decoder_layers': 6, 'decoder_layerdrop': 0.0, 'decoder_attention_heads': 4, 'decoder_learned_pos': False, 'decoder_normalize_before': False, 'no_token_positional_embeddings': False, 'decoder_dropout': 0.1, 'decoder_attention_dropout': 0.1, 'decoder_activation_dropout': 0.0, 'max_target_positions': 2048, 'share_decoder_input_output_embed': False, 'no_scale_embedding': True}
[2024-06-18 17:44:04,126][fairseq_cli.train][INFO] - avhubert_llm_seq2seq_cluster_count(
  (encoder): HubertEncoderWrapper(
    (w2v_model): AVHubertModel(
      (feature_extractor_audio): SubModel(
        (proj): Linear(in_features=104, out_features=1024, bias=True)
      )
      (feature_extractor_video): SubModel(
        (resnet): ResEncoder(
          (frontend3D): Sequential(
            (0): Conv3d(1, 64, kernel_size=(5, 7, 7), stride=(1, 2, 2), padding=(2, 3, 3), bias=False)
            (1): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (2): PReLU(num_parameters=64)
            (3): MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), dilation=1, ceil_mode=False)
          )
          (trunk): ResNet(
            (layer1): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=64)
                (relu2): PReLU(num_parameters=64)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
              (1): BasicBlock(
                (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=64)
                (relu2): PReLU(num_parameters=64)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer2): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=128)
                (relu2): PReLU(num_parameters=128)
                (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=128)
                (relu2): PReLU(num_parameters=128)
                (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer3): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=256)
                (relu2): PReLU(num_parameters=256)
                (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=256)
                (relu2): PReLU(num_parameters=256)
                (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer4): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=512)
                (relu2): PReLU(num_parameters=512)
                (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=512)
                (relu2): PReLU(num_parameters=512)
                (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (avgpool): AdaptiveAvgPool2d(output_size=1)
          )
        )
        (proj): Linear(in_features=512, out_features=1024, bias=True)
      )
      (post_extract_proj): Linear(in_features=2048, out_features=1024, bias=True)
      (dropout_input): Dropout(p=0.0, inplace=False)
      (dropout_features): Dropout(p=0.1, inplace=False)
      (encoder): TransformerEncoder(
        (pos_conv): Sequential(
          (0): Conv1d(1024, 1024, kernel_size=(128,), stride=(1,), padding=(64,), groups=16)
          (1): SamePad()
          (2): GELU(approximate='none')
        )
        (layers): ModuleList(
          (0-23): 24 x TransformerSentenceEncoderLayer(
            (self_attn): MultiheadAttention(
              (dropout_module): FairseqDropout()
              (k_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (v_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (q_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (out_proj): Linear(in_features=1024, out_features=1024, bias=True)
            )
            (dropout1): Dropout(p=0.0, inplace=False)
            (dropout2): Dropout(p=0.1, inplace=False)
            (dropout3): Dropout(p=0.0, inplace=False)
            (self_attn_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
            (fc1): Linear(in_features=1024, out_features=4096, bias=True)
            (fc2): Linear(in_features=4096, out_features=1024, bias=True)
            (final_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
          )
        )
        (layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
      )
      (layer_norm): LayerNorm((2048,), eps=1e-05, elementwise_affine=True)
      (final_proj): None
    )
  )
  (decoder): PeftModelForCausalLM(
    (base_model): LoraModel(
      (model): LlamaForCausalLM(
        (model): LlamaModel(
          (embed_tokens): Embedding(46304, 2560, padding_idx=0)
          (layers): ModuleList(
            (0-31): 32 x LlamaDecoderLayer(
              (self_attn): LlamaSdpaAttention(
                (q_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (k_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (v_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (o_proj): Linear4bit(in_features=2560, out_features=2560, bias=False)
                (rotary_emb): LlamaRotaryEmbedding()
              )
              (mlp): LlamaMLP(
                (gate_proj): Linear4bit(in_features=2560, out_features=6912, bias=False)
                (up_proj): Linear4bit(in_features=2560, out_features=6912, bias=False)
                (down_proj): Linear4bit(in_features=6912, out_features=2560, bias=False)
                (act_fn): SiLU()
              )
              (input_layernorm): LlamaRMSNorm()
              (post_attention_layernorm): LlamaRMSNorm()
            )
          )
          (norm): LlamaRMSNorm()
        )
        (lm_head): Linear(in_features=2560, out_features=46304, bias=False)
      )
    )
  )
  (avfeat_to_llm): Linear(in_features=1024, out_features=2560, bias=True)
)
[2024-06-18 17:44:04,134][fairseq_cli.train][INFO] - task: VSP_LLM_TrainingTask
[2024-06-18 17:44:04,134][fairseq_cli.train][INFO] - model: avhubert_llm_seq2seq_cluster_count
[2024-06-18 17:44:04,134][fairseq_cli.train][INFO] - criterion: decoder_only_language_modeling_loss
[2024-06-18 17:44:04,139][fairseq_cli.train][INFO] - num. shared model params: 1,841,644,264 (num. trained: 335,624,424)
[2024-06-18 17:44:04,143][fairseq_cli.train][INFO] - num. expert model params: 0 (num. trained: 0)
[2024-06-18 17:44:04,144][src.vsp_llm_training][INFO] - Using tokenizer
[2024-06-18 17:44:04,201][src.vsp_llm_dataset][INFO] - max_keep=500, min_keep=None, loaded 23990, skipped 0 short and 0 long and 0 unaligned, longest-loaded=76, shortest-loaded=76
[2024-06-18 17:44:04,603][src.vsp_llm_dataset][INFO] - /home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1/valid.wrd is sequence label. skipped
[2024-06-18 17:44:04,603][src.vsp_llm_dataset][INFO] - image transform: Compose(
    Normalize(mean=0.0, std=255.0)
    <src.utils_vsp_llm.CenterCrop object at 0x7729e0e07220>
    Normalize(mean=0.421, std=0.165)
)
[2024-06-18 17:44:04,603][src.vsp_llm_dataset][INFO] - pad_audio=True, random_crop=False, normalize=True, max_sample_size=500, seqs2seq data=True,
[2024-06-18 17:44:04,603][src.vsp_llm_dataset][INFO] - Noise wav: None->0 wav, Prob: 0.0, SNR: 0, Number of mixture: 1
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.0.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.0.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.1.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.1.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.downsample.0.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.1.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.1.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.downsample.0.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.1.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.1.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.downsample.0.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.1.conv1.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.1.conv2.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,116][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.o_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.gate_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.up_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.down_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.o_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.gate_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.up_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.down_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.o_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.gate_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.up_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.down_proj.bias
[2024-06-18 17:44:07,117][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.o_proj.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.gate_proj.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.up_proj.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.down_proj.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,118][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.o_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.gate_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.up_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.down_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.o_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.gate_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.up_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.down_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.o_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.gate_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.up_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.down_proj.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,119][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.o_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.gate_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.up_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.down_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.o_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.gate_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.up_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.down_proj.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,120][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.o_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.gate_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.up_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.down_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.o_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.gate_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.up_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.down_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.o_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.gate_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.up_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.down_proj.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,121][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.o_proj.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.gate_proj.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.up_proj.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.down_proj.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.o_proj.bias
[2024-06-18 17:44:07,122][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.gate_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.up_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.down_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.o_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.gate_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.up_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.down_proj.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,124][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.o_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.gate_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.up_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.down_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.o_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.gate_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.up_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.down_proj.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,126][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.o_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.gate_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.up_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.down_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.o_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.gate_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.up_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.down_proj.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,127][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.o_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.gate_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.up_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.down_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.o_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.gate_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.up_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.down_proj.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,128][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.o_proj.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.gate_proj.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.up_proj.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.down_proj.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,129][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.o_proj.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.gate_proj.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.up_proj.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.down_proj.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,130][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.o_proj.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.gate_proj.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.up_proj.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.down_proj.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,131][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.o_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.gate_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.up_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.down_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.o_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.gate_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.up_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.down_proj.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,132][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.o_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.gate_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.up_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.down_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.o_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.gate_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.up_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.down_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.o_proj.bias
[2024-06-18 17:44:07,133][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.gate_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.up_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.down_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.o_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.gate_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.up_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.down_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.o_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.gate_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.up_proj.bias
[2024-06-18 17:44:07,134][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.down_proj.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.base_layer.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.lora_A.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.lora_B.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.base_layer.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.lora_A.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.lora_B.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.base_layer.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.lora_A.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.lora_B.default.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.o_proj.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.gate_proj.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.up_proj.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.down_proj.bias
[2024-06-18 17:44:07,135][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.lm_head.bias
[2024-06-18 17:44:07,135][fairseq.utils][INFO] - ***********************CUDA enviroments for all 1 workers***********************
[2024-06-18 17:44:07,135][fairseq.utils][INFO] - rank   0: capabilities =  8.6  ; total memory = 15.729 GB ; name = NVIDIA RTX A4000                        
[2024-06-18 17:44:07,135][fairseq.utils][INFO] - ***********************CUDA enviroments for all 1 workers***********************
[2024-06-18 17:44:07,136][fairseq_cli.train][INFO] - training on 1 devices (GPUs/TPUs)
[2024-06-18 17:44:07,136][fairseq_cli.train][INFO] - max tokens per device = None and max sentences per device = 1
[2024-06-18 17:44:07,136][fairseq.trainer][INFO] - Preparing to load checkpoint checkpoints/checkpoint_last.pt
[2024-06-18 17:44:07,136][fairseq.trainer][INFO] - No existing checkpoint found checkpoints/checkpoint_last.pt
[2024-06-18 17:44:07,136][fairseq.trainer][INFO] - loading train data for epoch 1
[2024-06-18 17:44:07,136][src.vsp_llm_training][INFO] - Using tokenizer
[2024-06-18 17:44:07,406][src.vsp_llm_dataset][INFO] - max_keep=500, min_keep=None, loaded 120686, skipped 0 short and 0 long and 0 unaligned, longest-loaded=76, shortest-loaded=73
[2024-06-18 17:44:08,457][src.vsp_llm_dataset][INFO] - /home/theodore/Projects/VSP-LLM/data/processed/vasr/100h/1_1_1/train.wrd is sequence label. skipped
[2024-06-18 17:44:08,457][src.vsp_llm_dataset][INFO] - image transform: Compose(
    Normalize(mean=0.0, std=255.0)
    RandomCrop(size=(88, 88))
    <src.utils_vsp_llm.HorizontalFlip object at 0x7729e0f68df0>
    Normalize(mean=0.421, std=0.165)
)
[2024-06-18 17:44:08,457][src.vsp_llm_dataset][INFO] - pad_audio=True, random_crop=False, normalize=True, max_sample_size=500, seqs2seq data=True,
[2024-06-18 17:44:08,457][src.vsp_llm_dataset][INFO] - Noise wav: None->0 wav, Prob: 0.0, SNR: 0, Number of mixture: 1
[2024-06-18 17:44:14,376][fairseq.trainer][INFO] - begin training epoch 1
[2024-06-18 17:44:14,376][fairseq_cli.train][INFO] - Start iterating over samples
[2024-06-18 17:50:09,437][train_inner][INFO] - {"epoch": 1, "update": 0.013, "loss": "7.658", "ntokens": "126.725", "acc_total": "126.725", "n_correct": "17.97", "wer_total": "126.725", "n_error": "108.66", "ppl": "201.91", "accuracy": "14.18", "wer": "85.745", "wps": "71.6", "ups": "0.57", "wpb": "126.7", "bsz": "8", "num_updates": "200", "lr": "1.49e-05", "gnorm": "8.877", "loss_scale": "128", "train_wall": "354", "gb_free": "7.1", "wall": "362"}
[2024-06-18 17:56:03,748][train_inner][INFO] - {"epoch": 1, "update": 0.027, "loss": "6.202", "ntokens": "126.93", "acc_total": "126.93", "n_correct": "25.685", "wer_total": "126.93", "n_error": "101.045", "ppl": "73.6", "accuracy": "20.236", "wer": "79.607", "wps": "71.6", "ups": "0.56", "wpb": "126.9", "bsz": "8", "num_updates": "400", "lr": "2.48e-05", "gnorm": "3.688", "loss_scale": "128", "train_wall": "353", "gb_free": "7.1", "wall": "717"}
[2024-06-18 18:01:57,921][train_inner][INFO] - {"epoch": 1, "update": 0.04, "loss": "6.074", "ntokens": "127.015", "acc_total": "127.015", "n_correct": "28.625", "wer_total": "127.015", "n_error": "98.11", "ppl": "67.36", "accuracy": "22.537", "wer": "77.243", "wps": "71.7", "ups": "0.56", "wpb": "127", "bsz": "8", "num_updates": "600", "lr": "3.47e-05", "gnorm": "3.968", "loss_scale": "128", "train_wall": "353", "gb_free": "7.1", "wall": "1071"}
[2024-06-18 18:07:28,491][train_inner][INFO] - {"epoch": 1, "update": 0.053, "loss": "5.868", "ntokens": "126.865", "acc_total": "126.865", "n_correct": "30.71", "wer_total": "126.865", "n_error": "95.915", "ppl": "58.39", "accuracy": "24.207", "wer": "75.604", "wps": "76.8", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "800", "lr": "4.46e-05", "gnorm": "4.099", "loss_scale": "128", "train_wall": "330", "gb_free": "7.1", "wall": "1401"}
[2024-06-18 18:12:58,645][train_inner][INFO] - {"epoch": 1, "update": 0.066, "loss": "5.934", "ntokens": "127.025", "acc_total": "127.025", "n_correct": "30.5", "wer_total": "127.025", "n_error": "96.28", "ppl": "61.16", "accuracy": "24.011", "wer": "75.796", "wps": "76.9", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "1000", "lr": "5.45e-05", "gnorm": "3.781", "loss_scale": "128", "train_wall": "329", "gb_free": "7.1", "wall": "1732"}
[2024-06-18 18:18:28,589][train_inner][INFO] - {"epoch": 1, "update": 0.08, "loss": "5.883", "ntokens": "127.095", "acc_total": "127.095", "n_correct": "30.925", "wer_total": "127.095", "n_error": "95.94", "ppl": "59.03", "accuracy": "24.332", "wer": "75.487", "wps": "77", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "1200", "lr": "6.44e-05", "gnorm": "3.581", "loss_scale": "128", "train_wall": "329", "gb_free": "7.1", "wall": "2061"}
[2024-06-18 18:23:58,698][train_inner][INFO] - {"epoch": 1, "update": 0.093, "loss": "5.721", "ntokens": "127.62", "acc_total": "127.62", "n_correct": "32.235", "wer_total": "127.62", "n_error": "95.17", "ppl": "52.73", "accuracy": "25.259", "wer": "74.573", "wps": "77.3", "ups": "0.61", "wpb": "127.6", "bsz": "8", "num_updates": "1400", "lr": "7.43e-05", "gnorm": "3.386", "loss_scale": "128", "train_wall": "329", "gb_free": "7.1", "wall": "2392"}
[2024-06-18 18:29:28,836][train_inner][INFO] - {"epoch": 1, "update": 0.106, "loss": "5.737", "ntokens": "127.41", "acc_total": "127.41", "n_correct": "32.745", "wer_total": "127.41", "n_error": "94.39", "ppl": "53.33", "accuracy": "25.7", "wer": "74.084", "wps": "77.2", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "1600", "lr": "8.42e-05", "gnorm": "3.169", "loss_scale": "128", "train_wall": "329", "gb_free": "7.1", "wall": "2722"}
[2024-06-18 18:34:59,046][train_inner][INFO] - {"epoch": 1, "update": 0.119, "loss": "5.774", "ntokens": "126.56", "acc_total": "126.56", "n_correct": "32.23", "wer_total": "126.56", "n_error": "94.1", "ppl": "54.72", "accuracy": "25.466", "wer": "74.352", "wps": "76.7", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "1800", "lr": "9.41e-05", "gnorm": "2.996", "loss_scale": "128", "train_wall": "330", "gb_free": "7.1", "wall": "3052"}
[2024-06-18 18:40:29,204][train_inner][INFO] - {"epoch": 1, "update": 0.133, "loss": "5.673", "ntokens": "126.875", "acc_total": "126.875", "n_correct": "33.43", "wer_total": "126.875", "n_error": "93.225", "ppl": "51.01", "accuracy": "26.349", "wer": "73.478", "wps": "76.9", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "2000", "lr": "0.000104", "gnorm": "2.848", "loss_scale": "128", "train_wall": "329", "gb_free": "7.1", "wall": "3382"}
[2024-06-18 18:45:59,483][train_inner][INFO] - {"epoch": 1, "update": 0.146, "loss": "5.586", "ntokens": "128.2", "acc_total": "128.2", "n_correct": "35.22", "wer_total": "128.2", "n_error": "92.69", "ppl": "48.05", "accuracy": "27.473", "wer": "72.301", "wps": "77.6", "ups": "0.61", "wpb": "128.2", "bsz": "8", "num_updates": "2200", "lr": "0.0001139", "gnorm": "2.817", "loss_scale": "256", "train_wall": "330", "gb_free": "7.1", "wall": "3712"}
[2024-06-18 18:51:29,647][train_inner][INFO] - {"epoch": 1, "update": 0.159, "loss": "5.479", "ntokens": "127.775", "acc_total": "127.775", "n_correct": "37.435", "wer_total": "127.775", "n_error": "90.125", "ppl": "44.61", "accuracy": "29.298", "wer": "70.534", "wps": "77.4", "ups": "0.61", "wpb": "127.8", "bsz": "8", "num_updates": "2400", "lr": "0.0001238", "gnorm": "2.956", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "4043"}
[2024-06-18 18:54:14,690][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-18 19:36:16,901][valid][INFO] - {"epoch": 1, "valid_loss": "5.245", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "5.71805", "valid_wer_total": "18.1585", "valid_n_error": "12.4164", "valid_ppl": "37.92", "valid_accuracy": "31.49", "valid_wer": "68.378", "valid_wps": "172.7", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "2500"}
[2024-06-18 19:36:16,901][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 2500 updates
[2024-06-18 19:36:16,902][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_2500.pt
[2024-06-18 19:36:20,000][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_2500.pt
[2024-06-18 19:36:22,782][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_2500.pt (epoch 1 @ 2500 updates, score 31.49) (writing took 5.880521540995687 seconds)
[2024-06-18 19:39:07,354][train_inner][INFO] - {"epoch": 1, "update": 0.172, "loss": "5.283", "ntokens": "126.92", "acc_total": "126.92", "n_correct": "39.92", "wer_total": "126.92", "n_error": "86.745", "ppl": "38.93", "accuracy": "31.453", "wer": "68.346", "wps": "8.9", "ups": "0.07", "wpb": "126.9", "bsz": "8", "num_updates": "2600", "lr": "0.0001337", "gnorm": "3.331", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "6900"}
[2024-06-18 19:44:37,299][train_inner][INFO] - {"epoch": 1, "update": 0.186, "loss": "5.131", "ntokens": "125.685", "acc_total": "125.685", "n_correct": "42.37", "wer_total": "125.685", "n_error": "83.045", "ppl": "35.03", "accuracy": "33.711", "wer": "66.074", "wps": "76.2", "ups": "0.61", "wpb": "125.7", "bsz": "8", "num_updates": "2800", "lr": "0.0001436", "gnorm": "3.796", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "7230"}
[2024-06-18 19:50:07,364][train_inner][INFO] - {"epoch": 1, "update": 0.199, "loss": "4.897", "ntokens": "127.19", "acc_total": "127.19", "n_correct": "46.53", "wer_total": "127.19", "n_error": "80.375", "ppl": "29.79", "accuracy": "36.583", "wer": "63.193", "wps": "77.1", "ups": "0.61", "wpb": "127.2", "bsz": "8", "num_updates": "3000", "lr": "0.0001535", "gnorm": "4.035", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "7560"}
[2024-06-18 19:55:37,312][train_inner][INFO] - {"epoch": 1, "update": 0.212, "loss": "4.623", "ntokens": "126.535", "acc_total": "126.535", "n_correct": "50.29", "wer_total": "126.535", "n_error": "76.04", "ppl": "24.64", "accuracy": "39.744", "wer": "60.094", "wps": "76.7", "ups": "0.61", "wpb": "126.5", "bsz": "8", "num_updates": "3200", "lr": "0.0001634", "gnorm": "4.205", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "7890"}
[2024-06-18 20:01:07,340][train_inner][INFO] - {"epoch": 1, "update": 0.225, "loss": "4.457", "ntokens": "126.53", "acc_total": "126.53", "n_correct": "53.1", "wer_total": "126.53", "n_error": "73.245", "ppl": "21.96", "accuracy": "41.966", "wer": "57.887", "wps": "76.7", "ups": "0.61", "wpb": "126.5", "bsz": "8", "num_updates": "3400", "lr": "0.0001733", "gnorm": "4.316", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "8220"}
[2024-06-18 20:06:37,213][train_inner][INFO] - {"epoch": 1, "update": 0.239, "loss": "4.217", "ntokens": "127.025", "acc_total": "127.025", "n_correct": "55.67", "wer_total": "127.025", "n_error": "71.165", "ppl": "18.6", "accuracy": "43.826", "wer": "56.024", "wps": "77", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "3600", "lr": "0.0001832", "gnorm": "4.407", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "8550"}
[2024-06-18 20:12:07,108][train_inner][INFO] - {"epoch": 1, "update": 0.252, "loss": "4.058", "ntokens": "127.35", "acc_total": "127.35", "n_correct": "57.98", "wer_total": "127.35", "n_error": "69.195", "ppl": "16.65", "accuracy": "45.528", "wer": "54.335", "wps": "77.2", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "3800", "lr": "0.0001931", "gnorm": "4.383", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "8880"}
[2024-06-18 20:17:36,970][train_inner][INFO] - {"epoch": 1, "update": 0.265, "loss": "3.927", "ntokens": "127.785", "acc_total": "127.785", "n_correct": "59.47", "wer_total": "127.785", "n_error": "68.155", "ppl": "15.21", "accuracy": "46.539", "wer": "53.336", "wps": "77.5", "ups": "0.61", "wpb": "127.8", "bsz": "8", "num_updates": "4000", "lr": "0.000203", "gnorm": "4.461", "loss_scale": "256", "train_wall": "329", "gb_free": "7.1", "wall": "9210"}
[2024-06-18 20:23:06,706][train_inner][INFO] - {"epoch": 1, "update": 0.278, "loss": "3.782", "ntokens": "126.255", "acc_total": "126.255", "n_correct": "61.13", "wer_total": "126.255", "n_error": "64.945", "ppl": "13.76", "accuracy": "48.418", "wer": "51.44", "wps": "76.6", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "4200", "lr": "0.0002129", "gnorm": "4.417", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "9540"}
[2024-06-18 20:28:36,408][train_inner][INFO] - {"epoch": 1, "update": 0.292, "loss": "3.672", "ntokens": "125.9", "acc_total": "125.9", "n_correct": "62.31", "wer_total": "125.9", "n_error": "63.495", "ppl": "12.74", "accuracy": "49.492", "wer": "50.433", "wps": "76.4", "ups": "0.61", "wpb": "125.9", "bsz": "8", "num_updates": "4400", "lr": "0.0002228", "gnorm": "4.399", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "9869"}
[2024-06-18 20:34:06,234][train_inner][INFO] - {"epoch": 1, "update": 0.305, "loss": "3.505", "ntokens": "127.885", "acc_total": "127.885", "n_correct": "65.44", "wer_total": "127.885", "n_error": "62.33", "ppl": "11.36", "accuracy": "51.171", "wer": "48.739", "wps": "77.5", "ups": "0.61", "wpb": "127.9", "bsz": "8", "num_updates": "4600", "lr": "0.0002327", "gnorm": "4.309", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "10199"}
[2024-06-18 20:39:35,978][train_inner][INFO] - {"epoch": 1, "update": 0.318, "loss": "3.417", "ntokens": "126.21", "acc_total": "126.21", "n_correct": "65.52", "wer_total": "126.21", "n_error": "60.615", "ppl": "10.68", "accuracy": "51.913", "wer": "48.027", "wps": "76.6", "ups": "0.61", "wpb": "126.2", "bsz": "8", "num_updates": "4800", "lr": "0.0002426", "gnorm": "4.367", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "10529"}
[2024-06-18 20:45:06,152][train_inner][INFO] - {"epoch": 1, "update": 0.331, "loss": "3.335", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "66.315", "wer_total": "126.87", "n_error": "60.42", "ppl": "10.09", "accuracy": "52.27", "wer": "47.624", "wps": "76.9", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "5000", "lr": "0.0002525", "gnorm": "4.199", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "10859"}
[2024-06-18 20:45:06,152][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-18 21:27:10,484][valid][INFO] - {"epoch": 1, "valid_loss": "2.995", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "10.292", "valid_wer_total": "18.1585", "valid_n_error": "7.8446", "valid_ppl": "7.97", "valid_accuracy": "56.679", "valid_wer": "43.201", "valid_wps": "172.6", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "5000", "valid_best_accuracy": "56.679"}
[2024-06-18 21:27:10,484][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 5000 updates
[2024-06-18 21:27:10,485][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_5000.pt
[2024-06-18 21:27:13,607][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_5000.pt
[2024-06-18 21:27:17,781][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_5000.pt (epoch 1 @ 5000 updates, score 56.679) (writing took 7.297144099953584 seconds)
[2024-06-18 21:32:47,735][train_inner][INFO] - {"epoch": 1, "update": 0.345, "loss": "3.116", "ntokens": "126.51", "acc_total": "126.51", "n_correct": "69.215", "wer_total": "126.51", "n_error": "57.17", "ppl": "8.67", "accuracy": "54.711", "wer": "45.19", "wps": "8.8", "ups": "0.07", "wpb": "126.5", "bsz": "8", "num_updates": "5200", "lr": "0.0002624", "gnorm": "4.149", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "13721"}
[2024-06-18 21:38:17,874][train_inner][INFO] - {"epoch": 1, "update": 0.358, "loss": "3.137", "ntokens": "127.425", "acc_total": "127.425", "n_correct": "69.5", "wer_total": "127.425", "n_error": "57.81", "ppl": "8.79", "accuracy": "54.542", "wer": "45.368", "wps": "77.2", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "5400", "lr": "0.0002723", "gnorm": "4.206", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "14051"}
[2024-06-18 21:43:47,931][train_inner][INFO] - {"epoch": 1, "update": 0.371, "loss": "3.024", "ntokens": "127.52", "acc_total": "127.52", "n_correct": "70.805", "wer_total": "127.52", "n_error": "56.615", "ppl": "8.14", "accuracy": "55.525", "wer": "44.397", "wps": "77.3", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "5600", "lr": "0.0002822", "gnorm": "4.203", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "14381"}
[2024-06-18 21:49:17,797][train_inner][INFO] - {"epoch": 1, "update": 0.384, "loss": "3.113", "ntokens": "126.59", "acc_total": "126.59", "n_correct": "69.485", "wer_total": "126.59", "n_error": "56.975", "ppl": "8.65", "accuracy": "54.89", "wer": "45.008", "wps": "76.8", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "5800", "lr": "0.0002921", "gnorm": "4.101", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "14711"}
[2024-06-18 21:54:47,660][train_inner][INFO] - {"epoch": 1, "update": 0.398, "loss": "2.956", "ntokens": "127.395", "acc_total": "127.395", "n_correct": "71.58", "wer_total": "127.395", "n_error": "55.69", "ppl": "7.76", "accuracy": "56.187", "wer": "43.714", "wps": "77.2", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "6000", "lr": "0.000302", "gnorm": "4.067", "loss_scale": "512", "train_wall": "329", "gb_free": "7.1", "wall": "15041"}
[2024-06-18 22:00:17,399][train_inner][INFO] - {"epoch": 1, "update": 0.411, "loss": "2.894", "ntokens": "126.995", "acc_total": "126.995", "n_correct": "72.38", "wer_total": "126.995", "n_error": "54.495", "ppl": "7.43", "accuracy": "56.994", "wer": "42.911", "wps": "77", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "6200", "lr": "0.0003119", "gnorm": "4.063", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "15370"}
[2024-06-18 22:05:47,190][train_inner][INFO] - {"epoch": 1, "update": 0.424, "loss": "2.919", "ntokens": "126.01", "acc_total": "126.01", "n_correct": "71.7", "wer_total": "126.01", "n_error": "54.185", "ppl": "7.56", "accuracy": "56.9", "wer": "43.001", "wps": "76.4", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "6400", "lr": "0.0003218", "gnorm": "3.95", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "15700"}
[2024-06-18 22:11:16,941][train_inner][INFO] - {"epoch": 1, "update": 0.437, "loss": "2.831", "ntokens": "126.33", "acc_total": "126.33", "n_correct": "72.85", "wer_total": "126.33", "n_error": "53.36", "ppl": "7.11", "accuracy": "57.666", "wer": "42.239", "wps": "76.6", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "6600", "lr": "0.0003317", "gnorm": "4.15", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "16030"}
[2024-06-18 22:16:46,631][train_inner][INFO] - {"epoch": 1, "update": 0.451, "loss": "2.773", "ntokens": "126.735", "acc_total": "126.735", "n_correct": "73.955", "wer_total": "126.735", "n_error": "52.685", "ppl": "6.83", "accuracy": "58.354", "wer": "41.571", "wps": "76.9", "ups": "0.61", "wpb": "126.7", "bsz": "8", "num_updates": "6800", "lr": "0.0003416", "gnorm": "3.907", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "16359"}
[2024-06-18 22:22:16,209][train_inner][INFO] - {"epoch": 1, "update": 0.464, "loss": "2.702", "ntokens": "126.98", "acc_total": "126.98", "n_correct": "74.83", "wer_total": "126.98", "n_error": "52.025", "ppl": "6.5", "accuracy": "58.931", "wer": "40.971", "wps": "77.1", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "7000", "lr": "0.0003515", "gnorm": "3.981", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "16689"}
[2024-06-18 22:27:45,545][train_inner][INFO] - {"epoch": 1, "update": 0.477, "loss": "2.77", "ntokens": "126.97", "acc_total": "126.97", "n_correct": "74.08", "wer_total": "126.97", "n_error": "52.805", "ppl": "6.82", "accuracy": "58.344", "wer": "41.589", "wps": "77.1", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "7200", "lr": "0.0003614", "gnorm": "3.99", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "17018"}
[2024-06-18 22:33:15,048][train_inner][INFO] - {"epoch": 1, "update": 0.491, "loss": "2.607", "ntokens": "127.45", "acc_total": "127.45", "n_correct": "76.65", "wer_total": "127.45", "n_error": "50.67", "ppl": "6.09", "accuracy": "60.141", "wer": "39.757", "wps": "77.4", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "7400", "lr": "0.0003713", "gnorm": "4.011", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "17348"}
[2024-06-18 22:35:59,759][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-18 23:17:58,183][valid][INFO] - {"epoch": 1, "valid_loss": "2.392", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "11.4093", "valid_wer_total": "18.1585", "valid_n_error": "6.7331", "valid_ppl": "5.25", "valid_accuracy": "62.832", "valid_wer": "37.08", "valid_wps": "173", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "7500", "valid_best_accuracy": "62.832"}
[2024-06-18 23:17:58,184][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 7500 updates
[2024-06-18 23:17:58,184][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_7500.pt
[2024-06-18 23:18:01,416][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_7500.pt
[2024-06-18 23:18:06,089][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_7500.pt (epoch 1 @ 7500 updates, score 62.832) (writing took 7.905333639995661 seconds)
[2024-06-18 23:20:50,578][train_inner][INFO] - {"epoch": 1, "update": 0.504, "loss": "2.626", "ntokens": "127.32", "acc_total": "127.32", "n_correct": "75.82", "wer_total": "127.32", "n_error": "51.39", "ppl": "6.17", "accuracy": "59.551", "wer": "40.363", "wps": "8.9", "ups": "0.07", "wpb": "127.3", "bsz": "8", "num_updates": "7600", "lr": "0.0003812", "gnorm": "3.938", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "20203"}
[2024-06-18 23:26:20,192][train_inner][INFO] - {"epoch": 1, "update": 0.517, "loss": "2.625", "ntokens": "127.11", "acc_total": "127.11", "n_correct": "75.71", "wer_total": "127.11", "n_error": "51.28", "ppl": "6.17", "accuracy": "59.563", "wer": "40.343", "wps": "77.1", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "7800", "lr": "0.0003911", "gnorm": "3.93", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "20533"}
[2024-06-18 23:31:49,603][train_inner][INFO] - {"epoch": 1, "update": 0.53, "loss": "2.588", "ntokens": "126.875", "acc_total": "126.875", "n_correct": "76.575", "wer_total": "126.875", "n_error": "50.17", "ppl": "6.01", "accuracy": "60.355", "wer": "39.543", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "8000", "lr": "0.000401", "gnorm": "3.914", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "20862"}
[2024-06-18 23:37:19,197][train_inner][INFO] - {"epoch": 1, "update": 0.544, "loss": "2.562", "ntokens": "126.05", "acc_total": "126.05", "n_correct": "76.235", "wer_total": "126.05", "n_error": "49.705", "ppl": "5.9", "accuracy": "60.48", "wer": "39.433", "wps": "76.5", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "8200", "lr": "0.0004109", "gnorm": "4.011", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "21192"}
[2024-06-18 23:42:48,772][train_inner][INFO] - {"epoch": 1, "update": 0.557, "loss": "2.499", "ntokens": "126.785", "acc_total": "126.785", "n_correct": "77.605", "wer_total": "126.785", "n_error": "49.085", "ppl": "5.65", "accuracy": "61.21", "wer": "38.715", "wps": "76.9", "ups": "0.61", "wpb": "126.8", "bsz": "8", "num_updates": "8400", "lr": "0.0004208", "gnorm": "4.077", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "21522"}
[2024-06-18 23:48:18,245][train_inner][INFO] - {"epoch": 1, "update": 0.57, "loss": "2.509", "ntokens": "126.675", "acc_total": "126.675", "n_correct": "77.24", "wer_total": "126.675", "n_error": "49.335", "ppl": "5.69", "accuracy": "60.975", "wer": "38.946", "wps": "76.9", "ups": "0.61", "wpb": "126.7", "bsz": "8", "num_updates": "8600", "lr": "0.0004307", "gnorm": "4.013", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "21851"}
[2024-06-18 23:53:47,597][train_inner][INFO] - {"epoch": 1, "update": 0.583, "loss": "2.495", "ntokens": "127.09", "acc_total": "127.09", "n_correct": "78.11", "wer_total": "127.09", "n_error": "48.86", "ppl": "5.64", "accuracy": "61.46", "wer": "38.445", "wps": "77.2", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "8800", "lr": "0.0004406", "gnorm": "3.958", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "22180"}
[2024-06-18 23:59:17,097][train_inner][INFO] - {"epoch": 1, "update": 0.597, "loss": "2.507", "ntokens": "127.43", "acc_total": "127.43", "n_correct": "78.07", "wer_total": "127.43", "n_error": "49.27", "ppl": "5.68", "accuracy": "61.265", "wer": "38.664", "wps": "77.3", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "9000", "lr": "0.0004505", "gnorm": "4", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "22510"}
[2024-06-19 00:04:46,459][train_inner][INFO] - {"epoch": 1, "update": 0.61, "loss": "2.462", "ntokens": "127.145", "acc_total": "127.145", "n_correct": "78.165", "wer_total": "127.145", "n_error": "48.855", "ppl": "5.51", "accuracy": "61.477", "wer": "38.425", "wps": "77.2", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "9200", "lr": "0.0004604", "gnorm": "3.965", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "22839"}
[2024-06-19 00:10:16,017][train_inner][INFO] - {"epoch": 1, "update": 0.623, "loss": "2.472", "ntokens": "125.995", "acc_total": "125.995", "n_correct": "77.58", "wer_total": "125.995", "n_error": "48.345", "ppl": "5.55", "accuracy": "61.574", "wer": "38.371", "wps": "76.5", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "9400", "lr": "0.0004703", "gnorm": "4.119", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "23169"}
[2024-06-19 00:15:45,368][train_inner][INFO] - {"epoch": 1, "update": 0.636, "loss": "2.427", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "78.88", "wer_total": "126.87", "n_error": "47.875", "ppl": "5.38", "accuracy": "62.174", "wer": "37.735", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "9600", "lr": "0.0004802", "gnorm": "4.129", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "23498"}
[2024-06-19 00:21:44,534][train_inner][INFO] - {"epoch": 1, "update": 0.65, "loss": "2.409", "ntokens": "125.68", "acc_total": "125.68", "n_correct": "78.68", "wer_total": "125.68", "n_error": "46.905", "ppl": "5.31", "accuracy": "62.603", "wer": "37.321", "wps": "70", "ups": "0.56", "wpb": "125.7", "bsz": "8", "num_updates": "9800", "lr": "0.0004901", "gnorm": "4.049", "loss_scale": "2048", "train_wall": "358", "gb_free": "7.1", "wall": "23857"}
[2024-06-19 00:27:56,780][train_inner][INFO] - {"epoch": 1, "update": 0.663, "loss": "2.425", "ntokens": "127.56", "acc_total": "127.56", "n_correct": "79.065", "wer_total": "127.56", "n_error": "48.405", "ppl": "5.37", "accuracy": "61.983", "wer": "37.947", "wps": "68.5", "ups": "0.54", "wpb": "127.6", "bsz": "8", "num_updates": "10000", "lr": "0.0005", "gnorm": "4.154", "loss_scale": "2048", "train_wall": "371", "gb_free": "7.1", "wall": "24230"}
[2024-06-19 00:27:56,781][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 01:11:50,769][valid][INFO] - {"epoch": 1, "valid_loss": "2.173", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "11.9141", "valid_wer_total": "18.1585", "valid_n_error": "6.22522", "valid_ppl": "4.51", "valid_accuracy": "65.612", "valid_wer": "34.283", "valid_wps": "165.4", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "10000", "valid_best_accuracy": "65.612"}
[2024-06-19 01:11:50,770][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 10000 updates
[2024-06-19 01:11:50,770][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_10000.pt
[2024-06-19 01:11:53,842][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_10000.pt
[2024-06-19 01:11:57,984][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_10000.pt (epoch 1 @ 10000 updates, score 65.612) (writing took 7.214347000000998 seconds)
[2024-06-19 01:17:27,273][train_inner][INFO] - {"epoch": 1, "update": 0.676, "loss": "2.382", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "80.245", "wer_total": "126.605", "n_error": "46.265", "ppl": "5.21", "accuracy": "63.382", "wer": "36.543", "wps": "8.5", "ups": "0.07", "wpb": "126.6", "bsz": "8", "num_updates": "10200", "lr": "0.000485243", "gnorm": "4.116", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "27200"}
[2024-06-19 01:21:17,904][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 2048.0
[2024-06-19 01:22:58,432][train_inner][INFO] - {"epoch": 1, "update": 0.689, "loss": "2.352", "ntokens": "127.205", "acc_total": "127.205", "n_correct": "79.845", "wer_total": "127.205", "n_error": "47.25", "ppl": "5.1", "accuracy": "62.769", "wer": "37.145", "wps": "76.8", "ups": "0.6", "wpb": "127.2", "bsz": "8", "num_updates": "10400", "lr": "0.000470922", "gnorm": "3.869", "loss_scale": "2048", "train_wall": "330", "gb_free": "7.1", "wall": "27531"}
[2024-06-19 01:28:28,099][train_inner][INFO] - {"epoch": 1, "update": 0.703, "loss": "2.357", "ntokens": "127.885", "acc_total": "127.885", "n_correct": "80.46", "wer_total": "127.885", "n_error": "47.35", "ppl": "5.12", "accuracy": "62.916", "wer": "37.025", "wps": "77.6", "ups": "0.61", "wpb": "127.9", "bsz": "8", "num_updates": "10600", "lr": "0.000457024", "gnorm": "4.004", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "27861"}
[2024-06-19 01:33:57,494][train_inner][INFO] - {"epoch": 1, "update": 0.716, "loss": "2.347", "ntokens": "127.33", "acc_total": "127.33", "n_correct": "80.375", "wer_total": "127.33", "n_error": "46.87", "ppl": "5.09", "accuracy": "63.123", "wer": "36.81", "wps": "77.3", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "10800", "lr": "0.000443536", "gnorm": "3.828", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "28190"}
[2024-06-19 01:39:27,161][train_inner][INFO] - {"epoch": 1, "update": 0.729, "loss": "2.266", "ntokens": "126.09", "acc_total": "126.09", "n_correct": "80.465", "wer_total": "126.09", "n_error": "45.56", "ppl": "4.81", "accuracy": "63.816", "wer": "36.133", "wps": "76.5", "ups": "0.61", "wpb": "126.1", "bsz": "8", "num_updates": "11000", "lr": "0.000430446", "gnorm": "3.858", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "28520"}
[2024-06-19 01:44:56,718][train_inner][INFO] - {"epoch": 1, "update": 0.742, "loss": "2.181", "ntokens": "126.625", "acc_total": "126.625", "n_correct": "82.3", "wer_total": "126.625", "n_error": "44.265", "ppl": "4.53", "accuracy": "64.995", "wer": "34.958", "wps": "76.8", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "11200", "lr": "0.000417742", "gnorm": "3.834", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "28850"}
[2024-06-19 01:49:05,495][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-06-19 01:50:27,942][train_inner][INFO] - {"epoch": 1, "update": 0.756, "loss": "2.192", "ntokens": "125.885", "acc_total": "125.885", "n_correct": "81.595", "wer_total": "125.885", "n_error": "44.23", "ppl": "4.57", "accuracy": "64.817", "wer": "35.135", "wps": "76", "ups": "0.6", "wpb": "125.9", "bsz": "8", "num_updates": "11400", "lr": "0.000405413", "gnorm": "3.882", "loss_scale": "1024", "train_wall": "331", "gb_free": "7.1", "wall": "29181"}
[2024-06-19 01:55:57,664][train_inner][INFO] - {"epoch": 1, "update": 0.769, "loss": "2.186", "ntokens": "126.905", "acc_total": "126.905", "n_correct": "82.53", "wer_total": "126.905", "n_error": "44.31", "ppl": "4.55", "accuracy": "65.033", "wer": "34.916", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "11600", "lr": "0.000393448", "gnorm": "3.81", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "29511"}
[2024-06-19 02:01:27,354][train_inner][INFO] - {"epoch": 1, "update": 0.782, "loss": "2.156", "ntokens": "126.775", "acc_total": "126.775", "n_correct": "83.115", "wer_total": "126.775", "n_error": "43.6", "ppl": "4.46", "accuracy": "65.561", "wer": "34.392", "wps": "76.9", "ups": "0.61", "wpb": "126.8", "bsz": "8", "num_updates": "11800", "lr": "0.000381836", "gnorm": "3.731", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "29840"}
[2024-06-19 02:06:56,874][train_inner][INFO] - {"epoch": 1, "update": 0.796, "loss": "2.119", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "82.62", "wer_total": "126.605", "n_error": "43.92", "ppl": "4.34", "accuracy": "65.258", "wer": "34.691", "wps": "76.8", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "12000", "lr": "0.000370567", "gnorm": "3.833", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "30170"}
[2024-06-19 02:12:26,577][train_inner][INFO] - {"epoch": 1, "update": 0.809, "loss": "2.144", "ntokens": "126.91", "acc_total": "126.91", "n_correct": "84.61", "wer_total": "126.91", "n_error": "42.25", "ppl": "4.42", "accuracy": "66.669", "wer": "33.291", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "12200", "lr": "0.000359631", "gnorm": "3.85", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "30499"}
[2024-06-19 02:17:56,175][train_inner][INFO] - {"epoch": 1, "update": 0.822, "loss": "2.122", "ntokens": "126.95", "acc_total": "126.95", "n_correct": "84.745", "wer_total": "126.95", "n_error": "42.13", "ppl": "4.35", "accuracy": "66.755", "wer": "33.186", "wps": "77", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "12400", "lr": "0.000349017", "gnorm": "3.762", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "30829"}
[2024-06-19 02:20:40,906][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 03:02:39,635][valid][INFO] - {"epoch": 1, "valid_loss": "nan", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "12.6781", "valid_wer_total": "18.1585", "valid_n_error": "5.47424", "valid_ppl": "nan", "valid_accuracy": "69.819", "valid_wer": "30.147", "valid_wps": "173", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "12500", "valid_best_accuracy": "69.819"}
[2024-06-19 03:02:39,635][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 12500 updates
[2024-06-19 03:02:39,636][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_12500.pt
[2024-06-19 03:02:42,756][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_12500.pt
[2024-06-19 03:02:47,899][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_12500.pt (epoch 1 @ 12500 updates, score 69.819) (writing took 8.263699680042919 seconds)
[2024-06-19 03:05:32,184][train_inner][INFO] - {"epoch": 1, "update": 0.835, "loss": "2.021", "ntokens": "126.58", "acc_total": "126.58", "n_correct": "84.34", "wer_total": "126.58", "n_error": "42.155", "ppl": "4.06", "accuracy": "66.63", "wer": "33.303", "wps": "8.9", "ups": "0.07", "wpb": "126.6", "bsz": "8", "num_updates": "12600", "lr": "0.000338716", "gnorm": "3.654", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "33685"}
[2024-06-19 03:11:01,795][train_inner][INFO] - {"epoch": 1, "update": 0.849, "loss": "2.001", "ntokens": "127.27", "acc_total": "127.27", "n_correct": "85.455", "wer_total": "127.27", "n_error": "41.765", "ppl": "4", "accuracy": "67.145", "wer": "32.816", "wps": "77.2", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "12800", "lr": "0.00032872", "gnorm": "3.611", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "34015"}
[2024-06-19 03:16:31,156][train_inner][INFO] - {"epoch": 1, "update": 0.862, "loss": "2.023", "ntokens": "125.955", "acc_total": "125.955", "n_correct": "85.42", "wer_total": "125.955", "n_error": "40.475", "ppl": "4.06", "accuracy": "67.818", "wer": "32.134", "wps": "76.5", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "13000", "lr": "0.000319018", "gnorm": "3.634", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "34344"}
[2024-06-19 03:22:00,604][train_inner][INFO] - {"epoch": 1, "update": 0.875, "loss": "2.039", "ntokens": "126.89", "acc_total": "126.89", "n_correct": "84.755", "wer_total": "126.89", "n_error": "42.075", "ppl": "4.11", "accuracy": "66.794", "wer": "33.159", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "13200", "lr": "0.000309603", "gnorm": "3.546", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "34673"}
[2024-06-19 03:27:30,145][train_inner][INFO] - {"epoch": 1, "update": 0.888, "loss": "2.024", "ntokens": "127.49", "acc_total": "127.49", "n_correct": "86.35", "wer_total": "127.49", "n_error": "41.1", "ppl": "4.07", "accuracy": "67.731", "wer": "32.238", "wps": "77.4", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "13400", "lr": "0.000300466", "gnorm": "3.674", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "35003"}
[2024-06-19 03:32:59,456][train_inner][INFO] - {"epoch": 1, "update": 0.902, "loss": "1.94", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "86.065", "wer_total": "126.87", "n_error": "40.755", "ppl": "3.84", "accuracy": "67.837", "wer": "32.123", "wps": "77.1", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "13600", "lr": "0.000291598", "gnorm": "3.568", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "35332"}
[2024-06-19 03:38:28,929][train_inner][INFO] - {"epoch": 1, "update": 0.915, "loss": "1.9", "ntokens": "127.055", "acc_total": "127.055", "n_correct": "88.535", "wer_total": "127.055", "n_error": "38.435", "ppl": "3.73", "accuracy": "69.682", "wer": "30.251", "wps": "77.1", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "13800", "lr": "0.000282992", "gnorm": "3.511", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "35662"}
[2024-06-19 03:43:58,458][train_inner][INFO] - {"epoch": 1, "update": 0.928, "loss": "1.932", "ntokens": "127.135", "acc_total": "127.135", "n_correct": "87.575", "wer_total": "127.135", "n_error": "39.51", "ppl": "3.82", "accuracy": "68.883", "wer": "31.077", "wps": "77.2", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "14000", "lr": "0.00027464", "gnorm": "3.58", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "35991"}
[2024-06-19 03:49:27,939][train_inner][INFO] - {"epoch": 1, "update": 0.941, "loss": "1.894", "ntokens": "126.005", "acc_total": "126.005", "n_correct": "87.84", "wer_total": "126.005", "n_error": "38.115", "ppl": "3.72", "accuracy": "69.712", "wer": "30.249", "wps": "76.5", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "14200", "lr": "0.000266535", "gnorm": "3.497", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "36321"}
[2024-06-19 03:54:57,463][train_inner][INFO] - {"epoch": 1, "update": 0.955, "loss": "1.923", "ntokens": "126.9", "acc_total": "126.9", "n_correct": "89.67", "wer_total": "126.9", "n_error": "37.19", "ppl": "3.79", "accuracy": "70.662", "wer": "29.307", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "14400", "lr": "0.000258668", "gnorm": "3.428", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "36650"}
[2024-06-19 04:00:27,117][train_inner][INFO] - {"epoch": 1, "update": 0.968, "loss": "1.843", "ntokens": "127.485", "acc_total": "127.485", "n_correct": "91.46", "wer_total": "127.485", "n_error": "35.985", "ppl": "3.59", "accuracy": "71.742", "wer": "28.227", "wps": "77.3", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "14600", "lr": "0.000251034", "gnorm": "3.266", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "36980"}
[2024-06-19 04:05:56,611][train_inner][INFO] - {"epoch": 1, "update": 0.981, "loss": "1.88", "ntokens": "127.26", "acc_total": "127.26", "n_correct": "89.35", "wer_total": "127.26", "n_error": "37.895", "ppl": "3.68", "accuracy": "70.211", "wer": "29.778", "wps": "77.2", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "14800", "lr": "0.000243626", "gnorm": "3.268", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "37309"}
[2024-06-19 04:11:26,037][train_inner][INFO] - {"epoch": 1, "update": 0.994, "loss": "1.853", "ntokens": "127.385", "acc_total": "127.385", "n_correct": "90.85", "wer_total": "127.385", "n_error": "36.505", "ppl": "3.61", "accuracy": "71.319", "wer": "28.657", "wps": "77.3", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "15000", "lr": "0.000236435", "gnorm": "3.294", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "37639"}
[2024-06-19 04:11:26,037][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 04:53:22,091][valid][INFO] - {"epoch": 1, "valid_loss": "1.598", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.9227", "valid_wer_total": "18.1585", "valid_n_error": "4.22955", "valid_ppl": "3.03", "valid_accuracy": "76.673", "valid_wer": "23.292", "valid_wps": "173.1", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "15000", "valid_best_accuracy": "76.673"}
[2024-06-19 04:53:22,091][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 15000 updates
[2024-06-19 04:53:22,092][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_15000.pt
[2024-06-19 04:53:25,220][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_15000.pt
[2024-06-19 04:53:29,568][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_15000.pt (epoch 1 @ 15000 updates, score 76.673) (writing took 7.47604586702073 seconds)
[2024-06-19 04:55:47,119][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 05:37:43,539][valid][INFO] - {"epoch": 1, "valid_loss": "1.598", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.5176", "valid_wer_total": "18.1585", "valid_n_error": "4.63526", "valid_ppl": "3.03", "valid_accuracy": "74.443", "valid_wer": "25.527", "valid_wps": "173.1", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "15084", "valid_best_accuracy": "76.673"}
[2024-06-19 05:37:43,540][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 15084 updates
[2024-06-19 05:37:43,540][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_last.pt
[2024-06-19 05:37:47,346][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_last.pt
[2024-06-19 05:37:47,413][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_last.pt (epoch 1 @ 15084 updates, score 74.443) (writing took 3.8733164540026337 seconds)
[2024-06-19 05:37:47,413][fairseq_cli.train][INFO] - end of epoch 1 (average epoch stats below)
[2024-06-19 05:37:47,415][train][INFO] - {"epoch": 1, "train_loss": "3.264", "train_ntokens": "126.897", "train_acc_total": "126.897", "train_n_correct": "67.9822", "train_wer_total": "126.897", "train_n_error": "58.7906", "train_ppl": "9.61", "train_accuracy": "53.573", "train_wer": "46.329", "train_wps": "44.7", "train_ups": "0.35", "train_wpb": "126.9", "train_bsz": "8", "train_num_updates": "15084", "train_lr": "0.000233479", "train_gnorm": "3.913", "train_loss_scale": "2048", "train_train_wall": "24959", "train_gb_free": "7.1", "train_wall": "42820"}
[2024-06-19 05:37:47,465][fairseq.trainer][INFO] - begin training epoch 2
[2024-06-19 05:37:47,465][fairseq_cli.train][INFO] - Start iterating over samples
[2024-06-19 05:40:58,314][train_inner][INFO] - {"epoch": 2, "update": 1.008, "loss": "1.76", "ntokens": "127.05", "acc_total": "127.05", "n_correct": "91.455", "wer_total": "127.05", "n_error": "35.56", "ppl": "3.39", "accuracy": "71.983", "wer": "27.989", "wps": "4.7", "ups": "0.04", "wpb": "127", "bsz": "8", "num_updates": "15200", "lr": "0.000229457", "gnorm": "3.211", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "43011"}
[2024-06-19 05:46:27,601][train_inner][INFO] - {"epoch": 2, "update": 1.021, "loss": "1.692", "ntokens": "126.32", "acc_total": "126.32", "n_correct": "91.335", "wer_total": "126.32", "n_error": "34.945", "ppl": "3.23", "accuracy": "72.304", "wer": "27.664", "wps": "76.7", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "15400", "lr": "0.000222685", "gnorm": "3.215", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "43340"}
[2024-06-19 05:50:51,063][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 2048.0
[2024-06-19 05:51:58,553][train_inner][INFO] - {"epoch": 2, "update": 1.034, "loss": "1.681", "ntokens": "127.005", "acc_total": "127.005", "n_correct": "92.445", "wer_total": "127.005", "n_error": "34.51", "ppl": "3.21", "accuracy": "72.788", "wer": "27.172", "wps": "76.8", "ups": "0.6", "wpb": "127", "bsz": "8", "num_updates": "15600", "lr": "0.000216113", "gnorm": "3.261", "loss_scale": "2048", "train_wall": "330", "gb_free": "7.1", "wall": "43671"}
[2024-06-19 05:57:27,827][train_inner][INFO] - {"epoch": 2, "update": 1.048, "loss": "1.744", "ntokens": "125.545", "acc_total": "125.545", "n_correct": "92.565", "wer_total": "125.545", "n_error": "32.94", "ppl": "3.35", "accuracy": "73.731", "wer": "26.238", "wps": "76.3", "ups": "0.61", "wpb": "125.5", "bsz": "8", "num_updates": "15800", "lr": "0.000209735", "gnorm": "3.254", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44001"}
[2024-06-19 06:02:57,149][train_inner][INFO] - {"epoch": 2, "update": 1.061, "loss": "1.703", "ntokens": "127.405", "acc_total": "127.405", "n_correct": "91.79", "wer_total": "127.405", "n_error": "35.57", "ppl": "3.26", "accuracy": "72.046", "wer": "27.919", "wps": "77.4", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "16000", "lr": "0.000203545", "gnorm": "3.269", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44330"}
[2024-06-19 06:08:26,637][train_inner][INFO] - {"epoch": 2, "update": 1.074, "loss": "1.587", "ntokens": "126.915", "acc_total": "126.915", "n_correct": "91.965", "wer_total": "126.915", "n_error": "34.94", "ppl": "3", "accuracy": "72.462", "wer": "27.53", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "16200", "lr": "0.000197538", "gnorm": "3.146", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44660"}
[2024-06-19 06:13:56,245][train_inner][INFO] - {"epoch": 2, "update": 1.087, "loss": "1.615", "ntokens": "127.14", "acc_total": "127.14", "n_correct": "95.555", "wer_total": "127.14", "n_error": "31.565", "ppl": "3.06", "accuracy": "75.157", "wer": "24.827", "wps": "77.1", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "16400", "lr": "0.000191708", "gnorm": "3.135", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44989"}
[2024-06-19 06:19:25,555][train_inner][INFO] - {"epoch": 2, "update": 1.101, "loss": "1.606", "ntokens": "127.51", "acc_total": "127.51", "n_correct": "93.99", "wer_total": "127.51", "n_error": "33.47", "ppl": "3.04", "accuracy": "73.712", "wer": "26.249", "wps": "77.4", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "16600", "lr": "0.00018605", "gnorm": "3.198", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "45318"}
[2024-06-19 06:24:54,947][train_inner][INFO] - {"epoch": 2, "update": 1.114, "loss": "1.66", "ntokens": "126.405", "acc_total": "126.405", "n_correct": "94.395", "wer_total": "126.405", "n_error": "31.995", "ppl": "3.16", "accuracy": "74.677", "wer": "25.311", "wps": "76.8", "ups": "0.61", "wpb": "126.4", "bsz": "8", "num_updates": "16800", "lr": "0.000180559", "gnorm": "3.171", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "45648"}
[2024-06-19 06:30:24,267][train_inner][INFO] - {"epoch": 2, "update": 1.127, "loss": "1.571", "ntokens": "127.925", "acc_total": "127.925", "n_correct": "95.35", "wer_total": "127.925", "n_error": "32.55", "ppl": "2.97", "accuracy": "74.536", "wer": "25.445", "wps": "77.7", "ups": "0.61", "wpb": "127.9", "bsz": "8", "num_updates": "17000", "lr": "0.00017523", "gnorm": "3.027", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "45977"}
[2024-06-19 06:35:53,581][train_inner][INFO] - {"epoch": 2, "update": 1.14, "loss": "1.6", "ntokens": "126.125", "acc_total": "126.125", "n_correct": "94.8", "wer_total": "126.125", "n_error": "31.31", "ppl": "3.03", "accuracy": "75.164", "wer": "24.825", "wps": "76.6", "ups": "0.61", "wpb": "126.1", "bsz": "8", "num_updates": "17200", "lr": "0.000170059", "gnorm": "3.016", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "46306"}
[2024-06-19 06:41:22,945][train_inner][INFO] - {"epoch": 2, "update": 1.154, "loss": "1.615", "ntokens": "127.245", "acc_total": "127.245", "n_correct": "94.385", "wer_total": "127.245", "n_error": "32.81", "ppl": "3.06", "accuracy": "74.176", "wer": "25.785", "wps": "77.3", "ups": "0.61", "wpb": "127.2", "bsz": "8", "num_updates": "17400", "lr": "0.00016504", "gnorm": "3.116", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "46636"}
[2024-06-19 06:44:07,577][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 07:26:05,066][valid][INFO] - {"epoch": 2, "valid_loss": "1.476", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "14.2186", "valid_wer_total": "18.1585", "valid_n_error": "3.93564", "valid_ppl": "2.78", "valid_accuracy": "78.303", "valid_wer": "21.674", "valid_wps": "173", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "17500", "valid_best_accuracy": "78.303"}
[2024-06-19 07:26:05,066][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 17500 updates
[2024-06-19 07:26:05,067][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_17500.pt
[2024-06-19 07:26:08,235][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_17500.pt
[2024-06-19 07:26:12,556][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_17500.pt (epoch 2 @ 17500 updates, score 78.303) (writing took 7.489264687988907 seconds)
[2024-06-19 07:28:56,817][train_inner][INFO] - {"epoch": 2, "update": 1.167, "loss": "1.627", "ntokens": "126.2", "acc_total": "126.2", "n_correct": "95.75", "wer_total": "126.2", "n_error": "30.435", "ppl": "3.09", "accuracy": "75.872", "wer": "24.116", "wps": "8.8", "ups": "0.07", "wpb": "126.2", "bsz": "8", "num_updates": "17600", "lr": "0.000160169", "gnorm": "3.193", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "49490"}
[2024-06-19 07:33:44,925][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 2048.0
[2024-06-19 07:34:27,784][train_inner][INFO] - {"epoch": 2, "update": 1.18, "loss": "1.518", "ntokens": "128.235", "acc_total": "128.235", "n_correct": "98.46", "wer_total": "128.235", "n_error": "29.765", "ppl": "2.86", "accuracy": "76.781", "wer": "23.211", "wps": "77.5", "ups": "0.6", "wpb": "128.2", "bsz": "8", "num_updates": "17800", "lr": "0.000155442", "gnorm": "2.922", "loss_scale": "2048", "train_wall": "330", "gb_free": "7.1", "wall": "49821"}
[2024-06-19 07:39:57,058][train_inner][INFO] - {"epoch": 2, "update": 1.193, "loss": "1.583", "ntokens": "126.265", "acc_total": "126.265", "n_correct": "94.265", "wer_total": "126.265", "n_error": "31.99", "ppl": "3", "accuracy": "74.656", "wer": "25.336", "wps": "76.7", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "18000", "lr": "0.000150854", "gnorm": "3.001", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "50150"}
[2024-06-19 07:46:02,786][train_inner][INFO] - {"epoch": 2, "update": 1.207, "loss": "1.641", "ntokens": "127.715", "acc_total": "127.715", "n_correct": "96.35", "wer_total": "127.715", "n_error": "31.335", "ppl": "3.12", "accuracy": "75.441", "wer": "24.535", "wps": "69.8", "ups": "0.55", "wpb": "127.7", "bsz": "8", "num_updates": "18200", "lr": "0.000146402", "gnorm": "10.012", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "50516"}
[2024-06-19 07:52:07,848][train_inner][INFO] - {"epoch": 2, "update": 1.22, "loss": "1.572", "ntokens": "126.27", "acc_total": "126.27", "n_correct": "95.295", "wer_total": "126.27", "n_error": "30.95", "ppl": "2.97", "accuracy": "75.469", "wer": "24.511", "wps": "69.2", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "18400", "lr": "0.000142081", "gnorm": "9.877", "loss_scale": "2048", "train_wall": "364", "gb_free": "6.5", "wall": "50881"}
[2024-06-19 07:58:13,227][train_inner][INFO] - {"epoch": 2, "update": 1.233, "loss": "1.563", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "96.735", "wer_total": "126.87", "n_error": "30.11", "ppl": "2.96", "accuracy": "76.247", "wer": "23.733", "wps": "69.4", "ups": "0.55", "wpb": "126.9", "bsz": "8", "num_updates": "18600", "lr": "0.000137888", "gnorm": "9.146", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "51246"}
[2024-06-19 08:04:18,569][train_inner][INFO] - {"epoch": 2, "update": 1.246, "loss": "1.503", "ntokens": "127.155", "acc_total": "127.155", "n_correct": "99.01", "wer_total": "127.155", "n_error": "28.105", "ppl": "2.83", "accuracy": "77.866", "wer": "22.103", "wps": "69.6", "ups": "0.55", "wpb": "127.2", "bsz": "8", "num_updates": "18800", "lr": "0.000133819", "gnorm": "8.909", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "51611"}
[2024-06-19 08:10:23,919][train_inner][INFO] - {"epoch": 2, "update": 1.26, "loss": "1.459", "ntokens": "127.695", "acc_total": "127.695", "n_correct": "98.42", "wer_total": "127.695", "n_error": "29.255", "ppl": "2.75", "accuracy": "77.074", "wer": "22.91", "wps": "69.9", "ups": "0.55", "wpb": "127.7", "bsz": "8", "num_updates": "19000", "lr": "0.000129869", "gnorm": "8.517", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "51977"}
[2024-06-19 08:16:29,325][train_inner][INFO] - {"epoch": 2, "update": 1.273, "loss": "1.525", "ntokens": "127.145", "acc_total": "127.145", "n_correct": "97.93", "wer_total": "127.145", "n_error": "29.205", "ppl": "2.88", "accuracy": "77.022", "wer": "22.97", "wps": "69.6", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "19200", "lr": "0.000126036", "gnorm": "8.878", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "52342"}
[2024-06-19 08:22:34,636][train_inner][INFO] - {"epoch": 2, "update": 1.286, "loss": "1.38", "ntokens": "125.81", "acc_total": "125.81", "n_correct": "99.97", "wer_total": "125.81", "n_error": "25.83", "ppl": "2.6", "accuracy": "79.461", "wer": "20.531", "wps": "68.9", "ups": "0.55", "wpb": "125.8", "bsz": "8", "num_updates": "19400", "lr": "0.000122317", "gnorm": "8.208", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "52707"}
[2024-06-19 08:27:06,681][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-06-19 08:28:41,641][train_inner][INFO] - {"epoch": 2, "update": 1.3, "loss": "1.413", "ntokens": "125.655", "acc_total": "125.655", "n_correct": "99.41", "wer_total": "125.655", "n_error": "26.235", "ppl": "2.66", "accuracy": "79.113", "wer": "20.879", "wps": "68.5", "ups": "0.54", "wpb": "125.7", "bsz": "8", "num_updates": "19600", "lr": "0.000118707", "gnorm": "8.35", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "53075"}
[2024-06-19 08:34:47,031][train_inner][INFO] - {"epoch": 2, "update": 1.313, "loss": "1.384", "ntokens": "127.615", "acc_total": "127.615", "n_correct": "102.22", "wer_total": "127.615", "n_error": "25.39", "ppl": "2.61", "accuracy": "80.1", "wer": "19.896", "wps": "69.9", "ups": "0.55", "wpb": "127.6", "bsz": "8", "num_updates": "19800", "lr": "0.000115203", "gnorm": "8.003", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "53440"}
[2024-06-19 08:40:52,437][train_inner][INFO] - {"epoch": 2, "update": 1.326, "loss": "1.418", "ntokens": "126.165", "acc_total": "126.165", "n_correct": "100.64", "wer_total": "126.165", "n_error": "25.5", "ppl": "2.67", "accuracy": "79.769", "wer": "20.212", "wps": "69.1", "ups": "0.55", "wpb": "126.2", "bsz": "8", "num_updates": "20000", "lr": "0.000111803", "gnorm": "8.19", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "53805"}
[2024-06-19 08:40:52,438][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 09:22:52,794][valid][INFO] - {"epoch": 2, "valid_loss": "1.288", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "14.6005", "valid_wer_total": "18.1585", "valid_n_error": "3.55556", "valid_ppl": "2.44", "valid_accuracy": "80.406", "valid_wer": "19.581", "valid_wps": "172.8", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "20000", "valid_best_accuracy": "80.406"}
[2024-06-19 09:22:52,794][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 20000 updates
[2024-06-19 09:22:52,795][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_20000.pt
[2024-06-19 09:22:55,949][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_20000.pt
[2024-06-19 09:23:00,163][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_20000.pt (epoch 2 @ 20000 updates, score 80.406) (writing took 7.368800246040337 seconds)
[2024-06-19 09:29:05,490][train_inner][INFO] - {"epoch": 2, "update": 1.339, "loss": "1.421", "ntokens": "126.37", "acc_total": "126.37", "n_correct": "99.39", "wer_total": "126.37", "n_error": "26.965", "ppl": "2.68", "accuracy": "78.65", "wer": "21.338", "wps": "8.7", "ups": "0.07", "wpb": "126.4", "bsz": "8", "num_updates": "20200", "lr": "0.000108504", "gnorm": "8.417", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "56698"}
[2024-06-19 09:35:10,932][train_inner][INFO] - {"epoch": 2, "update": 1.353, "loss": "1.422", "ntokens": "127.095", "acc_total": "127.095", "n_correct": "100.84", "wer_total": "127.095", "n_error": "26.23", "ppl": "2.68", "accuracy": "79.342", "wer": "20.638", "wps": "69.6", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "20400", "lr": "0.000105301", "gnorm": "8.063", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "57064"}
[2024-06-19 09:41:16,401][train_inner][INFO] - {"epoch": 2, "update": 1.366, "loss": "1.359", "ntokens": "126.5", "acc_total": "126.5", "n_correct": "101.14", "wer_total": "126.5", "n_error": "25.355", "ppl": "2.56", "accuracy": "79.953", "wer": "20.043", "wps": "69.2", "ups": "0.55", "wpb": "126.5", "bsz": "8", "num_updates": "20600", "lr": "0.000102194", "gnorm": "7.776", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "57429"}
[2024-06-19 09:47:22,061][train_inner][INFO] - {"epoch": 2, "update": 1.379, "loss": "1.348", "ntokens": "127.265", "acc_total": "127.265", "n_correct": "101.315", "wer_total": "127.265", "n_error": "25.92", "ppl": "2.55", "accuracy": "79.609", "wer": "20.367", "wps": "69.6", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "20800", "lr": "9.91776e-05", "gnorm": "7.723", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "57795"}
[2024-06-19 09:53:27,367][train_inner][INFO] - {"epoch": 2, "update": 1.392, "loss": "1.345", "ntokens": "127.405", "acc_total": "127.405", "n_correct": "101.98", "wer_total": "127.405", "n_error": "25.41", "ppl": "2.54", "accuracy": "80.044", "wer": "19.944", "wps": "69.8", "ups": "0.55", "wpb": "127.4", "bsz": "8", "num_updates": "21000", "lr": "9.62506e-05", "gnorm": "7.604", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "58160"}
[2024-06-19 09:59:32,995][train_inner][INFO] - {"epoch": 2, "update": 1.406, "loss": "1.313", "ntokens": "126.7", "acc_total": "126.7", "n_correct": "102.375", "wer_total": "126.7", "n_error": "24.32", "ppl": "2.48", "accuracy": "80.801", "wer": "19.195", "wps": "69.3", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "21200", "lr": "9.341e-05", "gnorm": "7.507", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "58526"}
[2024-06-19 10:05:38,578][train_inner][INFO] - {"epoch": 2, "update": 1.419, "loss": "1.291", "ntokens": "127.32", "acc_total": "127.32", "n_correct": "103.005", "wer_total": "127.32", "n_error": "24.305", "ppl": "2.45", "accuracy": "80.902", "wer": "19.09", "wps": "69.7", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "21400", "lr": "9.06532e-05", "gnorm": "7.595", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "58891"}
[2024-06-19 10:11:44,176][train_inner][INFO] - {"epoch": 2, "update": 1.432, "loss": "1.312", "ntokens": "126.805", "acc_total": "126.805", "n_correct": "101.465", "wer_total": "126.805", "n_error": "25.295", "ppl": "2.48", "accuracy": "80.017", "wer": "19.948", "wps": "69.4", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "21600", "lr": "8.79777e-05", "gnorm": "7.425", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "59257"}
[2024-06-19 10:17:49,730][train_inner][INFO] - {"epoch": 2, "update": 1.445, "loss": "1.276", "ntokens": "125.555", "acc_total": "125.555", "n_correct": "102.115", "wer_total": "125.555", "n_error": "23.42", "ppl": "2.42", "accuracy": "81.331", "wer": "18.653", "wps": "68.7", "ups": "0.55", "wpb": "125.6", "bsz": "8", "num_updates": "21800", "lr": "8.53812e-05", "gnorm": "7.511", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "59623"}
[2024-06-19 10:21:41,879][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-06-19 10:23:57,107][train_inner][INFO] - {"epoch": 2, "update": 1.459, "loss": "1.332", "ntokens": "127.59", "acc_total": "127.59", "n_correct": "102.31", "wer_total": "127.59", "n_error": "25.26", "ppl": "2.52", "accuracy": "80.187", "wer": "19.798", "wps": "69.5", "ups": "0.54", "wpb": "127.6", "bsz": "8", "num_updates": "22000", "lr": "8.28614e-05", "gnorm": "7.478", "loss_scale": "1024", "train_wall": "367", "gb_free": "6.5", "wall": "59990"}
[2024-06-19 10:30:02,535][train_inner][INFO] - {"epoch": 2, "update": 1.472, "loss": "1.231", "ntokens": "126.58", "acc_total": "126.58", "n_correct": "102.975", "wer_total": "126.58", "n_error": "23.58", "ppl": "2.35", "accuracy": "81.352", "wer": "18.629", "wps": "69.3", "ups": "0.55", "wpb": "126.6", "bsz": "8", "num_updates": "22200", "lr": "8.04159e-05", "gnorm": "7.323", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "60355"}
[2024-06-19 10:36:07,685][train_inner][INFO] - {"epoch": 2, "update": 1.485, "loss": "1.265", "ntokens": "127.075", "acc_total": "127.075", "n_correct": "103.155", "wer_total": "127.075", "n_error": "23.91", "ppl": "2.4", "accuracy": "81.176", "wer": "18.816", "wps": "69.6", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "22400", "lr": "7.80425e-05", "gnorm": "7.653", "loss_scale": "1024", "train_wall": "364", "gb_free": "6.5", "wall": "60721"}
[2024-06-19 10:39:10,573][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 11:21:12,774][valid][INFO] - {"epoch": 2, "valid_loss": "1.147", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.0861", "valid_wer_total": "18.1585", "valid_n_error": "3.07036", "valid_ppl": "2.21", "valid_accuracy": "83.08", "valid_wer": "16.909", "valid_wps": "172.7", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "22500", "valid_best_accuracy": "83.08"}
[2024-06-19 11:21:12,774][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 22500 updates
[2024-06-19 11:21:12,775][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_22500.pt
[2024-06-19 11:21:15,989][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_22500.pt
[2024-06-19 11:21:20,345][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_22500.pt (epoch 2 @ 22500 updates, score 83.08) (writing took 7.570348905981518 seconds)
[2024-06-19 11:24:22,829][train_inner][INFO] - {"epoch": 2, "update": 1.498, "loss": "1.289", "ntokens": "126.79", "acc_total": "126.79", "n_correct": "103.025", "wer_total": "126.79", "n_error": "23.755", "ppl": "2.44", "accuracy": "81.256", "wer": "18.736", "wps": "8.8", "ups": "0.07", "wpb": "126.8", "bsz": "8", "num_updates": "22600", "lr": "7.57393e-05", "gnorm": "7.403", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "63616"}
[2024-06-19 11:25:26,748][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 512.0
[2024-06-19 11:30:30,300][train_inner][INFO] - {"epoch": 2, "update": 1.512, "loss": "1.236", "ntokens": "126.545", "acc_total": "126.545", "n_correct": "102.08", "wer_total": "126.545", "n_error": "24.435", "ppl": "2.35", "accuracy": "80.667", "wer": "19.309", "wps": "68.9", "ups": "0.54", "wpb": "126.5", "bsz": "8", "num_updates": "22800", "lr": "7.3504e-05", "gnorm": "6.943", "loss_scale": "512", "train_wall": "367", "gb_free": "6.5", "wall": "63983"}
[2024-06-19 11:36:36,019][train_inner][INFO] - {"epoch": 2, "update": 1.525, "loss": "1.297", "ntokens": "126.15", "acc_total": "126.15", "n_correct": "101.8", "wer_total": "126.15", "n_error": "24.345", "ppl": "2.46", "accuracy": "80.698", "wer": "19.298", "wps": "69", "ups": "0.55", "wpb": "126.2", "bsz": "8", "num_updates": "23000", "lr": "7.13346e-05", "gnorm": "7.305", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "64349"}
[2024-06-19 11:42:42,006][train_inner][INFO] - {"epoch": 2, "update": 1.538, "loss": "1.227", "ntokens": "126.92", "acc_total": "126.92", "n_correct": "104.125", "wer_total": "126.92", "n_error": "22.775", "ppl": "2.34", "accuracy": "82.04", "wer": "17.944", "wps": "69.4", "ups": "0.55", "wpb": "126.9", "bsz": "8", "num_updates": "23200", "lr": "6.92293e-05", "gnorm": "7.326", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "64715"}
[2024-06-19 11:48:47,587][train_inner][INFO] - {"epoch": 2, "update": 1.552, "loss": "1.169", "ntokens": "126.32", "acc_total": "126.32", "n_correct": "104.955", "wer_total": "126.32", "n_error": "21.34", "ppl": "2.25", "accuracy": "83.087", "wer": "16.894", "wps": "69.1", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "23400", "lr": "6.71862e-05", "gnorm": "6.986", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "65080"}
[2024-06-19 11:54:53,430][train_inner][INFO] - {"epoch": 2, "update": 1.565, "loss": "1.289", "ntokens": "126.68", "acc_total": "126.68", "n_correct": "103.225", "wer_total": "126.68", "n_error": "23.44", "ppl": "2.44", "accuracy": "81.485", "wer": "18.503", "wps": "69.3", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "23600", "lr": "6.52033e-05", "gnorm": "7.355", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "65446"}
[2024-06-19 12:00:59,179][train_inner][INFO] - {"epoch": 2, "update": 1.578, "loss": "1.24", "ntokens": "127.13", "acc_total": "127.13", "n_correct": "102.35", "wer_total": "127.13", "n_error": "24.775", "ppl": "2.36", "accuracy": "80.508", "wer": "19.488", "wps": "69.5", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "23800", "lr": "6.3279e-05", "gnorm": "7.162", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "65812"}
[2024-06-19 12:07:05,013][train_inner][INFO] - {"epoch": 2, "update": 1.591, "loss": "1.222", "ntokens": "126.725", "acc_total": "126.725", "n_correct": "102.735", "wer_total": "126.725", "n_error": "23.98", "ppl": "2.33", "accuracy": "81.069", "wer": "18.923", "wps": "69.3", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "24000", "lr": "6.14114e-05", "gnorm": "6.986", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "66178"}
[2024-06-19 12:13:11,013][train_inner][INFO] - {"epoch": 2, "update": 1.605, "loss": "1.218", "ntokens": "126.27", "acc_total": "126.27", "n_correct": "102.69", "wer_total": "126.27", "n_error": "23.565", "ppl": "2.33", "accuracy": "81.326", "wer": "18.662", "wps": "69", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "24200", "lr": "5.9599e-05", "gnorm": "7.163", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "66544"}
[2024-06-19 12:19:17,003][train_inner][INFO] - {"epoch": 2, "update": 1.618, "loss": "1.213", "ntokens": "127.195", "acc_total": "127.195", "n_correct": "103.695", "wer_total": "127.195", "n_error": "23.485", "ppl": "2.32", "accuracy": "81.524", "wer": "18.464", "wps": "69.5", "ups": "0.55", "wpb": "127.2", "bsz": "8", "num_updates": "24400", "lr": "5.784e-05", "gnorm": "7.144", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "66910"}
[2024-06-19 12:25:23,142][train_inner][INFO] - {"epoch": 2, "update": 1.631, "loss": "1.243", "ntokens": "127.34", "acc_total": "127.34", "n_correct": "102.535", "wer_total": "127.34", "n_error": "24.795", "ppl": "2.37", "accuracy": "80.521", "wer": "19.471", "wps": "69.6", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "24600", "lr": "5.6133e-05", "gnorm": "6.807", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "67276"}
[2024-06-19 12:31:29,432][train_inner][INFO] - {"epoch": 2, "update": 1.644, "loss": "1.191", "ntokens": "127.295", "acc_total": "127.295", "n_correct": "103.895", "wer_total": "127.295", "n_error": "23.395", "ppl": "2.28", "accuracy": "81.618", "wer": "18.379", "wps": "69.5", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "24800", "lr": "5.44763e-05", "gnorm": "6.921", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "67642"}
[2024-06-19 12:37:35,563][train_inner][INFO] - {"epoch": 2, "update": 1.658, "loss": "1.228", "ntokens": "125.91", "acc_total": "125.91", "n_correct": "102.165", "wer_total": "125.91", "n_error": "23.73", "ppl": "2.34", "accuracy": "81.141", "wer": "18.847", "wps": "68.8", "ups": "0.55", "wpb": "125.9", "bsz": "8", "num_updates": "25000", "lr": "5.28686e-05", "gnorm": "6.853", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "68008"}
[2024-06-19 12:37:35,564][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 13:19:39,346][valid][INFO] - {"epoch": 2, "valid_loss": "1.051", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.2911", "valid_wer_total": "18.1585", "valid_n_error": "2.86574", "valid_ppl": "2.07", "valid_accuracy": "84.209", "valid_wer": "15.782", "valid_wps": "172.6", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "25000", "valid_best_accuracy": "84.209"}
[2024-06-19 13:19:39,346][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 25000 updates
[2024-06-19 13:19:39,347][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_25000.pt
[2024-06-19 13:19:42,535][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_25000.pt
[2024-06-19 13:19:46,871][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_25000.pt (epoch 2 @ 25000 updates, score 84.209) (writing took 7.524892613990232 seconds)
[2024-06-19 13:25:52,678][train_inner][INFO] - {"epoch": 2, "update": 1.671, "loss": "1.132", "ntokens": "127.435", "acc_total": "127.435", "n_correct": "104.055", "wer_total": "127.435", "n_error": "23.375", "ppl": "2.19", "accuracy": "81.653", "wer": "18.343", "wps": "8.8", "ups": "0.07", "wpb": "127.4", "bsz": "8", "num_updates": "25200", "lr": "5.13083e-05", "gnorm": "6.689", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "70906"}
[2024-06-19 13:31:58,972][train_inner][INFO] - {"epoch": 2, "update": 1.684, "loss": "1.174", "ntokens": "125.305", "acc_total": "125.305", "n_correct": "101.08", "wer_total": "125.305", "n_error": "24.21", "ppl": "2.26", "accuracy": "80.667", "wer": "19.321", "wps": "68.4", "ups": "0.55", "wpb": "125.3", "bsz": "8", "num_updates": "25400", "lr": "4.9794e-05", "gnorm": "7.077", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "71272"}
[2024-06-19 13:38:04,878][train_inner][INFO] - {"epoch": 2, "update": 1.697, "loss": "1.123", "ntokens": "127.065", "acc_total": "127.065", "n_correct": "103.36", "wer_total": "127.065", "n_error": "23.7", "ppl": "2.18", "accuracy": "81.344", "wer": "18.652", "wps": "69.5", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "25600", "lr": "4.83244e-05", "gnorm": "6.64", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "71638"}
[2024-06-19 13:44:10,715][train_inner][INFO] - {"epoch": 2, "update": 1.711, "loss": "1.122", "ntokens": "127.065", "acc_total": "127.065", "n_correct": "103.77", "wer_total": "127.065", "n_error": "23.29", "ppl": "2.18", "accuracy": "81.667", "wer": "18.329", "wps": "69.5", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "25800", "lr": "4.68982e-05", "gnorm": "6.777", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "72004"}
[2024-06-19 13:50:16,906][train_inner][INFO] - {"epoch": 2, "update": 1.724, "loss": "1.184", "ntokens": "126.995", "acc_total": "126.995", "n_correct": "101.8", "wer_total": "126.995", "n_error": "25.19", "ppl": "2.27", "accuracy": "80.161", "wer": "19.835", "wps": "69.4", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "26000", "lr": "4.55141e-05", "gnorm": "7.005", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "72370"}
[2024-06-19 13:56:23,012][train_inner][INFO] - {"epoch": 2, "update": 1.737, "loss": "1.163", "ntokens": "126.31", "acc_total": "126.31", "n_correct": "101.655", "wer_total": "126.31", "n_error": "24.64", "ppl": "2.24", "accuracy": "80.481", "wer": "19.508", "wps": "69", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "26200", "lr": "4.41708e-05", "gnorm": "7.064", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "72736"}
[2024-06-19 14:02:29,156][train_inner][INFO] - {"epoch": 2, "update": 1.75, "loss": "1.143", "ntokens": "127.37", "acc_total": "127.37", "n_correct": "103.005", "wer_total": "127.37", "n_error": "24.35", "ppl": "2.21", "accuracy": "80.871", "wer": "19.118", "wps": "69.6", "ups": "0.55", "wpb": "127.4", "bsz": "8", "num_updates": "26400", "lr": "4.28672e-05", "gnorm": "6.744", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "73102"}
[2024-06-19 14:08:35,717][train_inner][INFO] - {"epoch": 2, "update": 1.764, "loss": "1.11", "ntokens": "126.69", "acc_total": "126.69", "n_correct": "104.61", "wer_total": "126.69", "n_error": "22.075", "ppl": "2.16", "accuracy": "82.572", "wer": "17.424", "wps": "69.1", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "26600", "lr": "4.16021e-05", "gnorm": "6.893", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "73469"}
[2024-06-19 14:14:41,884][train_inner][INFO] - {"epoch": 2, "update": 1.777, "loss": "1.103", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "104.47", "wer_total": "126.605", "n_error": "22.135", "ppl": "2.15", "accuracy": "82.516", "wer": "17.484", "wps": "69.2", "ups": "0.55", "wpb": "126.6", "bsz": "8", "num_updates": "26800", "lr": "4.03743e-05", "gnorm": "6.734", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "73835"}
[2024-06-19 14:20:48,331][train_inner][INFO] - {"epoch": 2, "update": 1.79, "loss": "1.172", "ntokens": "127.595", "acc_total": "127.595", "n_correct": "103.955", "wer_total": "127.595", "n_error": "23.64", "ppl": "2.25", "accuracy": "81.473", "wer": "18.527", "wps": "69.6", "ups": "0.55", "wpb": "127.6", "bsz": "8", "num_updates": "27000", "lr": "3.91827e-05", "gnorm": "6.716", "loss_scale": "2048", "train_wall": "366", "gb_free": "6.5", "wall": "74201"}
[2024-06-19 14:26:54,751][train_inner][INFO] - {"epoch": 2, "update": 1.803, "loss": "1.163", "ntokens": "127.94", "acc_total": "127.94", "n_correct": "105.69", "wer_total": "127.94", "n_error": "22.245", "ppl": "2.24", "accuracy": "82.609", "wer": "17.387", "wps": "69.8", "ups": "0.55", "wpb": "127.9", "bsz": "8", "num_updates": "27200", "lr": "3.80263e-05", "gnorm": "6.77", "loss_scale": "2048", "train_wall": "366", "gb_free": "6.5", "wall": "74568"}
[2024-06-19 14:33:01,264][train_inner][INFO] - {"epoch": 2, "update": 1.817, "loss": "1.163", "ntokens": "126.68", "acc_total": "126.68", "n_correct": "104.845", "wer_total": "126.68", "n_error": "21.83", "ppl": "2.24", "accuracy": "82.764", "wer": "17.232", "wps": "69.1", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "27400", "lr": "3.6904e-05", "gnorm": "6.677", "loss_scale": "2048", "train_wall": "366", "gb_free": "6.5", "wall": "74934"}
[2024-06-19 14:36:04,500][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 15:18:12,409][valid][INFO] - {"epoch": 2, "valid_loss": "1.004", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.3571", "valid_wer_total": "18.1585", "valid_n_error": "2.79987", "valid_ppl": "2", "valid_accuracy": "84.573", "valid_wer": "15.419", "valid_wps": "172.3", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "27500", "valid_best_accuracy": "84.573"}
[2024-06-19 15:18:12,410][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 27500 updates
[2024-06-19 15:18:12,410][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_27500.pt
[2024-06-19 15:18:15,645][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_27500.pt
[2024-06-19 15:18:19,913][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_27500.pt (epoch 2 @ 27500 updates, score 84.573) (writing took 7.502803003997542 seconds)
[2024-06-19 15:21:22,773][train_inner][INFO] - {"epoch": 2, "update": 1.83, "loss": "1.129", "ntokens": "126.45", "acc_total": "126.45", "n_correct": "103.92", "wer_total": "126.45", "n_error": "22.525", "ppl": "2.19", "accuracy": "82.183", "wer": "17.813", "wps": "8.7", "ups": "0.07", "wpb": "126.5", "bsz": "8", "num_updates": "27600", "lr": "3.58149e-05", "gnorm": "7.135", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "77836"}
[2024-06-19 15:27:29,454][train_inner][INFO] - {"epoch": 2, "update": 1.843, "loss": "1.136", "ntokens": "127.465", "acc_total": "127.465", "n_correct": "105.14", "wer_total": "127.465", "n_error": "22.32", "ppl": "2.2", "accuracy": "82.485", "wer": "17.511", "wps": "69.5", "ups": "0.55", "wpb": "127.5", "bsz": "8", "num_updates": "27800", "lr": "3.47579e-05", "gnorm": "6.766", "loss_scale": "2048", "train_wall": "366", "gb_free": "6.5", "wall": "78202"}
[2024-06-19 15:33:35,895][train_inner][INFO] - {"epoch": 2, "update": 1.856, "loss": "1.149", "ntokens": "127.49", "acc_total": "127.49", "n_correct": "104.665", "wer_total": "127.49", "n_error": "22.81", "ppl": "2.22", "accuracy": "82.097", "wer": "17.892", "wps": "69.6", "ups": "0.55", "wpb": "127.5", "bsz": "8", "num_updates": "28000", "lr": "3.37321e-05", "gnorm": "6.677", "loss_scale": "2048", "train_wall": "366", "gb_free": "6.5", "wall": "78569"}
[2024-06-19 15:35:44,133][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-06-19 15:39:44,181][train_inner][INFO] - {"epoch": 2, "update": 1.87, "loss": "1.11", "ntokens": "127.695", "acc_total": "127.695", "n_correct": "106.255", "wer_total": "127.695", "n_error": "21.435", "ppl": "2.16", "accuracy": "83.21", "wer": "16.786", "wps": "69.3", "ups": "0.54", "wpb": "127.7", "bsz": "8", "num_updates": "28200", "lr": "3.27365e-05", "gnorm": "6.869", "loss_scale": "1024", "train_wall": "368", "gb_free": "6.5", "wall": "78937"}
[2024-06-19 15:45:50,818][train_inner][INFO] - {"epoch": 2, "update": 1.883, "loss": "1.124", "ntokens": "127.145", "acc_total": "127.145", "n_correct": "105.445", "wer_total": "127.145", "n_error": "21.69", "ppl": "2.18", "accuracy": "82.933", "wer": "17.059", "wps": "69.4", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "28400", "lr": "3.17704e-05", "gnorm": "6.864", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "79304"}
[2024-06-19 15:51:57,276][train_inner][INFO] - {"epoch": 2, "update": 1.896, "loss": "1.133", "ntokens": "128.11", "acc_total": "128.11", "n_correct": "106.475", "wer_total": "128.11", "n_error": "21.62", "ppl": "2.19", "accuracy": "83.112", "wer": "16.876", "wps": "69.9", "ups": "0.55", "wpb": "128.1", "bsz": "8", "num_updates": "28600", "lr": "3.08327e-05", "gnorm": "6.687", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "79670"}
[2024-06-19 15:58:03,819][train_inner][INFO] - {"epoch": 2, "update": 1.91, "loss": "1.106", "ntokens": "126.685", "acc_total": "126.685", "n_correct": "106.015", "wer_total": "126.685", "n_error": "20.67", "ppl": "2.15", "accuracy": "83.684", "wer": "16.316", "wps": "69.1", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "28800", "lr": "2.99228e-05", "gnorm": "6.797", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "80037"}
[2024-06-19 16:04:10,190][train_inner][INFO] - {"epoch": 2, "update": 1.923, "loss": "1.078", "ntokens": "127.77", "acc_total": "127.77", "n_correct": "107.28", "wer_total": "127.77", "n_error": "20.48", "ppl": "2.11", "accuracy": "83.963", "wer": "16.029", "wps": "69.7", "ups": "0.55", "wpb": "127.8", "bsz": "8", "num_updates": "29000", "lr": "2.90397e-05", "gnorm": "6.698", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "80403"}
[2024-06-19 16:10:16,470][train_inner][INFO] - {"epoch": 2, "update": 1.936, "loss": "1.114", "ntokens": "126.785", "acc_total": "126.785", "n_correct": "105.44", "wer_total": "126.785", "n_error": "21.34", "ppl": "2.16", "accuracy": "83.164", "wer": "16.832", "wps": "69.2", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "29200", "lr": "2.81826e-05", "gnorm": "6.88", "loss_scale": "1024", "train_wall": "366", "gb_free": "6.5", "wall": "80769"}
[2024-06-19 16:16:22,324][train_inner][INFO] - {"epoch": 2, "update": 1.949, "loss": "1.119", "ntokens": "127.185", "acc_total": "127.185", "n_correct": "106.415", "wer_total": "127.185", "n_error": "20.77", "ppl": "2.17", "accuracy": "83.669", "wer": "16.331", "wps": "69.5", "ups": "0.55", "wpb": "127.2", "bsz": "8", "num_updates": "29400", "lr": "2.73509e-05", "gnorm": "6.946", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "81135"}
[2024-06-19 16:22:28,034][train_inner][INFO] - {"epoch": 2, "update": 1.963, "loss": "1.087", "ntokens": "127.1", "acc_total": "127.1", "n_correct": "106.29", "wer_total": "127.1", "n_error": "20.81", "ppl": "2.12", "accuracy": "83.627", "wer": "16.373", "wps": "69.5", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "29600", "lr": "2.65436e-05", "gnorm": "6.781", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "81501"}
[2024-06-19 16:28:33,697][train_inner][INFO] - {"epoch": 2, "update": 1.976, "loss": "1.111", "ntokens": "126.785", "acc_total": "126.785", "n_correct": "105.535", "wer_total": "126.785", "n_error": "21.245", "ppl": "2.16", "accuracy": "83.239", "wer": "16.757", "wps": "69.3", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "29800", "lr": "2.57603e-05", "gnorm": "6.764", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "81867"}
[2024-06-19 16:34:39,252][train_inner][INFO] - {"epoch": 2, "update": 1.989, "loss": "1.111", "ntokens": "126.96", "acc_total": "126.96", "n_correct": "106.05", "wer_total": "126.96", "n_error": "20.9", "ppl": "2.16", "accuracy": "83.53", "wer": "16.462", "wps": "69.5", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "30000", "lr": "2.5e-05", "gnorm": "6.606", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "82232"}
[2024-06-19 16:34:39,253][fairseq_cli.train][INFO] - Stopping training due to num_updates: 30000 >= max_update: 30000
[2024-06-19 16:34:39,253][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-06-19 17:16:37,397][valid][INFO] - {"epoch": 2, "valid_loss": "0.963", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.5373", "valid_wer_total": "18.1585", "valid_n_error": "2.61988", "valid_ppl": "1.95", "valid_accuracy": "85.565", "valid_wer": "14.428", "valid_wps": "173", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "30000", "valid_best_accuracy": "85.565"}
[2024-06-19 17:16:37,398][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 30000 updates
[2024-06-19 17:16:37,398][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_30000.pt
[2024-06-19 17:16:40,620][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_30000.pt
[2024-06-19 17:16:45,029][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_30000.pt (epoch 2 @ 30000 updates, score 85.565) (writing took 7.631447505031247 seconds)
[2024-06-19 17:16:45,061][fairseq_cli.train][INFO] - end of epoch 2 (average epoch stats below)
[2024-06-19 17:16:45,063][train][INFO] - {"epoch": 2, "train_loss": "1.326", "train_ntokens": "126.908", "train_acc_total": "126.908", "train_n_correct": "101.031", "train_wer_total": "126.908", "train_n_error": "25.8609", "train_ppl": "2.51", "train_accuracy": "79.61", "train_wer": "20.378", "train_wps": "45.1", "train_ups": "0.36", "train_wpb": "126.9", "train_bsz": "8", "train_num_updates": "30000", "train_lr": "2.5e-05", "train_gnorm": "6.55", "train_loss_scale": "1024", "train_train_wall": "26705", "train_gb_free": "6.5", "train_wall": "84758"}
[2024-06-19 17:16:45,063][fairseq_cli.train][INFO] - done training in 84756.3 seconds