hydra_train.log

[2024-06-30 23:31:59,068][fairseq_cli.train][INFO] - {'_name': None, 'common': {'_name': None, 'no_progress_bar': False, 'log_interval': 200, 'log_format': 'json', 'log_file': None, 'tensorboard_logdir': 'tblog', 'wandb_project': 'AVSP-LLM', 'azureml_logging': False, 'seed': 1337, 'cpu': False, 'tpu': False, 'bf16': False, 'memory_efficient_bf16': False, 'fp16': True, 'memory_efficient_fp16': False, 'fp16_no_flatten_grads': False, 'fp16_init_scale': 128, 'fp16_scale_window': None, 'fp16_scale_tolerance': 0.0, 'on_cpu_convert_precision': False, 'min_loss_scale': 0.0001, 'threshold_loss_scale': None, 'amp': False, 'amp_batch_retries': 2, 'amp_init_scale': 128, 'amp_scale_window': None, 'user_dir': '/home/theodore/Projects/VSP-LLM/src', 'empty_cache_freq': 0, 'all_gather_list_size': 16384, 'model_parallel_size': 1, 'quantization_config_path': None, 'profile': False, 'reset_logging': False, 'suppress_crashes': False, 'use_plasma_view': False, 'plasma_path': '/tmp/plasma'}, 'common_eval': {'_name': None, 'path': None, 'post_process': None, 'quiet': False, 'model_overrides': '{}', 'results_path': None}, 'distributed_training': {'_name': None, 'distributed_world_size': 1, 'distributed_num_procs': 1, 'distributed_rank': 0, 'distributed_backend': 'nccl', 'distributed_init_method': None, 'distributed_port': -1, 'device_id': 0, 'distributed_no_spawn': False, 'ddp_backend': 'no_c10d', 'ddp_comm_hook': 'none', 'bucket_cap_mb': 25, 'fix_batches_to_gpus': False, 'find_unused_parameters': True, 'fast_stat_sync': False, 'heartbeat_timeout': -1, 'broadcast_buffers': False, 'slowmo_momentum': None, 'slowmo_algorithm': 'LocalSGD', 'localsgd_frequency': 3, 'nprocs_per_node': 1, 'pipeline_model_parallel': False, 'pipeline_balance': None, 'pipeline_devices': None, 'pipeline_chunks': 0, 'pipeline_encoder_balance': None, 'pipeline_encoder_devices': None, 'pipeline_decoder_balance': None, 'pipeline_decoder_devices': None, 'pipeline_checkpoint': 'never', 'zero_sharding': 'none', 'fp16': True, 'memory_efficient_fp16': False, 'tpu': False, 'no_reshard_after_forward': False, 'fp32_reduce_scatter': False, 'cpu_offload': False, 'use_sharded_state': False}, 'dataset': {'_name': None, 'num_workers': 0, 'skip_invalid_size_inputs_valid_test': False, 'max_tokens': None, 'batch_size': 1, 'required_batch_size_multiple': 8, 'required_seq_len_multiple': 1, 'dataset_impl': None, 'data_buffer_size': 10, 'train_subset': 'train', 'valid_subset': 'valid', 'combine_valid_subsets': None, 'ignore_unused_valid_subsets': False, 'validate_interval': 1, 'validate_interval_updates': 0, 'validate_after_updates': 0, 'fixed_validation_seed': None, 'disable_validation': False, 'max_tokens_valid': None, 'batch_size_valid': 1, 'max_valid_steps': None, 'curriculum': 0, 'gen_subset': 'test', 'num_shards': 1, 'shard_id': 0}, 'optimization': {'_name': None, 'max_epoch': 0, 'max_update': 30000, 'stop_time_hours': 0.0, 'clip_norm': 0.0, 'sentence_avg': True, 'update_freq': [8], 'lr': [0.0005], 'stop_min_lr': -1.0, 'use_bmuf': False}, 'checkpoint': {'_name': None, 'save_dir': 'checkpoints', 'restore_file': 'checkpoint_last.pt', 'finetune_from_model': None, 'reset_dataloader': False, 'reset_lr_scheduler': False, 'reset_meters': False, 'reset_optimizer': False, 'optimizer_overrides': '{}', 'save_interval': 1, 'save_interval_updates': 2500, 'keep_interval_updates': 1, 'keep_interval_updates_pattern': -1, 'keep_last_epochs': -1, 'keep_best_checkpoints': -1, 'no_save': False, 'no_epoch_checkpoints': True, 'no_last_checkpoints': False, 'no_save_optimizer_state': False, 'best_checkpoint_metric': 'accuracy', 'maximize_best_checkpoint_metric': True, 'patience': -1, 'checkpoint_suffix': '', 'checkpoint_shard_count': 1, 'load_checkpoint_on_all_dp_ranks': False, 'write_checkpoints_asynchronously': False, 'model_parallel_size': 1}, 'bmuf': {'_name': None, 'block_lr': 1.0, 'block_momentum': 0.875, 'global_sync_iter': 50, 'warmup_iterations': 500, 'use_nbm': False, 'average_sync': False, 'distributed_world_size': 1}, 'generation': {'_name': None, 'beam': 5, 'nbest': 1, 'max_len_a': 0.0, 'max_len_b': 200, 'min_len': 1, 'match_source_len': False, 'unnormalized': False, 'no_early_stop': False, 'no_beamable_mm': False, 'lenpen': 1.0, 'unkpen': 0.0, 'replace_unk': None, 'sacrebleu': False, 'score_reference': False, 'prefix_size': 0, 'no_repeat_ngram_size': 0, 'sampling': False, 'sampling_topk': -1, 'sampling_topp': -1.0, 'constraints': None, 'temperature': 1.0, 'diverse_beam_groups': -1, 'diverse_beam_strength': 0.5, 'diversity_rate': -1.0, 'print_alignment': None, 'print_step': False, 'lm_path': None, 'lm_weight': 0.0, 'iter_decode_eos_penalty': 0.0, 'iter_decode_max_iter': 10, 'iter_decode_force_max_iter': False, 'iter_decode_with_beam': 1, 'iter_decode_with_external_reranker': False, 'retain_iter_history': False, 'retain_dropout': False, 'retain_dropout_modules': None, 'decoding_format': None, 'no_seed_provided': False}, 'eval_lm': {'_name': None, 'output_word_probs': False, 'output_word_stats': False, 'context_window': 0, 'softmax_batch': 9223372036854775807}, 'interactive': {'_name': None, 'buffer_size': 0, 'input': '-'}, 'model': {'_name': 'vsp_llm', 'w2v_path': '/home/theodore/Projects/VSP-LLM/checkpoints/large_vox_iter5.pt', 'llm_ckpt_path': 'vilm/vinallama-2.7b', 'apply_mask': False, 'mask_selection': 'static', 'mask_length': 10, 'mask_other': 0, 'mask_prob': 0.75, 'mask_channel_selection': 'static', 'mask_channel_length': 64, 'mask_channel_other': 0, 'mask_channel_prob': 0.5, 'layerdrop': 0.1, 'dropout': 0.0, 'activation_dropout': 0.1, 'attention_dropout': 0.0, 'feature_grad_mult': 1.0, 'encoder_embed_dim': 1024, 'decoder_embed_dim': 4096, 'freeze_finetune_updates': 18000}, 'task': {'_name': 'vsp_llm_training', 'is_s2s': True, 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h', 'label_dir': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h', 'normalize': True, 'labels': ['wrd'], 'single_target': True, 'fine_tuning': True, 'stack_order_audio': 4, 'max_sample_size': 500, 'modalities': ['video', 'audio'], 'image_aug': True, 'pad_audio': True, 'random_crop': False, 'llm_ckpt_path': 'vilm/vinallama-2.7b'}, 'criterion': {'_name': 'decoder_only_language_modeling_loss', 'report_accuracy': True, 'label_smoothing': 0.1}, 'optimizer': {'_name': 'adam', 'adam_betas': '(0.9,0.98)', 'adam_eps': 1e-08, 'weight_decay': 0.0, 'use_old_adam': False, 'tpu': False, 'lr': [0.0005]}, 'lr_scheduler': {'_name': 'tri_stage', 'warmup_steps': 10000, 'hold_steps': 0, 'decay_steps': 20000, 'phase_ratio': None, 'init_lr_scale': 0.01, 'final_lr_scale': 0.05, 'max_update': 30000, 'lr': [0.0005]}, 'scoring': None, 'bpe': None, 'tokenizer': None, 'job_logging_cfg': {'version': 1, 'formatters': {'simple': {'format': '[%(asctime)s][%(name)s][%(levelname)s] - %(message)s'}}, 'handlers': {'console': {'class': 'logging.StreamHandler', 'formatter': 'simple', 'stream': 'ext://sys.stdout'}, 'file': {'class': 'logging.FileHandler', 'formatter': 'simple', 'filename': 'hydra_train.log'}}, 'root': {'level': 'INFO', 'handlers': ['console', 'file']}, 'disable_existing_loggers': False}}
[2024-06-30 23:31:59,070][src.vsp_llm_training][INFO] - current directory is /home/theodore/Projects/VSP-LLM/experiments/ViAVSP-LLM_v1.7
[2024-06-30 23:31:59,070][src.vsp_llm_training][INFO] - AVHubertPretrainingTask Config {'_name': 'vsp_llm_training', 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h', 'labels': ['wrd'], 'label_dir': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h', 'label_rate': -1, 'sample_rate': 16000, 'llm_ckpt_path': 'vilm/vinallama-2.7b', 'normalize': True, 'enable_padding': False, 'max_sample_size': 500, 'min_sample_size': None, 'max_trim_sample_size': '${task.max_sample_size}', 'single_target': True, 'random_crop': False, 'pad_audio': True, 'pdb': False, 'stack_order_audio': 4, 'skip_verify': False, 'image_aug': True, 'image_crop_size': 88, 'image_mean': 0.421, 'image_std': 0.165, 'modalities': ['video', 'audio'], 'is_s2s': True, 'tokenizer_bpe_name': None, 'tokenizer_bpe_model': None, 'noise_wav': None, 'noise_prob': 0.0, 'noise_snr': '0', 'noise_num': 1, 'fine_tuning': True}
[2024-06-30 23:32:00,799][src.hubert_pretraining][INFO] - current directory is /home/theodore/Projects/VSP-LLM/experiments/ViAVSP-LLM_v1.7
[2024-06-30 23:32:00,799][src.hubert_pretraining][INFO] - AVHubertPretrainingTask Config {'_name': 'av_hubert_pretraining', 'data': '/home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h', 'labels': ['km'], 'label_dir': '/checkpoint/bshi/data/lrs3//video/hubert/stitch-iters/envox-iter4-l12c2000/', 'label_rate': 25, 'sample_rate': 25, 'normalize': True, 'enable_padding': False, 'max_sample_size': 2000, 'min_sample_size': 5, 'max_trim_sample_size': 400, 'single_target': False, 'random_crop': True, 'pad_audio': False, 'pdb': False, 'stack_order_audio': 4, 'skip_verify': False, 'image_aug': True, 'image_crop_size': 88, 'image_mean': 0.421, 'image_std': 0.165, 'modalities': ['audio', 'video'], 'is_s2s': False, 'tokenizer_bpe_name': None, 'tokenizer_bpe_model': None, 'noise_wav': None, 'noise_prob': 0.0, 'noise_snr': '0', 'noise_num': 1, 'fine_tuning': False}
[2024-06-30 23:32:00,802][src.hubert][INFO] - HubertModel Config: {'_name': 'av_hubert', 'label_rate': 25, 'input_modality': '${task.input_modality}', 'extractor_mode': default, 'encoder_layers': 24, 'encoder_embed_dim': 1024, 'encoder_ffn_embed_dim': 4096, 'encoder_attention_heads': 16, 'activation_fn': gelu, 'dropout': 0.0, 'attention_dropout': 0.0, 'activation_dropout': 0.1, 'encoder_layerdrop': 0.1, 'dropout_input': 0.0, 'dropout_features': 0.1, 'final_dim': 256, 'untie_final_proj': True, 'layer_norm_first': True, 'conv_feature_layers': '[(512,10,5)] + [(512,3,2)] * 4 + [(512,2,2)] * 2', 'conv_bias': False, 'logit_temp': 0.1, 'target_glu': False, 'feature_grad_mult': 1.0, 'mask_length_audio': 10, 'mask_prob_audio': 0.8, 'mask_length_image': 5, 'mask_prob_image': 0.3, 'mask_selection': static, 'mask_other': 0.0, 'no_mask_overlap': False, 'mask_min_space': 1, 'mask_channel_length': 10, 'mask_channel_prob': 0.0, 'mask_channel_selection': static, 'mask_channel_other': 0.0, 'no_mask_channel_overlap': False, 'mask_channel_min_space': 1, 'conv_pos': 128, 'conv_pos_groups': 16, 'latent_temp': [2.0, 0.5, 0.999995], 'skip_masked': False, 'skip_nomask': False, 'resnet_relu_type': 'prelu', 'resnet_weights': None, 'sim_type': 'cosine', 'sub_encoder_layers': 0, 'audio_feat_dim': 104, 'modality_dropout': 0.5, 'audio_dropout': 0.5, 'modality_fuse': 'concat', 'selection_type': 'same_seq', 'masking_type': 'input', 'decoder_embed_dim': 768, 'decoder_ffn_embed_dim': 3072, 'decoder_layers': 6, 'decoder_layerdrop': 0.0, 'decoder_attention_heads': 4, 'decoder_learned_pos': False, 'decoder_normalize_before': False, 'no_token_positional_embeddings': False, 'decoder_dropout': 0.1, 'decoder_attention_dropout': 0.1, 'decoder_activation_dropout': 0.0, 'max_target_positions': 2048, 'share_decoder_input_output_embed': False, 'no_scale_embedding': True}
[2024-06-30 23:32:07,707][fairseq_cli.train][INFO] - avhubert_llm_seq2seq_cluster_count(
  (encoder): HubertEncoderWrapper(
    (w2v_model): AVHubertModel(
      (feature_extractor_audio): SubModel(
        (proj): Linear(in_features=104, out_features=1024, bias=True)
      )
      (feature_extractor_video): SubModel(
        (resnet): ResEncoder(
          (frontend3D): Sequential(
            (0): Conv3d(1, 64, kernel_size=(5, 7, 7), stride=(1, 2, 2), padding=(2, 3, 3), bias=False)
            (1): BatchNorm3d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
            (2): PReLU(num_parameters=64)
            (3): MaxPool3d(kernel_size=(1, 3, 3), stride=(1, 2, 2), padding=(0, 1, 1), dilation=1, ceil_mode=False)
          )
          (trunk): ResNet(
            (layer1): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=64)
                (relu2): PReLU(num_parameters=64)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
              (1): BasicBlock(
                (conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=64)
                (relu2): PReLU(num_parameters=64)
                (conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer2): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=128)
                (relu2): PReLU(num_parameters=128)
                (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=128)
                (relu2): PReLU(num_parameters=128)
                (conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer3): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=256)
                (relu2): PReLU(num_parameters=256)
                (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=256)
                (relu2): PReLU(num_parameters=256)
                (conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (layer4): Sequential(
              (0): BasicBlock(
                (conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=512)
                (relu2): PReLU(num_parameters=512)
                (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (downsample): Sequential(
                  (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
                  (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                )
              )
              (1): BasicBlock(
                (conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
                (relu1): PReLU(num_parameters=512)
                (relu2): PReLU(num_parameters=512)
                (conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
                (bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
              )
            )
            (avgpool): AdaptiveAvgPool2d(output_size=1)
          )
        )
        (proj): Linear(in_features=512, out_features=1024, bias=True)
      )
      (post_extract_proj): Linear(in_features=2048, out_features=1024, bias=True)
      (dropout_input): Dropout(p=0.0, inplace=False)
      (dropout_features): Dropout(p=0.1, inplace=False)
      (encoder): TransformerEncoder(
        (pos_conv): Sequential(
          (0): Conv1d(1024, 1024, kernel_size=(128,), stride=(1,), padding=(64,), groups=16)
          (1): SamePad()
          (2): GELU(approximate='none')
        )
        (layers): ModuleList(
          (0-23): 24 x TransformerSentenceEncoderLayer(
            (self_attn): MultiheadAttention(
              (dropout_module): FairseqDropout()
              (k_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (v_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (q_proj): Linear(in_features=1024, out_features=1024, bias=True)
              (out_proj): Linear(in_features=1024, out_features=1024, bias=True)
            )
            (dropout1): Dropout(p=0.0, inplace=False)
            (dropout2): Dropout(p=0.1, inplace=False)
            (dropout3): Dropout(p=0.0, inplace=False)
            (self_attn_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
            (fc1): Linear(in_features=1024, out_features=4096, bias=True)
            (fc2): Linear(in_features=4096, out_features=1024, bias=True)
            (final_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
          )
        )
        (layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
      )
      (layer_norm): LayerNorm((2048,), eps=1e-05, elementwise_affine=True)
      (final_proj): None
    )
  )
  (decoder): PeftModelForCausalLM(
    (base_model): LoraModel(
      (model): LlamaForCausalLM(
        (model): LlamaModel(
          (embed_tokens): Embedding(46304, 2560, padding_idx=0)
          (layers): ModuleList(
            (0-31): 32 x LlamaDecoderLayer(
              (self_attn): LlamaSdpaAttention(
                (q_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (k_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (v_proj): lora.Linear4bit(
                  (base_layer): Linear4bit(in_features=2560, out_features=2560, bias=False)
                  (lora_dropout): ModuleDict(
                    (default): Dropout(p=0.05, inplace=False)
                  )
                  (lora_A): ModuleDict(
                    (default): Linear(in_features=2560, out_features=16, bias=False)
                  )
                  (lora_B): ModuleDict(
                    (default): Linear(in_features=16, out_features=2560, bias=False)
                  )
                  (lora_embedding_A): ParameterDict()
                  (lora_embedding_B): ParameterDict()
                )
                (o_proj): Linear4bit(in_features=2560, out_features=2560, bias=False)
                (rotary_emb): LlamaRotaryEmbedding()
              )
              (mlp): LlamaMLP(
                (gate_proj): Linear4bit(in_features=2560, out_features=6912, bias=False)
                (up_proj): Linear4bit(in_features=2560, out_features=6912, bias=False)
                (down_proj): Linear4bit(in_features=6912, out_features=2560, bias=False)
                (act_fn): SiLU()
              )
              (input_layernorm): LlamaRMSNorm()
              (post_attention_layernorm): LlamaRMSNorm()
            )
          )
          (norm): LlamaRMSNorm()
        )
        (lm_head): Linear(in_features=2560, out_features=46304, bias=False)
      )
    )
  )
  (avfeat_to_llm): Linear(in_features=1024, out_features=2560, bias=True)
)
[2024-06-30 23:32:07,713][fairseq_cli.train][INFO] - task: VSP_LLM_TrainingTask
[2024-06-30 23:32:07,713][fairseq_cli.train][INFO] - model: avhubert_llm_seq2seq_cluster_count
[2024-06-30 23:32:07,713][fairseq_cli.train][INFO] - criterion: decoder_only_language_modeling_loss
[2024-06-30 23:32:07,716][fairseq_cli.train][INFO] - num. shared model params: 1,841,644,264 (num. trained: 335,624,424)
[2024-06-30 23:32:07,719][fairseq_cli.train][INFO] - num. expert model params: 0 (num. trained: 0)
[2024-06-30 23:32:07,719][src.vsp_llm_training][INFO] - Using tokenizer
[2024-06-30 23:32:07,756][src.vsp_llm_dataset][INFO] - max_keep=500, min_keep=None, loaded 23990, skipped 0 short and 0 long and 0 unaligned, longest-loaded=76, shortest-loaded=76
[2024-06-30 23:32:08,151][src.vsp_llm_dataset][INFO] - /home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h/valid.wrd is sequence label. skipped
[2024-06-30 23:32:08,151][src.vsp_llm_dataset][INFO] - image transform: Compose(
    Normalize(mean=0.0, std=255.0)
    <src.utils_vsp_llm.CenterCrop object at 0x71d36702c940>
    Normalize(mean=0.421, std=0.165)
)
[2024-06-30 23:32:08,151][src.vsp_llm_dataset][INFO] - pad_audio=True, random_crop=False, normalize=True, max_sample_size=500, seqs2seq data=True,
[2024-06-30 23:32:08,151][src.vsp_llm_dataset][INFO] - Noise wav: None->0 wav, Prob: 0.0, SNR: 0, Number of mixture: 1
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.0.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.0.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.1.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer1.1.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.0.downsample.0.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.1.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer2.1.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.0.downsample.0.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.1.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer3.1.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.0.downsample.0.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.1.conv1.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- encoder.w2v_model.feature_extractor_video.resnet.trunk.layer4.1.conv2.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,288][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.self_attn.o_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.gate_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.up_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.0.mlp.down_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.self_attn.o_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.gate_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.up_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.1.mlp.down_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.self_attn.o_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.gate_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.up_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.2.mlp.down_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.self_attn.o_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.gate_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.up_proj.bias
[2024-06-30 23:32:08,289][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.3.mlp.down_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.self_attn.o_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.gate_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.up_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.4.mlp.down_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.self_attn.o_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.gate_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.up_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.5.mlp.down_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.self_attn.o_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.gate_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.up_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.6.mlp.down_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.self_attn.o_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.gate_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.up_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.7.mlp.down_proj.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,290][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.self_attn.o_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.gate_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.up_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.8.mlp.down_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.self_attn.o_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.gate_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.up_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.9.mlp.down_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.self_attn.o_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.gate_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.up_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.10.mlp.down_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.self_attn.o_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.gate_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.up_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.11.mlp.down_proj.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,291][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.self_attn.o_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.gate_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.up_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.12.mlp.down_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.self_attn.o_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.gate_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.up_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.13.mlp.down_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.self_attn.o_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.gate_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.up_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.14.mlp.down_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.self_attn.o_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.gate_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.up_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.15.mlp.down_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.self_attn.o_proj.bias
[2024-06-30 23:32:08,292][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.gate_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.up_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.16.mlp.down_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.self_attn.o_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.gate_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.up_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.17.mlp.down_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.self_attn.o_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.gate_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.up_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.18.mlp.down_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.self_attn.o_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.gate_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.up_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.19.mlp.down_proj.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,293][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.self_attn.o_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.gate_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.up_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.20.mlp.down_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.self_attn.o_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.gate_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.up_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.21.mlp.down_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.self_attn.o_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.gate_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.up_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.22.mlp.down_proj.bias
[2024-06-30 23:32:08,294][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.self_attn.o_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.gate_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.up_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.23.mlp.down_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.self_attn.o_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.gate_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.up_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.24.mlp.down_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.self_attn.o_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.gate_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.up_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.25.mlp.down_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.self_attn.o_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.gate_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.up_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.26.mlp.down_proj.bias
[2024-06-30 23:32:08,295][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.self_attn.o_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.gate_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.up_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.27.mlp.down_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.self_attn.o_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.gate_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.up_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.28.mlp.down_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.self_attn.o_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.gate_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.up_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.29.mlp.down_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.self_attn.o_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.gate_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.up_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.30.mlp.down_proj.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.base_layer.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.lora_A.default.bias
[2024-06-30 23:32:08,296][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.q_proj.lora_B.default.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.base_layer.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.lora_A.default.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.k_proj.lora_B.default.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.base_layer.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.lora_A.default.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.v_proj.lora_B.default.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.self_attn.o_proj.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.gate_proj.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.up_proj.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.model.layers.31.mlp.down_proj.bias
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - detected shared parameter: encoder.w2v_model.feature_extractor_video.resnet.frontend3D.0.bias <- decoder.base_model.model.lm_head.bias
[2024-06-30 23:32:08,297][fairseq.utils][INFO] - ***********************CUDA enviroments for all 1 workers***********************
[2024-06-30 23:32:08,297][fairseq.utils][INFO] - rank   0: capabilities =  8.6  ; total memory = 15.731 GB ; name = NVIDIA RTX A4000                        
[2024-06-30 23:32:08,297][fairseq.utils][INFO] - ***********************CUDA enviroments for all 1 workers***********************
[2024-06-30 23:32:08,297][fairseq_cli.train][INFO] - training on 1 devices (GPUs/TPUs)
[2024-06-30 23:32:08,297][fairseq_cli.train][INFO] - max tokens per device = None and max sentences per device = 1
[2024-06-30 23:32:08,297][fairseq.trainer][INFO] - Preparing to load checkpoint checkpoints/checkpoint_last.pt
[2024-06-30 23:32:08,298][fairseq.trainer][INFO] - No existing checkpoint found checkpoints/checkpoint_last.pt
[2024-06-30 23:32:08,298][fairseq.trainer][INFO] - loading train data for epoch 1
[2024-06-30 23:32:08,298][src.vsp_llm_training][INFO] - Using tokenizer
[2024-06-30 23:32:08,464][src.vsp_llm_dataset][INFO] - max_keep=500, min_keep=None, loaded 120686, skipped 0 short and 0 long and 0 unaligned, longest-loaded=76, shortest-loaded=73
[2024-06-30 23:32:08,825][src.vsp_llm_dataset][INFO] - /home/theodore/Projects/VSP-LLM/data/processed/vasr/audio-visual/100h/train.wrd is sequence label. skipped
[2024-06-30 23:32:08,825][src.vsp_llm_dataset][INFO] - image transform: Compose(
    Normalize(mean=0.0, std=255.0)
    RandomCrop(size=(88, 88))
    <src.utils_vsp_llm.HorizontalFlip object at 0x71d3698d1a90>
    Normalize(mean=0.421, std=0.165)
)
[2024-06-30 23:32:08,825][src.vsp_llm_dataset][INFO] - pad_audio=True, random_crop=False, normalize=True, max_sample_size=500, seqs2seq data=True,
[2024-06-30 23:32:08,825][src.vsp_llm_dataset][INFO] - Noise wav: None->0 wav, Prob: 0.0, SNR: 0, Number of mixture: 1
[2024-06-30 23:32:13,218][fairseq.trainer][INFO] - begin training epoch 1
[2024-06-30 23:32:13,218][fairseq_cli.train][INFO] - Start iterating over samples
[2024-06-30 23:37:40,150][train_inner][INFO] - {"epoch": 1, "update": 0.013, "loss": "7.644", "ntokens": "126.725", "acc_total": "126.725", "n_correct": "18.135", "wer_total": "126.725", "n_error": "108.495", "ppl": "200.04", "accuracy": "14.311", "wer": "85.615", "wps": "77.7", "ups": "0.61", "wpb": "126.7", "bsz": "8", "num_updates": "200", "lr": "1.49e-05", "gnorm": "8.82", "loss_scale": "128", "train_wall": "326", "gb_free": "7.1", "wall": "332"}
[2024-06-30 23:43:08,694][train_inner][INFO] - {"epoch": 1, "update": 0.027, "loss": "6.197", "ntokens": "126.93", "acc_total": "126.93", "n_correct": "25.74", "wer_total": "126.93", "n_error": "100.995", "ppl": "73.38", "accuracy": "20.279", "wer": "79.567", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "400", "lr": "2.48e-05", "gnorm": "3.705", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "660"}
[2024-06-30 23:48:37,026][train_inner][INFO] - {"epoch": 1, "update": 0.04, "loss": "6.071", "ntokens": "127.015", "acc_total": "127.015", "n_correct": "28.725", "wer_total": "127.015", "n_error": "97.99", "ppl": "67.25", "accuracy": "22.615", "wer": "77.148", "wps": "77.4", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "600", "lr": "3.47e-05", "gnorm": "3.976", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "989"}
[2024-06-30 23:54:05,306][train_inner][INFO] - {"epoch": 1, "update": 0.053, "loss": "5.864", "ntokens": "126.865", "acc_total": "126.865", "n_correct": "30.905", "wer_total": "126.865", "n_error": "95.72", "ppl": "58.23", "accuracy": "24.361", "wer": "75.45", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "800", "lr": "4.46e-05", "gnorm": "4.109", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "1317"}
[2024-06-30 23:59:33,611][train_inner][INFO] - {"epoch": 1, "update": 0.066, "loss": "5.933", "ntokens": "127.025", "acc_total": "127.025", "n_correct": "30.61", "wer_total": "127.025", "n_error": "96.185", "ppl": "61.08", "accuracy": "24.098", "wer": "75.721", "wps": "77.4", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "1000", "lr": "5.45e-05", "gnorm": "3.82", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "1645"}
[2024-07-01 00:05:01,816][train_inner][INFO] - {"epoch": 1, "update": 0.08, "loss": "5.88", "ntokens": "127.095", "acc_total": "127.095", "n_correct": "30.93", "wer_total": "127.095", "n_error": "95.885", "ppl": "58.87", "accuracy": "24.336", "wer": "75.444", "wps": "77.4", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "1200", "lr": "6.44e-05", "gnorm": "3.597", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "1974"}
[2024-07-01 00:10:30,224][train_inner][INFO] - {"epoch": 1, "update": 0.093, "loss": "5.722", "ntokens": "127.62", "acc_total": "127.62", "n_correct": "32.305", "wer_total": "127.62", "n_error": "95.09", "ppl": "52.77", "accuracy": "25.313", "wer": "74.51", "wps": "77.7", "ups": "0.61", "wpb": "127.6", "bsz": "8", "num_updates": "1400", "lr": "7.43e-05", "gnorm": "3.403", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "2302"}
[2024-07-01 00:15:58,384][train_inner][INFO] - {"epoch": 1, "update": 0.106, "loss": "5.736", "ntokens": "127.41", "acc_total": "127.41", "n_correct": "32.445", "wer_total": "127.41", "n_error": "94.675", "ppl": "53.3", "accuracy": "25.465", "wer": "74.307", "wps": "77.7", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "1600", "lr": "8.42e-05", "gnorm": "3.19", "loss_scale": "128", "train_wall": "327", "gb_free": "7.1", "wall": "2630"}
[2024-07-01 00:21:26,623][train_inner][INFO] - {"epoch": 1, "update": 0.119, "loss": "5.775", "ntokens": "126.56", "acc_total": "126.56", "n_correct": "32.14", "wer_total": "126.56", "n_error": "94.195", "ppl": "54.77", "accuracy": "25.395", "wer": "74.427", "wps": "77.1", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "1800", "lr": "9.41e-05", "gnorm": "2.963", "loss_scale": "128", "train_wall": "328", "gb_free": "7.1", "wall": "2958"}
[2024-07-01 00:26:54,682][train_inner][INFO] - {"epoch": 1, "update": 0.133, "loss": "5.67", "ntokens": "126.875", "acc_total": "126.875", "n_correct": "33.56", "wer_total": "126.875", "n_error": "93.1", "ppl": "50.91", "accuracy": "26.451", "wer": "73.379", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "2000", "lr": "0.000104", "gnorm": "2.852", "loss_scale": "128", "train_wall": "327", "gb_free": "7.1", "wall": "3286"}
[2024-07-01 00:32:22,793][train_inner][INFO] - {"epoch": 1, "update": 0.146, "loss": "5.578", "ntokens": "128.2", "acc_total": "128.2", "n_correct": "35.58", "wer_total": "128.2", "n_error": "92.325", "ppl": "47.75", "accuracy": "27.754", "wer": "72.016", "wps": "78.1", "ups": "0.61", "wpb": "128.2", "bsz": "8", "num_updates": "2200", "lr": "0.0001139", "gnorm": "2.854", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "3614"}
[2024-07-01 00:37:50,905][train_inner][INFO] - {"epoch": 1, "update": 0.159, "loss": "5.456", "ntokens": "127.775", "acc_total": "127.775", "n_correct": "37.875", "wer_total": "127.775", "n_error": "89.665", "ppl": "43.91", "accuracy": "29.642", "wer": "70.174", "wps": "77.9", "ups": "0.61", "wpb": "127.8", "bsz": "8", "num_updates": "2400", "lr": "0.0001238", "gnorm": "3.031", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "3943"}
[2024-07-01 00:40:34,993][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 01:22:17,831][valid][INFO] - {"epoch": 1, "valid_loss": "5.178", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "5.88654", "valid_wer_total": "18.1585", "valid_n_error": "12.25", "valid_ppl": "36.21", "valid_accuracy": "32.418", "valid_wer": "67.462", "valid_wps": "174.1", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "2500"}
[2024-07-01 01:22:17,832][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 2500 updates
[2024-07-01 01:22:17,832][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_2500.pt
[2024-07-01 01:22:20,946][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_2500.pt
[2024-07-01 01:22:23,531][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_2500.pt (epoch 1 @ 2500 updates, score 32.418) (writing took 5.698742372002016 seconds)
[2024-07-01 01:25:07,129][train_inner][INFO] - {"epoch": 1, "update": 0.172, "loss": "5.233", "ntokens": "126.92", "acc_total": "126.92", "n_correct": "41.015", "wer_total": "126.92", "n_error": "85.635", "ppl": "37.6", "accuracy": "32.316", "wer": "67.472", "wps": "8.9", "ups": "0.07", "wpb": "126.9", "bsz": "8", "num_updates": "2600", "lr": "0.0001337", "gnorm": "3.481", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "6779"}
[2024-07-01 01:30:34,987][train_inner][INFO] - {"epoch": 1, "update": 0.186, "loss": "5.071", "ntokens": "125.685", "acc_total": "125.685", "n_correct": "43.3", "wer_total": "125.685", "n_error": "82.14", "ppl": "33.62", "accuracy": "34.451", "wer": "65.354", "wps": "76.7", "ups": "0.61", "wpb": "125.7", "bsz": "8", "num_updates": "2800", "lr": "0.0001436", "gnorm": "3.882", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "7107"}
[2024-07-01 01:36:02,850][train_inner][INFO] - {"epoch": 1, "update": 0.199, "loss": "4.821", "ntokens": "127.19", "acc_total": "127.19", "n_correct": "48.09", "wer_total": "127.19", "n_error": "78.85", "ppl": "28.26", "accuracy": "37.81", "wer": "61.994", "wps": "77.6", "ups": "0.61", "wpb": "127.2", "bsz": "8", "num_updates": "3000", "lr": "0.0001535", "gnorm": "4.105", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "7435"}
[2024-07-01 01:41:30,773][train_inner][INFO] - {"epoch": 1, "update": 0.212, "loss": "4.542", "ntokens": "126.535", "acc_total": "126.535", "n_correct": "51.565", "wer_total": "126.535", "n_error": "74.735", "ppl": "23.29", "accuracy": "40.752", "wer": "59.063", "wps": "77.2", "ups": "0.61", "wpb": "126.5", "bsz": "8", "num_updates": "3200", "lr": "0.0001634", "gnorm": "4.282", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "7762"}
[2024-07-01 01:46:58,698][train_inner][INFO] - {"epoch": 1, "update": 0.225, "loss": "4.392", "ntokens": "126.53", "acc_total": "126.53", "n_correct": "53.805", "wer_total": "126.53", "n_error": "72.52", "ppl": "20.99", "accuracy": "42.524", "wer": "57.314", "wps": "77.2", "ups": "0.61", "wpb": "126.5", "bsz": "8", "num_updates": "3400", "lr": "0.0001733", "gnorm": "4.409", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "8090"}
[2024-07-01 01:52:26,596][train_inner][INFO] - {"epoch": 1, "update": 0.239, "loss": "4.144", "ntokens": "127.025", "acc_total": "127.025", "n_correct": "56.895", "wer_total": "127.025", "n_error": "69.965", "ppl": "17.68", "accuracy": "44.79", "wer": "55.08", "wps": "77.5", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "3600", "lr": "0.0001832", "gnorm": "4.436", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "8418"}
[2024-07-01 01:57:54,699][train_inner][INFO] - {"epoch": 1, "update": 0.252, "loss": "4.002", "ntokens": "127.35", "acc_total": "127.35", "n_correct": "59.02", "wer_total": "127.35", "n_error": "68.185", "ppl": "16.03", "accuracy": "46.345", "wer": "53.541", "wps": "77.6", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "3800", "lr": "0.0001931", "gnorm": "4.463", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "8746"}
[2024-07-01 02:03:22,701][train_inner][INFO] - {"epoch": 1, "update": 0.265, "loss": "3.874", "ntokens": "127.785", "acc_total": "127.785", "n_correct": "60.29", "wer_total": "127.785", "n_error": "67.35", "ppl": "14.66", "accuracy": "47.181", "wer": "52.706", "wps": "77.9", "ups": "0.61", "wpb": "127.8", "bsz": "8", "num_updates": "4000", "lr": "0.000203", "gnorm": "4.505", "loss_scale": "256", "train_wall": "327", "gb_free": "7.1", "wall": "9074"}
[2024-07-01 02:08:50,689][train_inner][INFO] - {"epoch": 1, "update": 0.278, "loss": "3.712", "ntokens": "126.255", "acc_total": "126.255", "n_correct": "62.235", "wer_total": "126.255", "n_error": "63.835", "ppl": "13.1", "accuracy": "49.293", "wer": "50.56", "wps": "77", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "4200", "lr": "0.0002129", "gnorm": "4.462", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "9402"}
[2024-07-01 02:14:18,681][train_inner][INFO] - {"epoch": 1, "update": 0.292, "loss": "3.605", "ntokens": "125.9", "acc_total": "125.9", "n_correct": "63.15", "wer_total": "125.9", "n_error": "62.645", "ppl": "12.17", "accuracy": "50.159", "wer": "49.758", "wps": "76.8", "ups": "0.61", "wpb": "125.9", "bsz": "8", "num_updates": "4400", "lr": "0.0002228", "gnorm": "4.415", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "9730"}
[2024-07-01 02:19:46,636][train_inner][INFO] - {"epoch": 1, "update": 0.305, "loss": "3.446", "ntokens": "127.885", "acc_total": "127.885", "n_correct": "66.465", "wer_total": "127.885", "n_error": "61.295", "ppl": "10.9", "accuracy": "51.972", "wer": "47.93", "wps": "78", "ups": "0.61", "wpb": "127.9", "bsz": "8", "num_updates": "4600", "lr": "0.0002327", "gnorm": "4.339", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "10058"}
[2024-07-01 02:25:14,669][train_inner][INFO] - {"epoch": 1, "update": 0.318, "loss": "3.373", "ntokens": "126.21", "acc_total": "126.21", "n_correct": "66.275", "wer_total": "126.21", "n_error": "59.83", "ppl": "10.36", "accuracy": "52.512", "wer": "47.405", "wps": "76.9", "ups": "0.61", "wpb": "126.2", "bsz": "8", "num_updates": "4800", "lr": "0.0002426", "gnorm": "4.336", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "10386"}
[2024-07-01 02:30:42,659][train_inner][INFO] - {"epoch": 1, "update": 0.331, "loss": "3.28", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "67.56", "wer_total": "126.87", "n_error": "59.195", "ppl": "9.71", "accuracy": "53.251", "wer": "46.658", "wps": "77.4", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "5000", "lr": "0.0002525", "gnorm": "4.246", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "10714"}
[2024-07-01 02:30:42,660][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 03:12:25,625][valid][INFO] - {"epoch": 1, "valid_loss": "nan", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "10.4304", "valid_wer_total": "18.1585", "valid_n_error": "7.70884", "valid_ppl": "nan", "valid_accuracy": "57.441", "valid_wer": "42.453", "valid_wps": "174", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "5000", "valid_best_accuracy": "57.441"}
[2024-07-01 03:12:25,626][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 5000 updates
[2024-07-01 03:12:25,626][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_5000.pt
[2024-07-01 03:12:28,759][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_5000.pt
[2024-07-01 03:12:33,547][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_5000.pt (epoch 1 @ 5000 updates, score 57.441) (writing took 7.921251633000793 seconds)
[2024-07-01 03:18:01,534][train_inner][INFO] - {"epoch": 1, "update": 0.345, "loss": "3.052", "ntokens": "126.51", "acc_total": "126.51", "n_correct": "70.245", "wer_total": "126.51", "n_error": "56.14", "ppl": "8.29", "accuracy": "55.525", "wer": "44.376", "wps": "8.9", "ups": "0.07", "wpb": "126.5", "bsz": "8", "num_updates": "5200", "lr": "0.0002624", "gnorm": "4.178", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "13553"}
[2024-07-01 03:23:29,671][train_inner][INFO] - {"epoch": 1, "update": 0.358, "loss": "3.082", "ntokens": "127.425", "acc_total": "127.425", "n_correct": "70.32", "wer_total": "127.425", "n_error": "57.02", "ppl": "8.47", "accuracy": "55.185", "wer": "44.748", "wps": "77.7", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "5400", "lr": "0.0002723", "gnorm": "4.262", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "13881"}
[2024-07-01 03:28:57,710][train_inner][INFO] - {"epoch": 1, "update": 0.371, "loss": "2.962", "ntokens": "127.52", "acc_total": "127.52", "n_correct": "71.69", "wer_total": "127.52", "n_error": "55.72", "ppl": "7.79", "accuracy": "56.219", "wer": "43.695", "wps": "77.7", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "5600", "lr": "0.0002822", "gnorm": "4.176", "loss_scale": "512", "train_wall": "327", "gb_free": "7.1", "wall": "14209"}
[2024-07-01 03:34:25,997][train_inner][INFO] - {"epoch": 1, "update": 0.384, "loss": "3.063", "ntokens": "126.59", "acc_total": "126.59", "n_correct": "70.595", "wer_total": "126.59", "n_error": "55.895", "ppl": "8.36", "accuracy": "55.767", "wer": "44.154", "wps": "77.1", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "5800", "lr": "0.0002921", "gnorm": "4.199", "loss_scale": "512", "train_wall": "328", "gb_free": "7.1", "wall": "14538"}
[2024-07-01 03:39:54,324][train_inner][INFO] - {"epoch": 1, "update": 0.398, "loss": "2.9", "ntokens": "127.395", "acc_total": "127.395", "n_correct": "72.73", "wer_total": "127.395", "n_error": "54.535", "ppl": "7.46", "accuracy": "57.09", "wer": "42.808", "wps": "77.6", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "6000", "lr": "0.000302", "gnorm": "4.026", "loss_scale": "512", "train_wall": "328", "gb_free": "7.1", "wall": "14866"}
[2024-07-01 03:45:22,664][train_inner][INFO] - {"epoch": 1, "update": 0.411, "loss": "2.831", "ntokens": "126.995", "acc_total": "126.995", "n_correct": "73.045", "wer_total": "126.995", "n_error": "53.84", "ppl": "7.11", "accuracy": "57.518", "wer": "42.395", "wps": "77.4", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "6200", "lr": "0.0003119", "gnorm": "4.074", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "15194"}
[2024-07-01 03:50:50,690][train_inner][INFO] - {"epoch": 1, "update": 0.424, "loss": "2.868", "ntokens": "126.01", "acc_total": "126.01", "n_correct": "72.48", "wer_total": "126.01", "n_error": "53.41", "ppl": "7.3", "accuracy": "57.519", "wer": "42.386", "wps": "76.8", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "6400", "lr": "0.0003218", "gnorm": "3.961", "loss_scale": "1024", "train_wall": "327", "gb_free": "7.1", "wall": "15522"}
[2024-07-01 03:56:18,860][train_inner][INFO] - {"epoch": 1, "update": 0.437, "loss": "2.782", "ntokens": "126.33", "acc_total": "126.33", "n_correct": "73.625", "wer_total": "126.33", "n_error": "52.615", "ppl": "6.88", "accuracy": "58.28", "wer": "41.649", "wps": "77", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "6600", "lr": "0.0003317", "gnorm": "4.04", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "15851"}
[2024-07-01 04:01:46,971][train_inner][INFO] - {"epoch": 1, "update": 0.451, "loss": "2.729", "ntokens": "126.735", "acc_total": "126.735", "n_correct": "74.375", "wer_total": "126.735", "n_error": "52.255", "ppl": "6.63", "accuracy": "58.685", "wer": "41.232", "wps": "77.3", "ups": "0.61", "wpb": "126.7", "bsz": "8", "num_updates": "6800", "lr": "0.0003416", "gnorm": "3.895", "loss_scale": "1024", "train_wall": "327", "gb_free": "7.1", "wall": "16179"}
[2024-07-01 04:07:15,205][train_inner][INFO] - {"epoch": 1, "update": 0.464, "loss": "2.614", "ntokens": "126.98", "acc_total": "126.98", "n_correct": "76.275", "wer_total": "126.98", "n_error": "50.64", "ppl": "6.12", "accuracy": "60.069", "wer": "39.88", "wps": "77.4", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "7000", "lr": "0.0003515", "gnorm": "3.876", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "16507"}
[2024-07-01 04:12:43,547][train_inner][INFO] - {"epoch": 1, "update": 0.477, "loss": "2.729", "ntokens": "126.97", "acc_total": "126.97", "n_correct": "74.77", "wer_total": "126.97", "n_error": "52.1", "ppl": "6.63", "accuracy": "58.888", "wer": "41.033", "wps": "77.3", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "7200", "lr": "0.0003614", "gnorm": "3.973", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "16835"}
[2024-07-01 04:18:12,060][train_inner][INFO] - {"epoch": 1, "update": 0.491, "loss": "2.532", "ntokens": "127.45", "acc_total": "127.45", "n_correct": "77.53", "wer_total": "127.45", "n_error": "49.78", "ppl": "5.78", "accuracy": "60.832", "wer": "39.058", "wps": "77.6", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "7400", "lr": "0.0003713", "gnorm": "3.884", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "17164"}
[2024-07-01 04:20:56,268][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 05:02:42,285][valid][INFO] - {"epoch": 1, "valid_loss": "2.315", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "11.5925", "valid_wer_total": "18.1585", "valid_n_error": "6.55373", "valid_ppl": "4.98", "valid_accuracy": "63.84", "valid_wer": "36.092", "valid_wps": "173.8", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "7500", "valid_best_accuracy": "63.84"}
[2024-07-01 05:02:42,286][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 7500 updates
[2024-07-01 05:02:42,286][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_7500.pt
[2024-07-01 05:02:45,390][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_7500.pt
[2024-07-01 05:02:49,772][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_7500.pt (epoch 1 @ 7500 updates, score 63.84) (writing took 7.486369587997615 seconds)
[2024-07-01 05:05:33,799][train_inner][INFO] - {"epoch": 1, "update": 0.504, "loss": "2.574", "ntokens": "127.32", "acc_total": "127.32", "n_correct": "76.57", "wer_total": "127.32", "n_error": "50.675", "ppl": "5.95", "accuracy": "60.14", "wer": "39.801", "wps": "9", "ups": "0.07", "wpb": "127.3", "bsz": "8", "num_updates": "7600", "lr": "0.0003812", "gnorm": "3.923", "loss_scale": "1024", "train_wall": "327", "gb_free": "7.1", "wall": "20006"}
[2024-07-01 05:11:02,221][train_inner][INFO] - {"epoch": 1, "update": 0.517, "loss": "2.593", "ntokens": "127.11", "acc_total": "127.11", "n_correct": "76.36", "wer_total": "127.11", "n_error": "50.695", "ppl": "6.03", "accuracy": "60.074", "wer": "39.883", "wps": "77.4", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "7800", "lr": "0.0003911", "gnorm": "3.942", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "20334"}
[2024-07-01 05:16:30,306][train_inner][INFO] - {"epoch": 1, "update": 0.53, "loss": "2.545", "ntokens": "126.875", "acc_total": "126.875", "n_correct": "76.97", "wer_total": "126.875", "n_error": "49.775", "ppl": "5.84", "accuracy": "60.666", "wer": "39.232", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "8000", "lr": "0.000401", "gnorm": "3.859", "loss_scale": "1024", "train_wall": "327", "gb_free": "7.1", "wall": "20662"}
[2024-07-01 05:21:58,648][train_inner][INFO] - {"epoch": 1, "update": 0.544, "loss": "2.511", "ntokens": "126.05", "acc_total": "126.05", "n_correct": "77.055", "wer_total": "126.05", "n_error": "48.92", "ppl": "5.7", "accuracy": "61.131", "wer": "38.81", "wps": "76.8", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "8200", "lr": "0.0004109", "gnorm": "3.898", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "20990"}
[2024-07-01 05:27:27,092][train_inner][INFO] - {"epoch": 1, "update": 0.557, "loss": "2.454", "ntokens": "126.785", "acc_total": "126.785", "n_correct": "78.355", "wer_total": "126.785", "n_error": "48.37", "ppl": "5.48", "accuracy": "61.801", "wer": "38.151", "wps": "77.2", "ups": "0.61", "wpb": "126.8", "bsz": "8", "num_updates": "8400", "lr": "0.0004208", "gnorm": "3.956", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "21319"}
[2024-07-01 05:32:55,563][train_inner][INFO] - {"epoch": 1, "update": 0.57, "loss": "2.468", "ntokens": "126.675", "acc_total": "126.675", "n_correct": "77.635", "wer_total": "126.675", "n_error": "48.94", "ppl": "5.53", "accuracy": "61.287", "wer": "38.634", "wps": "77.1", "ups": "0.61", "wpb": "126.7", "bsz": "8", "num_updates": "8600", "lr": "0.0004307", "gnorm": "3.963", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "21647"}
[2024-07-01 05:38:23,625][train_inner][INFO] - {"epoch": 1, "update": 0.583, "loss": "2.458", "ntokens": "127.09", "acc_total": "127.09", "n_correct": "78.555", "wer_total": "127.09", "n_error": "48.485", "ppl": "5.49", "accuracy": "61.811", "wer": "38.15", "wps": "77.5", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "8800", "lr": "0.0004406", "gnorm": "3.842", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "21975"}
[2024-07-01 05:43:51,579][train_inner][INFO] - {"epoch": 1, "update": 0.597, "loss": "2.475", "ntokens": "127.43", "acc_total": "127.43", "n_correct": "78.835", "wer_total": "127.43", "n_error": "48.5", "ppl": "5.56", "accuracy": "61.865", "wer": "38.06", "wps": "77.7", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "9000", "lr": "0.0004505", "gnorm": "3.896", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "22303"}
[2024-07-01 05:49:19,201][train_inner][INFO] - {"epoch": 1, "update": 0.61, "loss": "2.437", "ntokens": "127.145", "acc_total": "127.145", "n_correct": "78.83", "wer_total": "127.145", "n_error": "48.21", "ppl": "5.42", "accuracy": "62", "wer": "37.917", "wps": "77.6", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "9200", "lr": "0.0004604", "gnorm": "3.921", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "22631"}
[2024-07-01 05:54:47,282][train_inner][INFO] - {"epoch": 1, "update": 0.623, "loss": "2.425", "ntokens": "125.995", "acc_total": "125.995", "n_correct": "78.23", "wer_total": "125.995", "n_error": "47.705", "ppl": "5.37", "accuracy": "62.09", "wer": "37.863", "wps": "76.8", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "9400", "lr": "0.0004703", "gnorm": "3.931", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "22959"}
[2024-07-01 06:00:15,646][train_inner][INFO] - {"epoch": 1, "update": 0.636, "loss": "2.392", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "79.435", "wer_total": "126.87", "n_error": "47.33", "ppl": "5.25", "accuracy": "62.611", "wer": "37.306", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "9600", "lr": "0.0004802", "gnorm": "4.038", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "23287"}
[2024-07-01 06:05:43,893][train_inner][INFO] - {"epoch": 1, "update": 0.65, "loss": "2.378", "ntokens": "125.68", "acc_total": "125.68", "n_correct": "79.37", "wer_total": "125.68", "n_error": "46.215", "ppl": "5.2", "accuracy": "63.152", "wer": "36.772", "wps": "76.6", "ups": "0.61", "wpb": "125.7", "bsz": "8", "num_updates": "9800", "lr": "0.0004901", "gnorm": "4.061", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "23616"}
[2024-07-01 06:11:12,140][train_inner][INFO] - {"epoch": 1, "update": 0.663, "loss": "2.371", "ntokens": "127.56", "acc_total": "127.56", "n_correct": "80.15", "wer_total": "127.56", "n_error": "47.32", "ppl": "5.17", "accuracy": "62.833", "wer": "37.096", "wps": "77.7", "ups": "0.61", "wpb": "127.6", "bsz": "8", "num_updates": "10000", "lr": "0.0005", "gnorm": "3.92", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "23944"}
[2024-07-01 06:11:12,141][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 06:52:57,799][valid][INFO] - {"epoch": 1, "valid_loss": "2.118", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "11.9978", "valid_wer_total": "18.1585", "valid_n_error": "6.1496", "valid_ppl": "4.34", "valid_accuracy": "66.073", "valid_wer": "33.866", "valid_wps": "173.9", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "10000", "valid_best_accuracy": "66.073"}
[2024-07-01 06:52:57,800][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 10000 updates
[2024-07-01 06:52:57,800][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_10000.pt
[2024-07-01 06:53:00,864][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_10000.pt
[2024-07-01 06:53:05,929][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_10000.pt (epoch 1 @ 10000 updates, score 66.073) (writing took 8.128808640001807 seconds)
[2024-07-01 06:58:33,877][train_inner][INFO] - {"epoch": 1, "update": 0.676, "loss": "2.369", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "79.405", "wer_total": "126.605", "n_error": "47.08", "ppl": "5.17", "accuracy": "62.719", "wer": "37.187", "wps": "8.9", "ups": "0.07", "wpb": "126.6", "bsz": "8", "num_updates": "10200", "lr": "0.000485243", "gnorm": "4.033", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "26786"}
[2024-07-01 07:02:02,102][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 2048.0
[2024-07-01 07:03:43,673][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 07:04:04,992][train_inner][INFO] - {"epoch": 1, "update": 0.69, "loss": "2.323", "ntokens": "127.255", "acc_total": "127.255", "n_correct": "80.965", "wer_total": "127.255", "n_error": "46.215", "ppl": "5.01", "accuracy": "63.624", "wer": "36.317", "wps": "76.9", "ups": "0.6", "wpb": "127.3", "bsz": "8", "num_updates": "10400", "lr": "0.000470922", "gnorm": "3.932", "loss_scale": "1024", "train_wall": "330", "gb_free": "7.1", "wall": "27117"}
[2024-07-01 07:09:33,213][train_inner][INFO] - {"epoch": 1, "update": 0.703, "loss": "2.306", "ntokens": "127.92", "acc_total": "127.92", "n_correct": "81.825", "wer_total": "127.92", "n_error": "46.025", "ppl": "4.95", "accuracy": "63.966", "wer": "35.98", "wps": "77.9", "ups": "0.61", "wpb": "127.9", "bsz": "8", "num_updates": "10600", "lr": "0.000457024", "gnorm": "3.737", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "27445"}
[2024-07-01 07:15:01,462][train_inner][INFO] - {"epoch": 1, "update": 0.716, "loss": "2.316", "ntokens": "127.35", "acc_total": "127.35", "n_correct": "82.02", "wer_total": "127.35", "n_error": "45.255", "ppl": "4.98", "accuracy": "64.405", "wer": "35.536", "wps": "77.6", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "10800", "lr": "0.000443536", "gnorm": "3.905", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "27773"}
[2024-07-01 07:20:29,810][train_inner][INFO] - {"epoch": 1, "update": 0.729, "loss": "2.242", "ntokens": "126.115", "acc_total": "126.115", "n_correct": "80.975", "wer_total": "126.115", "n_error": "45.04", "ppl": "4.73", "accuracy": "64.207", "wer": "35.713", "wps": "76.8", "ups": "0.61", "wpb": "126.1", "bsz": "8", "num_updates": "11000", "lr": "0.000430446", "gnorm": "3.738", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "28102"}
[2024-07-01 07:25:58,093][train_inner][INFO] - {"epoch": 1, "update": 0.743, "loss": "2.172", "ntokens": "126.545", "acc_total": "126.545", "n_correct": "85.42", "wer_total": "126.545", "n_error": "41.015", "ppl": "4.5", "accuracy": "67.502", "wer": "32.411", "wps": "77.1", "ups": "0.61", "wpb": "126.5", "bsz": "8", "num_updates": "11200", "lr": "0.000417742", "gnorm": "3.882", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "28430"}
[2024-07-01 07:31:26,345][train_inner][INFO] - {"epoch": 1, "update": 0.756, "loss": "2.181", "ntokens": "125.9", "acc_total": "125.9", "n_correct": "87.305", "wer_total": "125.9", "n_error": "38.57", "ppl": "4.54", "accuracy": "69.345", "wer": "30.635", "wps": "76.7", "ups": "0.61", "wpb": "125.9", "bsz": "8", "num_updates": "11400", "lr": "0.000405413", "gnorm": "3.981", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "28758"}
[2024-07-01 07:36:54,568][train_inner][INFO] - {"epoch": 1, "update": 0.769, "loss": "2.17", "ntokens": "126.905", "acc_total": "126.905", "n_correct": "85.945", "wer_total": "126.905", "n_error": "40.875", "ppl": "4.5", "accuracy": "67.724", "wer": "32.209", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "11600", "lr": "0.000393448", "gnorm": "3.633", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "29086"}
[2024-07-01 07:42:22,888][train_inner][INFO] - {"epoch": 1, "update": 0.782, "loss": "2.117", "ntokens": "126.775", "acc_total": "126.775", "n_correct": "85.42", "wer_total": "126.775", "n_error": "41.3", "ppl": "4.34", "accuracy": "67.379", "wer": "32.577", "wps": "77.2", "ups": "0.61", "wpb": "126.8", "bsz": "8", "num_updates": "11800", "lr": "0.000381836", "gnorm": "3.695", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "29415"}
[2024-07-01 07:47:51,145][train_inner][INFO] - {"epoch": 1, "update": 0.796, "loss": "2.101", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "84.99", "wer_total": "126.605", "n_error": "41.545", "ppl": "4.29", "accuracy": "67.13", "wer": "32.815", "wps": "77.1", "ups": "0.61", "wpb": "126.6", "bsz": "8", "num_updates": "12000", "lr": "0.000370567", "gnorm": "3.591", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "29743"}
[2024-07-01 07:53:19,395][train_inner][INFO] - {"epoch": 1, "update": 0.809, "loss": "2.104", "ntokens": "126.91", "acc_total": "126.91", "n_correct": "85.895", "wer_total": "126.91", "n_error": "40.965", "ppl": "4.3", "accuracy": "67.682", "wer": "32.279", "wps": "77.3", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "12200", "lr": "0.000359631", "gnorm": "3.806", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "30071"}
[2024-07-01 07:58:47,679][train_inner][INFO] - {"epoch": 1, "update": 0.822, "loss": "2.089", "ntokens": "126.95", "acc_total": "126.95", "n_correct": "85.81", "wer_total": "126.95", "n_error": "41.105", "ppl": "4.25", "accuracy": "67.594", "wer": "32.379", "wps": "77.3", "ups": "0.61", "wpb": "127", "bsz": "8", "num_updates": "12400", "lr": "0.000349017", "gnorm": "3.637", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "30399"}
[2024-07-01 08:01:31,878][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 08:43:18,589][valid][INFO] - {"epoch": 1, "valid_loss": "1.813", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.1689", "valid_wer_total": "18.1585", "valid_n_error": "4.98387", "valid_ppl": "3.51", "valid_accuracy": "72.522", "valid_wer": "27.447", "valid_wps": "173.8", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "12500", "valid_best_accuracy": "72.522"}
[2024-07-01 08:43:18,589][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 12500 updates
[2024-07-01 08:43:18,590][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_12500.pt
[2024-07-01 08:43:21,706][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_12500.pt
[2024-07-01 08:43:25,959][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_12500.pt (epoch 1 @ 12500 updates, score 72.522) (writing took 7.369027613996877 seconds)
[2024-07-01 08:46:09,921][train_inner][INFO] - {"epoch": 1, "update": 0.835, "loss": "1.991", "ntokens": "126.58", "acc_total": "126.58", "n_correct": "85.82", "wer_total": "126.58", "n_error": "40.705", "ppl": "3.97", "accuracy": "67.799", "wer": "32.158", "wps": "8.9", "ups": "0.07", "wpb": "126.6", "bsz": "8", "num_updates": "12600", "lr": "0.000338716", "gnorm": "3.664", "loss_scale": "2048", "train_wall": "327", "gb_free": "7.1", "wall": "33242"}
[2024-07-01 08:51:38,545][train_inner][INFO] - {"epoch": 1, "update": 0.849, "loss": "1.981", "ntokens": "127.27", "acc_total": "127.27", "n_correct": "87.42", "wer_total": "127.27", "n_error": "39.825", "ppl": "3.95", "accuracy": "68.689", "wer": "31.292", "wps": "77.5", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "12800", "lr": "0.00032872", "gnorm": "3.538", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "33570"}
[2024-07-01 08:57:07,061][train_inner][INFO] - {"epoch": 1, "update": 0.862, "loss": "2.01", "ntokens": "125.955", "acc_total": "125.955", "n_correct": "88.18", "wer_total": "125.955", "n_error": "37.73", "ppl": "4.03", "accuracy": "70.009", "wer": "29.955", "wps": "76.7", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "13000", "lr": "0.000319018", "gnorm": "3.647", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "33899"}
[2024-07-01 09:02:35,666][train_inner][INFO] - {"epoch": 1, "update": 0.875, "loss": "1.999", "ntokens": "126.89", "acc_total": "126.89", "n_correct": "90.625", "wer_total": "126.89", "n_error": "36.235", "ppl": "4", "accuracy": "71.42", "wer": "28.556", "wps": "77.2", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "13200", "lr": "0.000309603", "gnorm": "3.425", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "34227"}
[2024-07-01 09:08:04,370][train_inner][INFO] - {"epoch": 1, "update": 0.888, "loss": "2.003", "ntokens": "127.49", "acc_total": "127.49", "n_correct": "90.465", "wer_total": "127.49", "n_error": "36.915", "ppl": "4.01", "accuracy": "70.959", "wer": "28.955", "wps": "77.6", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "13400", "lr": "0.000300466", "gnorm": "3.462", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "34556"}
[2024-07-01 09:13:33,055][train_inner][INFO] - {"epoch": 1, "update": 0.902, "loss": "1.904", "ntokens": "126.87", "acc_total": "126.87", "n_correct": "88.44", "wer_total": "126.87", "n_error": "38.375", "ppl": "3.74", "accuracy": "69.709", "wer": "30.247", "wps": "77.2", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "13600", "lr": "0.000291598", "gnorm": "3.55", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "34885"}
[2024-07-01 09:19:01,720][train_inner][INFO] - {"epoch": 1, "update": 0.915, "loss": "1.858", "ntokens": "127.055", "acc_total": "127.055", "n_correct": "88.55", "wer_total": "127.055", "n_error": "38.47", "ppl": "3.62", "accuracy": "69.694", "wer": "30.278", "wps": "77.3", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "13800", "lr": "0.000282992", "gnorm": "3.368", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "35213"}
[2024-07-01 09:24:30,755][train_inner][INFO] - {"epoch": 1, "update": 0.928, "loss": "1.902", "ntokens": "127.135", "acc_total": "127.135", "n_correct": "90.9", "wer_total": "127.135", "n_error": "36.205", "ppl": "3.74", "accuracy": "71.499", "wer": "28.478", "wps": "77.3", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "14000", "lr": "0.00027464", "gnorm": "3.46", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "35542"}
[2024-07-01 09:29:59,635][train_inner][INFO] - {"epoch": 1, "update": 0.941, "loss": "1.844", "ntokens": "126.005", "acc_total": "126.005", "n_correct": "92.73", "wer_total": "126.005", "n_error": "33.24", "ppl": "3.59", "accuracy": "73.592", "wer": "26.38", "wps": "76.6", "ups": "0.61", "wpb": "126", "bsz": "8", "num_updates": "14200", "lr": "0.000266535", "gnorm": "3.252", "loss_scale": "2048", "train_wall": "328", "gb_free": "7.1", "wall": "35871"}
[2024-07-01 09:30:17,667][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 09:35:30,338][train_inner][INFO] - {"epoch": 1, "update": 0.955, "loss": "1.882", "ntokens": "126.935", "acc_total": "126.935", "n_correct": "91.44", "wer_total": "126.935", "n_error": "35.405", "ppl": "3.69", "accuracy": "72.037", "wer": "27.892", "wps": "76.8", "ups": "0.6", "wpb": "126.9", "bsz": "8", "num_updates": "14400", "lr": "0.000258668", "gnorm": "3.355", "loss_scale": "1024", "train_wall": "330", "gb_free": "7.1", "wall": "36202"}
[2024-07-01 09:40:59,494][train_inner][INFO] - {"epoch": 1, "update": 0.968, "loss": "1.802", "ntokens": "127.515", "acc_total": "127.515", "n_correct": "92.09", "wer_total": "127.515", "n_error": "35.4", "ppl": "3.49", "accuracy": "72.219", "wer": "27.761", "wps": "77.5", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "14600", "lr": "0.000251034", "gnorm": "3.278", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "36531"}
[2024-07-01 09:46:28,555][train_inner][INFO] - {"epoch": 1, "update": 0.981, "loss": "1.855", "ntokens": "127.235", "acc_total": "127.235", "n_correct": "92.545", "wer_total": "127.235", "n_error": "34.66", "ppl": "3.62", "accuracy": "72.735", "wer": "27.241", "wps": "77.3", "ups": "0.61", "wpb": "127.2", "bsz": "8", "num_updates": "14800", "lr": "0.000243626", "gnorm": "3.37", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "36860"}
[2024-07-01 09:51:57,798][train_inner][INFO] - {"epoch": 1, "update": 0.994, "loss": "1.823", "ntokens": "127.245", "acc_total": "127.245", "n_correct": "92.74", "wer_total": "127.245", "n_error": "34.45", "ppl": "3.54", "accuracy": "72.883", "wer": "27.074", "wps": "77.3", "ups": "0.61", "wpb": "127.2", "bsz": "8", "num_updates": "15000", "lr": "0.000236435", "gnorm": "3.442", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "37189"}
[2024-07-01 09:51:57,798][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 10:33:50,932][valid][INFO] - {"epoch": 1, "valid_loss": "1.576", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.6453", "valid_wer_total": "18.1585", "valid_n_error": "4.50909", "valid_ppl": "2.98", "valid_accuracy": "75.146", "valid_wer": "24.832", "valid_wps": "173.3", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "15000", "valid_best_accuracy": "75.146"}
[2024-07-01 10:33:50,933][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 15000 updates
[2024-07-01 10:33:50,933][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_1_15000.pt
[2024-07-01 10:33:54,006][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_1_15000.pt
[2024-07-01 10:33:58,350][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_1_15000.pt (epoch 1 @ 15000 updates, score 75.146) (writing took 7.417635685997084 seconds)
[2024-07-01 10:36:14,348][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 11:18:08,689][valid][INFO] - {"epoch": 1, "valid_loss": "1.572", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.7147", "valid_wer_total": "18.1585", "valid_n_error": "4.43952", "valid_ppl": "2.97", "valid_accuracy": "75.528", "valid_wer": "24.449", "valid_wps": "173.3", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "15083", "valid_best_accuracy": "75.528"}
[2024-07-01 11:18:08,690][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 1 @ 15083 updates
[2024-07-01 11:18:08,690][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_best.pt
[2024-07-01 11:18:12,700][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_best.pt
[2024-07-01 11:18:15,133][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_best.pt (epoch 1 @ 15083 updates, score 75.528) (writing took 6.442558657989139 seconds)
[2024-07-01 11:18:15,133][fairseq_cli.train][INFO] - end of epoch 1 (average epoch stats below)
[2024-07-01 11:18:15,138][train][INFO] - {"epoch": 1, "train_loss": "3.227", "train_ntokens": "126.898", "train_acc_total": "126.898", "train_n_correct": "69.1939", "train_wer_total": "126.898", "train_n_error": "57.5848", "train_ppl": "9.36", "train_accuracy": "54.527", "train_wer": "45.379", "train_wps": "45.2", "train_ups": "0.36", "train_wpb": "126.9", "train_bsz": "8", "train_num_updates": "15083", "train_lr": "0.000233514", "train_gnorm": "3.888", "train_loss_scale": "1024", "train_train_wall": "24709", "train_gb_free": "7.1", "train_wall": "42367"}
[2024-07-01 11:18:15,213][fairseq.trainer][INFO] - begin training epoch 2
[2024-07-01 11:18:15,213][fairseq_cli.train][INFO] - Start iterating over samples
[2024-07-01 11:21:27,817][train_inner][INFO] - {"epoch": 2, "update": 1.008, "loss": "1.731", "ntokens": "127.23", "acc_total": "127.23", "n_correct": "93.7", "wer_total": "127.23", "n_error": "33.49", "ppl": "3.32", "accuracy": "73.646", "wer": "26.322", "wps": "4.7", "ups": "0.04", "wpb": "127.2", "bsz": "8", "num_updates": "15200", "lr": "0.000229457", "gnorm": "3.357", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "42560"}
[2024-07-01 11:26:57,129][train_inner][INFO] - {"epoch": 2, "update": 1.021, "loss": "1.673", "ntokens": "126.325", "acc_total": "126.325", "n_correct": "90.775", "wer_total": "126.325", "n_error": "35.525", "ppl": "3.19", "accuracy": "71.858", "wer": "28.122", "wps": "76.7", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "15400", "lr": "0.000222685", "gnorm": "3.145", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "42889"}
[2024-07-01 11:32:26,682][train_inner][INFO] - {"epoch": 2, "update": 1.034, "loss": "1.648", "ntokens": "126.915", "acc_total": "126.915", "n_correct": "91.26", "wer_total": "126.915", "n_error": "35.62", "ppl": "3.13", "accuracy": "71.906", "wer": "28.066", "wps": "77", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "15600", "lr": "0.000216113", "gnorm": "3.208", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "43218"}
[2024-07-01 11:37:56,041][train_inner][INFO] - {"epoch": 2, "update": 1.048, "loss": "1.723", "ntokens": "125.545", "acc_total": "125.545", "n_correct": "90.38", "wer_total": "125.545", "n_error": "35.115", "ppl": "3.3", "accuracy": "71.99", "wer": "27.97", "wps": "76.2", "ups": "0.61", "wpb": "125.5", "bsz": "8", "num_updates": "15800", "lr": "0.000209735", "gnorm": "3.103", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "43548"}
[2024-07-01 11:43:25,781][train_inner][INFO] - {"epoch": 2, "update": 1.061, "loss": "1.683", "ntokens": "127.405", "acc_total": "127.405", "n_correct": "91.22", "wer_total": "127.405", "n_error": "36.13", "ppl": "3.21", "accuracy": "71.598", "wer": "28.358", "wps": "77.3", "ups": "0.61", "wpb": "127.4", "bsz": "8", "num_updates": "16000", "lr": "0.000203545", "gnorm": "3.28", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "43877"}
[2024-07-01 11:48:55,716][train_inner][INFO] - {"epoch": 2, "update": 1.074, "loss": "1.544", "ntokens": "126.915", "acc_total": "126.915", "n_correct": "93.835", "wer_total": "126.915", "n_error": "33.055", "ppl": "2.92", "accuracy": "73.935", "wer": "26.045", "wps": "76.9", "ups": "0.61", "wpb": "126.9", "bsz": "8", "num_updates": "16200", "lr": "0.000197538", "gnorm": "3.034", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "44207"}
[2024-07-01 11:54:25,805][train_inner][INFO] - {"epoch": 2, "update": 1.087, "loss": "1.603", "ntokens": "127.14", "acc_total": "127.14", "n_correct": "96.265", "wer_total": "127.14", "n_error": "30.865", "ppl": "3.04", "accuracy": "75.716", "wer": "24.276", "wps": "77", "ups": "0.61", "wpb": "127.1", "bsz": "8", "num_updates": "16400", "lr": "0.000191708", "gnorm": "3.087", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44537"}
[2024-07-01 11:59:55,659][train_inner][INFO] - {"epoch": 2, "update": 1.101, "loss": "1.588", "ntokens": "127.51", "acc_total": "127.51", "n_correct": "94.44", "wer_total": "127.51", "n_error": "33.03", "ppl": "3.01", "accuracy": "74.065", "wer": "25.904", "wps": "77.3", "ups": "0.61", "wpb": "127.5", "bsz": "8", "num_updates": "16600", "lr": "0.00018605", "gnorm": "3.048", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "44867"}
[2024-07-01 12:05:24,972][train_inner][INFO] - {"epoch": 2, "update": 1.114, "loss": "1.614", "ntokens": "126.405", "acc_total": "126.405", "n_correct": "93.39", "wer_total": "126.405", "n_error": "32.985", "ppl": "3.06", "accuracy": "73.882", "wer": "26.095", "wps": "76.8", "ups": "0.61", "wpb": "126.4", "bsz": "8", "num_updates": "16800", "lr": "0.000180559", "gnorm": "3.092", "loss_scale": "2048", "train_wall": "329", "gb_free": "7.1", "wall": "45197"}
[2024-07-01 12:06:37,390][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 12:10:56,163][train_inner][INFO] - {"epoch": 2, "update": 1.127, "loss": "1.548", "ntokens": "127.875", "acc_total": "127.875", "n_correct": "93.395", "wer_total": "127.875", "n_error": "34.46", "ppl": "2.92", "accuracy": "73.036", "wer": "26.948", "wps": "77.2", "ups": "0.6", "wpb": "127.9", "bsz": "8", "num_updates": "17000", "lr": "0.00017523", "gnorm": "3.046", "loss_scale": "1024", "train_wall": "331", "gb_free": "7.1", "wall": "45528"}
[2024-07-01 12:16:25,508][train_inner][INFO] - {"epoch": 2, "update": 1.14, "loss": "1.576", "ntokens": "126.18", "acc_total": "126.18", "n_correct": "93.405", "wer_total": "126.18", "n_error": "32.73", "ppl": "2.98", "accuracy": "74.025", "wer": "25.939", "wps": "76.6", "ups": "0.61", "wpb": "126.2", "bsz": "8", "num_updates": "17200", "lr": "0.000170059", "gnorm": "2.941", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "45857"}
[2024-07-01 12:21:55,043][train_inner][INFO] - {"epoch": 2, "update": 1.154, "loss": "1.596", "ntokens": "127.295", "acc_total": "127.295", "n_correct": "93.3", "wer_total": "127.295", "n_error": "33.96", "ppl": "3.02", "accuracy": "73.294", "wer": "26.678", "wps": "77.3", "ups": "0.61", "wpb": "127.3", "bsz": "8", "num_updates": "17400", "lr": "0.00016504", "gnorm": "3", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "46187"}
[2024-07-01 12:24:39,770][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 13:06:33,408][valid][INFO] - {"epoch": 2, "valid_loss": "1.45", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "13.7322", "valid_wer_total": "18.1585", "valid_n_error": "4.42193", "valid_ppl": "2.73", "valid_accuracy": "75.624", "valid_wer": "24.352", "valid_wps": "173.3", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "17500", "valid_best_accuracy": "75.624"}
[2024-07-01 13:06:33,409][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 17500 updates
[2024-07-01 13:06:33,409][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_17500.pt
[2024-07-01 13:06:36,531][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_17500.pt
[2024-07-01 13:06:40,993][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_17500.pt (epoch 2 @ 17500 updates, score 75.624) (writing took 7.584205965991714 seconds)
[2024-07-01 13:09:25,353][train_inner][INFO] - {"epoch": 2, "update": 1.167, "loss": "1.596", "ntokens": "126.19", "acc_total": "126.19", "n_correct": "93.73", "wer_total": "126.19", "n_error": "32.42", "ppl": "3.02", "accuracy": "74.277", "wer": "25.691", "wps": "8.9", "ups": "0.07", "wpb": "126.2", "bsz": "8", "num_updates": "17600", "lr": "0.000160169", "gnorm": "3.048", "loss_scale": "1024", "train_wall": "328", "gb_free": "7.1", "wall": "49037"}
[2024-07-01 13:14:54,985][train_inner][INFO] - {"epoch": 2, "update": 1.18, "loss": "1.488", "ntokens": "128.19", "acc_total": "128.19", "n_correct": "95.735", "wer_total": "128.19", "n_error": "32.435", "ppl": "2.81", "accuracy": "74.682", "wer": "25.302", "wps": "77.8", "ups": "0.61", "wpb": "128.2", "bsz": "8", "num_updates": "17800", "lr": "0.000155442", "gnorm": "2.856", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "49367"}
[2024-07-01 13:20:24,570][train_inner][INFO] - {"epoch": 2, "update": 1.193, "loss": "1.567", "ntokens": "126.265", "acc_total": "126.265", "n_correct": "93.405", "wer_total": "126.265", "n_error": "32.855", "ppl": "2.96", "accuracy": "73.975", "wer": "26.021", "wps": "76.6", "ups": "0.61", "wpb": "126.3", "bsz": "8", "num_updates": "18000", "lr": "0.000150854", "gnorm": "2.999", "loss_scale": "1024", "train_wall": "329", "gb_free": "7.1", "wall": "49696"}
[2024-07-01 13:20:33,730][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 512.0
[2024-07-01 13:26:32,345][train_inner][INFO] - {"epoch": 2, "update": 1.207, "loss": "1.605", "ntokens": "127.655", "acc_total": "127.655", "n_correct": "95.585", "wer_total": "127.655", "n_error": "32.045", "ppl": "3.04", "accuracy": "74.878", "wer": "25.103", "wps": "69.4", "ups": "0.54", "wpb": "127.7", "bsz": "8", "num_updates": "18200", "lr": "0.000146402", "gnorm": "9.856", "loss_scale": "512", "train_wall": "367", "gb_free": "6.5", "wall": "50064"}
[2024-07-01 13:32:38,089][train_inner][INFO] - {"epoch": 2, "update": 1.22, "loss": "1.561", "ntokens": "126.285", "acc_total": "126.285", "n_correct": "92.885", "wer_total": "126.285", "n_error": "33.36", "ppl": "2.95", "accuracy": "73.552", "wer": "26.416", "wps": "69.1", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "18400", "lr": "0.000142081", "gnorm": "9.849", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "50430"}
[2024-07-01 13:38:43,616][train_inner][INFO] - {"epoch": 2, "update": 1.233, "loss": "1.534", "ntokens": "126.92", "acc_total": "126.92", "n_correct": "94.61", "wer_total": "126.92", "n_error": "32.275", "ppl": "2.9", "accuracy": "74.543", "wer": "25.429", "wps": "69.4", "ups": "0.55", "wpb": "126.9", "bsz": "8", "num_updates": "18600", "lr": "0.000137888", "gnorm": "8.776", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "50795"}
[2024-07-01 13:44:49,045][train_inner][INFO] - {"epoch": 2, "update": 1.247, "loss": "1.487", "ntokens": "127.115", "acc_total": "127.115", "n_correct": "95.91", "wer_total": "127.115", "n_error": "31.185", "ppl": "2.8", "accuracy": "75.451", "wer": "24.533", "wps": "69.6", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "18800", "lr": "0.000133819", "gnorm": "8.683", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "51161"}
[2024-07-01 13:50:54,432][train_inner][INFO] - {"epoch": 2, "update": 1.26, "loss": "1.44", "ntokens": "127.745", "acc_total": "127.745", "n_correct": "98.35", "wer_total": "127.745", "n_error": "29.375", "ppl": "2.71", "accuracy": "76.989", "wer": "22.995", "wps": "69.9", "ups": "0.55", "wpb": "127.7", "bsz": "8", "num_updates": "19000", "lr": "0.000129869", "gnorm": "8.33", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "51526"}
[2024-07-01 13:57:00,146][train_inner][INFO] - {"epoch": 2, "update": 1.273, "loss": "1.515", "ntokens": "127.085", "acc_total": "127.085", "n_correct": "98.755", "wer_total": "127.085", "n_error": "28.31", "ppl": "2.86", "accuracy": "77.708", "wer": "22.276", "wps": "69.5", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "19200", "lr": "0.000126036", "gnorm": "8.429", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "51892"}
[2024-07-01 14:03:05,959][train_inner][INFO] - {"epoch": 2, "update": 1.286, "loss": "1.369", "ntokens": "125.85", "acc_total": "125.85", "n_correct": "99.37", "wer_total": "125.85", "n_error": "26.465", "ppl": "2.58", "accuracy": "78.959", "wer": "21.029", "wps": "68.8", "ups": "0.55", "wpb": "125.8", "bsz": "8", "num_updates": "19400", "lr": "0.000122317", "gnorm": "8.051", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "52258"}
[2024-07-01 14:09:11,528][train_inner][INFO] - {"epoch": 2, "update": 1.3, "loss": "1.4", "ntokens": "125.665", "acc_total": "125.665", "n_correct": "99.32", "wer_total": "125.665", "n_error": "26.34", "ppl": "2.64", "accuracy": "79.036", "wer": "20.96", "wps": "68.8", "ups": "0.55", "wpb": "125.7", "bsz": "8", "num_updates": "19600", "lr": "0.000118707", "gnorm": "8.728", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "52623"}
[2024-07-01 14:15:16,644][train_inner][INFO] - {"epoch": 2, "update": 1.313, "loss": "1.367", "ntokens": "127.615", "acc_total": "127.615", "n_correct": "100.89", "wer_total": "127.615", "n_error": "26.71", "ppl": "2.58", "accuracy": "79.058", "wer": "20.93", "wps": "69.9", "ups": "0.55", "wpb": "127.6", "bsz": "8", "num_updates": "19800", "lr": "0.000115203", "gnorm": "7.797", "loss_scale": "512", "train_wall": "364", "gb_free": "6.5", "wall": "52988"}
[2024-07-01 14:21:22,177][train_inner][INFO] - {"epoch": 2, "update": 1.326, "loss": "1.404", "ntokens": "126.165", "acc_total": "126.165", "n_correct": "99.54", "wer_total": "126.165", "n_error": "26.61", "ppl": "2.65", "accuracy": "78.897", "wer": "21.091", "wps": "69", "ups": "0.55", "wpb": "126.2", "bsz": "8", "num_updates": "20000", "lr": "0.000111803", "gnorm": "7.856", "loss_scale": "512", "train_wall": "365", "gb_free": "6.5", "wall": "53354"}
[2024-07-01 14:21:22,178][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 15:03:07,557][valid][INFO] - {"epoch": 2, "valid_loss": "1.264", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "14.8014", "valid_wer_total": "18.1585", "valid_n_error": "3.35486", "valid_ppl": "2.4", "valid_accuracy": "81.512", "valid_wer": "18.475", "valid_wps": "173.9", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "20000", "valid_best_accuracy": "81.512"}
[2024-07-01 15:03:07,557][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 20000 updates
[2024-07-01 15:03:07,558][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_20000.pt
[2024-07-01 15:03:10,644][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_20000.pt
[2024-07-01 15:03:14,986][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_20000.pt (epoch 2 @ 20000 updates, score 81.512) (writing took 7.42853797400312 seconds)
[2024-07-01 15:09:17,849][train_inner][INFO] - {"epoch": 2, "update": 1.339, "loss": "1.395", "ntokens": "126.37", "acc_total": "126.37", "n_correct": "99.27", "wer_total": "126.37", "n_error": "27.08", "ppl": "2.63", "accuracy": "78.555", "wer": "21.429", "wps": "8.8", "ups": "0.07", "wpb": "126.4", "bsz": "8", "num_updates": "20200", "lr": "0.000108504", "gnorm": "7.879", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "56230"}
[2024-07-01 15:15:20,925][train_inner][INFO] - {"epoch": 2, "update": 1.353, "loss": "1.393", "ntokens": "127.095", "acc_total": "127.095", "n_correct": "101.5", "wer_total": "127.095", "n_error": "25.585", "ppl": "2.63", "accuracy": "79.862", "wer": "20.131", "wps": "70", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "20400", "lr": "0.000105301", "gnorm": "7.963", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "56593"}
[2024-07-01 15:21:23,860][train_inner][INFO] - {"epoch": 2, "update": 1.366, "loss": "1.354", "ntokens": "126.5", "acc_total": "126.5", "n_correct": "101.94", "wer_total": "126.5", "n_error": "24.54", "ppl": "2.56", "accuracy": "80.585", "wer": "19.399", "wps": "69.7", "ups": "0.55", "wpb": "126.5", "bsz": "8", "num_updates": "20600", "lr": "0.000102194", "gnorm": "7.681", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "56956"}
[2024-07-01 15:27:27,200][train_inner][INFO] - {"epoch": 2, "update": 1.379, "loss": "1.339", "ntokens": "127.265", "acc_total": "127.265", "n_correct": "102.42", "wer_total": "127.265", "n_error": "24.83", "ppl": "2.53", "accuracy": "80.478", "wer": "19.51", "wps": "70.1", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "20800", "lr": "9.91776e-05", "gnorm": "7.449", "loss_scale": "1024", "train_wall": "363", "gb_free": "6.5", "wall": "57319"}
[2024-07-01 15:33:30,099][train_inner][INFO] - {"epoch": 2, "update": 1.392, "loss": "1.321", "ntokens": "127.405", "acc_total": "127.405", "n_correct": "102.74", "wer_total": "127.405", "n_error": "24.64", "ppl": "2.5", "accuracy": "80.64", "wer": "19.34", "wps": "70.2", "ups": "0.55", "wpb": "127.4", "bsz": "8", "num_updates": "21000", "lr": "9.62506e-05", "gnorm": "7.672", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "57682"}
[2024-07-01 15:39:32,909][train_inner][INFO] - {"epoch": 2, "update": 1.406, "loss": "1.302", "ntokens": "126.7", "acc_total": "126.7", "n_correct": "102.91", "wer_total": "126.7", "n_error": "23.775", "ppl": "2.47", "accuracy": "81.223", "wer": "18.765", "wps": "69.8", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "21200", "lr": "9.341e-05", "gnorm": "7.567", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "58045"}
[2024-07-01 15:45:35,747][train_inner][INFO] - {"epoch": 2, "update": 1.419, "loss": "1.275", "ntokens": "127.32", "acc_total": "127.32", "n_correct": "102.585", "wer_total": "127.32", "n_error": "24.73", "ppl": "2.42", "accuracy": "80.573", "wer": "19.423", "wps": "70.2", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "21400", "lr": "9.06532e-05", "gnorm": "7.356", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "58407"}
[2024-07-01 15:51:38,582][train_inner][INFO] - {"epoch": 2, "update": 1.432, "loss": "1.305", "ntokens": "126.805", "acc_total": "126.805", "n_correct": "101.49", "wer_total": "126.805", "n_error": "25.275", "ppl": "2.47", "accuracy": "80.036", "wer": "19.932", "wps": "69.9", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "21600", "lr": "8.79777e-05", "gnorm": "7.567", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "58770"}
[2024-07-01 15:57:41,260][train_inner][INFO] - {"epoch": 2, "update": 1.445, "loss": "1.258", "ntokens": "125.555", "acc_total": "125.555", "n_correct": "102.86", "wer_total": "125.555", "n_error": "22.685", "ppl": "2.39", "accuracy": "81.924", "wer": "18.068", "wps": "69.2", "ups": "0.55", "wpb": "125.6", "bsz": "8", "num_updates": "21800", "lr": "8.53812e-05", "gnorm": "7.711", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "59133"}
[2024-07-01 16:03:44,040][train_inner][INFO] - {"epoch": 2, "update": 1.459, "loss": "1.316", "ntokens": "127.64", "acc_total": "127.64", "n_correct": "103.14", "wer_total": "127.64", "n_error": "24.465", "ppl": "2.49", "accuracy": "80.805", "wer": "19.167", "wps": "70.4", "ups": "0.55", "wpb": "127.6", "bsz": "8", "num_updates": "22000", "lr": "8.28614e-05", "gnorm": "7.574", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "59496"}
[2024-07-01 16:08:41,241][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 16:09:48,345][train_inner][INFO] - {"epoch": 2, "update": 1.472, "loss": "1.22", "ntokens": "126.53", "acc_total": "126.53", "n_correct": "102.48", "wer_total": "126.53", "n_error": "24.04", "ppl": "2.33", "accuracy": "80.993", "wer": "18.999", "wps": "69.5", "ups": "0.55", "wpb": "126.5", "bsz": "8", "num_updates": "22200", "lr": "8.04159e-05", "gnorm": "7.303", "loss_scale": "1024", "train_wall": "364", "gb_free": "6.5", "wall": "59860"}
[2024-07-01 16:15:50,859][train_inner][INFO] - {"epoch": 2, "update": 1.485, "loss": "1.249", "ntokens": "127.075", "acc_total": "127.075", "n_correct": "103.99", "wer_total": "127.075", "n_error": "23.055", "ppl": "2.38", "accuracy": "81.834", "wer": "18.143", "wps": "70.1", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "22400", "lr": "7.80425e-05", "gnorm": "7.341", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "60223"}
[2024-07-01 16:18:52,300][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 17:00:37,931][valid][INFO] - {"epoch": 2, "valid_loss": "1.125", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.2284", "valid_wer_total": "18.1585", "valid_n_error": "2.92897", "valid_ppl": "2.18", "valid_accuracy": "83.864", "valid_wer": "16.13", "valid_wps": "173.9", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "22500", "valid_best_accuracy": "83.864"}
[2024-07-01 17:00:37,932][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 22500 updates
[2024-07-01 17:00:37,932][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_22500.pt
[2024-07-01 17:00:41,070][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_22500.pt
[2024-07-01 17:00:45,516][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_22500.pt (epoch 2 @ 22500 updates, score 83.864) (writing took 7.584021136994124 seconds)
[2024-07-01 17:03:47,822][train_inner][INFO] - {"epoch": 2, "update": 1.498, "loss": "1.268", "ntokens": "126.79", "acc_total": "126.79", "n_correct": "103.63", "wer_total": "126.79", "n_error": "23.14", "ppl": "2.41", "accuracy": "81.734", "wer": "18.251", "wps": "8.8", "ups": "0.07", "wpb": "126.8", "bsz": "8", "num_updates": "22600", "lr": "7.57393e-05", "gnorm": "7.448", "loss_scale": "1024", "train_wall": "363", "gb_free": "6.5", "wall": "63100"}
[2024-07-01 17:09:53,130][train_inner][INFO] - {"epoch": 2, "update": 1.512, "loss": "1.218", "ntokens": "126.58", "acc_total": "126.58", "n_correct": "104.265", "wer_total": "126.58", "n_error": "22.305", "ppl": "2.33", "accuracy": "82.371", "wer": "17.621", "wps": "69.3", "ups": "0.55", "wpb": "126.6", "bsz": "8", "num_updates": "22800", "lr": "7.3504e-05", "gnorm": "7.056", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "63465"}
[2024-07-01 17:15:58,391][train_inner][INFO] - {"epoch": 2, "update": 1.525, "loss": "1.275", "ntokens": "126.155", "acc_total": "126.155", "n_correct": "103.09", "wer_total": "126.155", "n_error": "23.06", "ppl": "2.42", "accuracy": "81.717", "wer": "18.279", "wps": "69.1", "ups": "0.55", "wpb": "126.2", "bsz": "8", "num_updates": "23000", "lr": "7.13346e-05", "gnorm": "7.406", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "63830"}
[2024-07-01 17:22:03,725][train_inner][INFO] - {"epoch": 2, "update": 1.538, "loss": "1.202", "ntokens": "126.97", "acc_total": "126.97", "n_correct": "104.805", "wer_total": "126.97", "n_error": "22.16", "ppl": "2.3", "accuracy": "82.543", "wer": "17.453", "wps": "69.5", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "23200", "lr": "6.92293e-05", "gnorm": "7.117", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "64195"}
[2024-07-01 17:28:09,023][train_inner][INFO] - {"epoch": 2, "update": 1.552, "loss": "1.153", "ntokens": "126.29", "acc_total": "126.29", "n_correct": "105.34", "wer_total": "126.29", "n_error": "20.93", "ppl": "2.22", "accuracy": "83.411", "wer": "16.573", "wps": "69.1", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "23400", "lr": "6.71862e-05", "gnorm": "6.995", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "64561"}
[2024-07-01 17:34:14,453][train_inner][INFO] - {"epoch": 2, "update": 1.565, "loss": "1.279", "ntokens": "126.595", "acc_total": "126.595", "n_correct": "103.04", "wer_total": "126.595", "n_error": "23.55", "ppl": "2.43", "accuracy": "81.393", "wer": "18.603", "wps": "69.3", "ups": "0.55", "wpb": "126.6", "bsz": "8", "num_updates": "23600", "lr": "6.52033e-05", "gnorm": "7.179", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "64926"}
[2024-07-01 17:40:19,587][train_inner][INFO] - {"epoch": 2, "update": 1.578, "loss": "1.227", "ntokens": "127.175", "acc_total": "127.175", "n_correct": "103.645", "wer_total": "127.175", "n_error": "23.51", "ppl": "2.34", "accuracy": "81.498", "wer": "18.486", "wps": "69.7", "ups": "0.55", "wpb": "127.2", "bsz": "8", "num_updates": "23800", "lr": "6.3279e-05", "gnorm": "7.007", "loss_scale": "1024", "train_wall": "364", "gb_free": "6.5", "wall": "65291"}
[2024-07-01 17:46:24,974][train_inner][INFO] - {"epoch": 2, "update": 1.591, "loss": "1.214", "ntokens": "126.775", "acc_total": "126.775", "n_correct": "104.22", "wer_total": "126.775", "n_error": "22.555", "ppl": "2.32", "accuracy": "82.209", "wer": "17.791", "wps": "69.4", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "24000", "lr": "6.14114e-05", "gnorm": "7.285", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "65657"}
[2024-07-01 17:52:30,281][train_inner][INFO] - {"epoch": 2, "update": 1.605, "loss": "1.194", "ntokens": "126.17", "acc_total": "126.17", "n_correct": "103.575", "wer_total": "126.17", "n_error": "22.585", "ppl": "2.29", "accuracy": "82.092", "wer": "17.9", "wps": "69.1", "ups": "0.55", "wpb": "126.2", "bsz": "8", "num_updates": "24200", "lr": "5.9599e-05", "gnorm": "6.776", "loss_scale": "1024", "train_wall": "365", "gb_free": "6.5", "wall": "66022"}
[2024-07-01 17:58:35,521][train_inner][INFO] - {"epoch": 2, "update": 1.618, "loss": "1.201", "ntokens": "127.2", "acc_total": "127.2", "n_correct": "104.3", "wer_total": "127.2", "n_error": "22.885", "ppl": "2.3", "accuracy": "81.997", "wer": "17.991", "wps": "69.7", "ups": "0.55", "wpb": "127.2", "bsz": "8", "num_updates": "24400", "lr": "5.784e-05", "gnorm": "7.333", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "66387"}
[2024-07-01 18:04:40,897][train_inner][INFO] - {"epoch": 2, "update": 1.631, "loss": "1.254", "ntokens": "127.38", "acc_total": "127.38", "n_correct": "101.805", "wer_total": "127.38", "n_error": "25.57", "ppl": "2.38", "accuracy": "79.922", "wer": "20.074", "wps": "69.7", "ups": "0.55", "wpb": "127.4", "bsz": "8", "num_updates": "24600", "lr": "5.6133e-05", "gnorm": "6.756", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "66753"}
[2024-07-01 18:10:46,442][train_inner][INFO] - {"epoch": 2, "update": 1.644, "loss": "1.18", "ntokens": "127.26", "acc_total": "127.26", "n_correct": "104.41", "wer_total": "127.26", "n_error": "22.84", "ppl": "2.27", "accuracy": "82.045", "wer": "17.948", "wps": "69.6", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "24800", "lr": "5.44763e-05", "gnorm": "6.795", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "67118"}
[2024-07-01 18:16:51,827][train_inner][INFO] - {"epoch": 2, "update": 1.658, "loss": "1.218", "ntokens": "125.92", "acc_total": "125.92", "n_correct": "103.065", "wer_total": "125.92", "n_error": "22.825", "ppl": "2.33", "accuracy": "81.85", "wer": "18.127", "wps": "68.9", "ups": "0.55", "wpb": "125.9", "bsz": "8", "num_updates": "25000", "lr": "5.28686e-05", "gnorm": "6.888", "loss_scale": "2048", "train_wall": "365", "gb_free": "6.5", "wall": "67484"}
[2024-07-01 18:16:51,828][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 18:58:47,719][valid][INFO] - {"epoch": 2, "valid_loss": "1.05", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.1195", "valid_wer_total": "18.1585", "valid_n_error": "3.0376", "valid_ppl": "2.07", "valid_accuracy": "83.264", "valid_wer": "16.728", "valid_wps": "173.2", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "25000", "valid_best_accuracy": "83.864"}
[2024-07-01 18:58:47,719][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 25000 updates
[2024-07-01 18:58:47,719][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_25000.pt
[2024-07-01 18:58:50,807][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_25000.pt
[2024-07-01 18:58:52,982][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_25000.pt (epoch 2 @ 25000 updates, score 83.264) (writing took 5.262691922995145 seconds)
[2024-07-01 19:04:57,487][train_inner][INFO] - {"epoch": 2, "update": 1.671, "loss": "1.12", "ntokens": "127.44", "acc_total": "127.44", "n_correct": "105.645", "wer_total": "127.44", "n_error": "21.785", "ppl": "2.17", "accuracy": "82.898", "wer": "17.094", "wps": "8.8", "ups": "0.07", "wpb": "127.4", "bsz": "8", "num_updates": "25200", "lr": "5.13083e-05", "gnorm": "6.804", "loss_scale": "2048", "train_wall": "364", "gb_free": "6.5", "wall": "70369"}
[2024-07-01 19:11:01,160][train_inner][INFO] - {"epoch": 2, "update": 1.684, "loss": "1.181", "ntokens": "125.36", "acc_total": "125.36", "n_correct": "102.205", "wer_total": "125.36", "n_error": "23.135", "ppl": "2.27", "accuracy": "81.529", "wer": "18.455", "wps": "68.9", "ups": "0.55", "wpb": "125.4", "bsz": "8", "num_updates": "25400", "lr": "4.9794e-05", "gnorm": "6.86", "loss_scale": "2048", "train_wall": "363", "gb_free": "6.5", "wall": "70733"}
[2024-07-01 19:17:04,499][train_inner][INFO] - {"epoch": 2, "update": 1.697, "loss": "1.104", "ntokens": "127.07", "acc_total": "127.07", "n_correct": "103.78", "wer_total": "127.07", "n_error": "23.29", "ppl": "2.15", "accuracy": "81.672", "wer": "18.328", "wps": "69.9", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "25600", "lr": "4.83244e-05", "gnorm": "6.628", "loss_scale": "2048", "train_wall": "363", "gb_free": "6.5", "wall": "71096"}
[2024-07-01 19:19:09,760][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 19:23:09,487][train_inner][INFO] - {"epoch": 2, "update": 1.711, "loss": "1.123", "ntokens": "127.01", "acc_total": "127.01", "n_correct": "105.095", "wer_total": "127.01", "n_error": "21.91", "ppl": "2.18", "accuracy": "82.745", "wer": "17.251", "wps": "69.6", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "25800", "lr": "4.68982e-05", "gnorm": "6.635", "loss_scale": "1024", "train_wall": "364", "gb_free": "6.5", "wall": "71461"}
[2024-07-01 19:29:12,524][train_inner][INFO] - {"epoch": 2, "update": 1.724, "loss": "1.174", "ntokens": "126.995", "acc_total": "126.995", "n_correct": "104.815", "wer_total": "126.995", "n_error": "22.16", "ppl": "2.26", "accuracy": "82.535", "wer": "17.45", "wps": "70", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "26000", "lr": "4.55141e-05", "gnorm": "6.936", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "71824"}
[2024-07-01 19:35:15,462][train_inner][INFO] - {"epoch": 2, "update": 1.737, "loss": "1.144", "ntokens": "126.31", "acc_total": "126.31", "n_correct": "104.13", "wer_total": "126.31", "n_error": "22.175", "ppl": "2.21", "accuracy": "82.44", "wer": "17.556", "wps": "69.6", "ups": "0.55", "wpb": "126.3", "bsz": "8", "num_updates": "26200", "lr": "4.41708e-05", "gnorm": "6.93", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "72187"}
[2024-07-01 19:41:18,719][train_inner][INFO] - {"epoch": 2, "update": 1.75, "loss": "1.14", "ntokens": "127.37", "acc_total": "127.37", "n_correct": "104.17", "wer_total": "127.37", "n_error": "23.17", "ppl": "2.2", "accuracy": "81.785", "wer": "18.191", "wps": "70.1", "ups": "0.55", "wpb": "127.4", "bsz": "8", "num_updates": "26400", "lr": "4.28672e-05", "gnorm": "6.881", "loss_scale": "1024", "train_wall": "363", "gb_free": "6.5", "wall": "72550"}
[2024-07-01 19:47:21,836][train_inner][INFO] - {"epoch": 2, "update": 1.764, "loss": "1.088", "ntokens": "126.69", "acc_total": "126.69", "n_correct": "105.865", "wer_total": "126.69", "n_error": "20.805", "ppl": "2.13", "accuracy": "83.562", "wer": "16.422", "wps": "69.8", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "26600", "lr": "4.16021e-05", "gnorm": "6.815", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "72914"}
[2024-07-01 19:53:24,706][train_inner][INFO] - {"epoch": 2, "update": 1.777, "loss": "1.097", "ntokens": "126.605", "acc_total": "126.605", "n_correct": "105.675", "wer_total": "126.605", "n_error": "20.925", "ppl": "2.14", "accuracy": "83.468", "wer": "16.528", "wps": "69.8", "ups": "0.55", "wpb": "126.6", "bsz": "8", "num_updates": "26800", "lr": "4.03743e-05", "gnorm": "6.645", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "73276"}
[2024-07-01 19:59:27,904][train_inner][INFO] - {"epoch": 2, "update": 1.79, "loss": "1.16", "ntokens": "127.595", "acc_total": "127.595", "n_correct": "105.115", "wer_total": "127.595", "n_error": "22.455", "ppl": "2.23", "accuracy": "82.382", "wer": "17.599", "wps": "70.3", "ups": "0.55", "wpb": "127.6", "bsz": "8", "num_updates": "27000", "lr": "3.91827e-05", "gnorm": "6.969", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "73640"}
[2024-07-01 20:05:31,201][train_inner][INFO] - {"epoch": 2, "update": 1.803, "loss": "1.15", "ntokens": "127.94", "acc_total": "127.94", "n_correct": "105.83", "wer_total": "127.94", "n_error": "22.1", "ppl": "2.22", "accuracy": "82.718", "wer": "17.274", "wps": "70.4", "ups": "0.55", "wpb": "127.9", "bsz": "8", "num_updates": "27200", "lr": "3.80263e-05", "gnorm": "6.705", "loss_scale": "1024", "train_wall": "363", "gb_free": "6.5", "wall": "74003"}
[2024-07-01 20:11:34,245][train_inner][INFO] - {"epoch": 2, "update": 1.817, "loss": "1.148", "ntokens": "126.68", "acc_total": "126.68", "n_correct": "104.25", "wer_total": "126.68", "n_error": "22.415", "ppl": "2.22", "accuracy": "82.294", "wer": "17.694", "wps": "69.8", "ups": "0.55", "wpb": "126.7", "bsz": "8", "num_updates": "27400", "lr": "3.6904e-05", "gnorm": "6.647", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "74366"}
[2024-07-01 20:14:35,785][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 20:56:15,761][valid][INFO] - {"epoch": 2, "valid_loss": "0.992", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.2409", "valid_wer_total": "18.1585", "valid_n_error": "2.91638", "valid_ppl": "1.99", "valid_accuracy": "83.933", "valid_wer": "16.061", "valid_wps": "174.3", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "27500", "valid_best_accuracy": "83.933"}
[2024-07-01 20:56:15,762][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 27500 updates
[2024-07-01 20:56:15,762][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_27500.pt
[2024-07-01 20:56:18,927][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_27500.pt
[2024-07-01 20:56:23,300][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_27500.pt (epoch 2 @ 27500 updates, score 83.933) (writing took 7.538334308992489 seconds)
[2024-07-01 20:59:24,395][train_inner][INFO] - {"epoch": 2, "update": 1.83, "loss": "1.104", "ntokens": "126.45", "acc_total": "126.45", "n_correct": "103.465", "wer_total": "126.45", "n_error": "22.975", "ppl": "2.15", "accuracy": "81.823", "wer": "18.169", "wps": "8.8", "ups": "0.07", "wpb": "126.5", "bsz": "8", "num_updates": "27600", "lr": "3.58149e-05", "gnorm": "6.827", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "77236"}
[2024-07-01 21:05:27,068][train_inner][INFO] - {"epoch": 2, "update": 1.843, "loss": "1.129", "ntokens": "127.465", "acc_total": "127.465", "n_correct": "104.895", "wer_total": "127.465", "n_error": "22.565", "ppl": "2.19", "accuracy": "82.293", "wer": "17.703", "wps": "70.3", "ups": "0.55", "wpb": "127.5", "bsz": "8", "num_updates": "27800", "lr": "3.47579e-05", "gnorm": "6.765", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "77599"}
[2024-07-01 21:11:29,617][train_inner][INFO] - {"epoch": 2, "update": 1.856, "loss": "1.143", "ntokens": "127.49", "acc_total": "127.49", "n_correct": "103.815", "wer_total": "127.49", "n_error": "23.675", "ppl": "2.21", "accuracy": "81.43", "wer": "18.57", "wps": "70.3", "ups": "0.55", "wpb": "127.5", "bsz": "8", "num_updates": "28000", "lr": "3.37321e-05", "gnorm": "6.495", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "77961"}
[2024-07-01 21:17:32,287][train_inner][INFO] - {"epoch": 2, "update": 1.87, "loss": "1.111", "ntokens": "127.75", "acc_total": "127.75", "n_correct": "105.185", "wer_total": "127.75", "n_error": "22.565", "ppl": "2.16", "accuracy": "82.337", "wer": "17.663", "wps": "70.5", "ups": "0.55", "wpb": "127.8", "bsz": "8", "num_updates": "28200", "lr": "3.27365e-05", "gnorm": "6.728", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "78324"}
[2024-07-01 21:23:34,785][train_inner][INFO] - {"epoch": 2, "update": 1.883, "loss": "1.107", "ntokens": "127.125", "acc_total": "127.125", "n_correct": "104.14", "wer_total": "127.125", "n_error": "22.985", "ppl": "2.15", "accuracy": "81.919", "wer": "18.081", "wps": "70.1", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "28400", "lr": "3.17704e-05", "gnorm": "6.813", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "78686"}
[2024-07-01 21:29:37,234][train_inner][INFO] - {"epoch": 2, "update": 1.896, "loss": "1.118", "ntokens": "128.015", "acc_total": "128.015", "n_correct": "104.945", "wer_total": "128.015", "n_error": "23.06", "ppl": "2.17", "accuracy": "81.979", "wer": "18.014", "wps": "70.6", "ups": "0.55", "wpb": "128", "bsz": "8", "num_updates": "28600", "lr": "3.08327e-05", "gnorm": "6.707", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "79049"}
[2024-07-01 21:35:39,738][train_inner][INFO] - {"epoch": 2, "update": 1.91, "loss": "1.093", "ntokens": "126.755", "acc_total": "126.755", "n_correct": "105.045", "wer_total": "126.755", "n_error": "21.71", "ppl": "2.13", "accuracy": "82.872", "wer": "17.128", "wps": "69.9", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "28800", "lr": "2.99228e-05", "gnorm": "6.61", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "79411"}
[2024-07-01 21:41:42,124][train_inner][INFO] - {"epoch": 2, "update": 1.923, "loss": "1.073", "ntokens": "127.74", "acc_total": "127.74", "n_correct": "105.81", "wer_total": "127.74", "n_error": "21.92", "ppl": "2.1", "accuracy": "82.832", "wer": "17.16", "wps": "70.5", "ups": "0.55", "wpb": "127.7", "bsz": "8", "num_updates": "29000", "lr": "2.90397e-05", "gnorm": "6.53", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "79774"}
[2024-07-01 21:47:44,441][train_inner][INFO] - {"epoch": 2, "update": 1.936, "loss": "1.109", "ntokens": "126.765", "acc_total": "126.765", "n_correct": "105.15", "wer_total": "126.765", "n_error": "21.61", "ppl": "2.16", "accuracy": "82.949", "wer": "17.047", "wps": "70", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "29200", "lr": "2.81826e-05", "gnorm": "6.793", "loss_scale": "2048", "train_wall": "362", "gb_free": "6.5", "wall": "80136"}
[2024-07-01 21:49:29,377][fairseq.trainer][INFO] - NOTE: gradient overflow detected, ignoring gradient, setting loss scale to: 1024.0
[2024-07-01 21:53:48,539][train_inner][INFO] - {"epoch": 2, "update": 1.949, "loss": "1.107", "ntokens": "127.28", "acc_total": "127.28", "n_correct": "106.215", "wer_total": "127.28", "n_error": "21.06", "ppl": "2.15", "accuracy": "83.45", "wer": "16.546", "wps": "69.9", "ups": "0.55", "wpb": "127.3", "bsz": "8", "num_updates": "29400", "lr": "2.73509e-05", "gnorm": "6.736", "loss_scale": "1024", "train_wall": "363", "gb_free": "6.5", "wall": "80500"}
[2024-07-01 21:59:50,696][train_inner][INFO] - {"epoch": 2, "update": 1.963, "loss": "1.077", "ntokens": "127.1", "acc_total": "127.1", "n_correct": "106.38", "wer_total": "127.1", "n_error": "20.705", "ppl": "2.11", "accuracy": "83.698", "wer": "16.29", "wps": "70.2", "ups": "0.55", "wpb": "127.1", "bsz": "8", "num_updates": "29600", "lr": "2.65436e-05", "gnorm": "6.898", "loss_scale": "1024", "train_wall": "361", "gb_free": "6.5", "wall": "80862"}
[2024-07-01 22:05:53,301][train_inner][INFO] - {"epoch": 2, "update": 1.976, "loss": "1.103", "ntokens": "126.785", "acc_total": "126.785", "n_correct": "105.28", "wer_total": "126.785", "n_error": "21.505", "ppl": "2.15", "accuracy": "83.038", "wer": "16.962", "wps": "69.9", "ups": "0.55", "wpb": "126.8", "bsz": "8", "num_updates": "29800", "lr": "2.57603e-05", "gnorm": "6.669", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "81225"}
[2024-07-01 22:11:55,587][train_inner][INFO] - {"epoch": 2, "update": 1.989, "loss": "1.087", "ntokens": "126.96", "acc_total": "126.96", "n_correct": "105.86", "wer_total": "126.96", "n_error": "21.085", "ppl": "2.12", "accuracy": "83.381", "wer": "16.608", "wps": "70.1", "ups": "0.55", "wpb": "127", "bsz": "8", "num_updates": "30000", "lr": "2.5e-05", "gnorm": "6.814", "loss_scale": "1024", "train_wall": "362", "gb_free": "6.5", "wall": "81587"}
[2024-07-01 22:11:55,587][fairseq_cli.train][INFO] - Stopping training due to num_updates: 30000 >= max_update: 30000
[2024-07-01 22:11:55,587][fairseq_cli.train][INFO] - begin validation on "valid" subset
[2024-07-01 22:53:42,679][valid][INFO] - {"epoch": 2, "valid_loss": "0.957", "valid_ntokens": "18.1585", "valid_acc_total": "18.1585", "valid_n_correct": "15.4728", "valid_wer_total": "18.1585", "valid_n_error": "2.68462", "valid_ppl": "1.94", "valid_accuracy": "85.21", "valid_wer": "14.784", "valid_wps": "173.8", "valid_wpb": "18.2", "valid_bsz": "1", "valid_num_updates": "30000", "valid_best_accuracy": "85.21"}
[2024-07-01 22:53:42,679][fairseq.checkpoint_utils][INFO] - Preparing to save checkpoint for epoch 2 @ 30000 updates
[2024-07-01 22:53:42,680][fairseq.trainer][INFO] - Saving checkpoint to checkpoints/checkpoint_2_30000.pt
[2024-07-01 22:53:45,779][fairseq.trainer][INFO] - Finished saving checkpoint to checkpoints/checkpoint_2_30000.pt
[2024-07-01 22:53:50,206][fairseq.checkpoint_utils][INFO] - Saved checkpoint checkpoints/checkpoint_2_30000.pt (epoch 2 @ 30000 updates, score 85.21) (writing took 7.526304693994462 seconds)
[2024-07-01 22:53:50,237][fairseq_cli.train][INFO] - end of epoch 2 (average epoch stats below)
[2024-07-01 22:53:50,239][train][INFO] - {"epoch": 2, "train_loss": "1.31", "train_ntokens": "126.908", "train_acc_total": "126.908", "train_n_correct": "101.026", "train_wer_total": "126.908", "train_n_error": "25.8651", "train_ppl": "2.48", "train_accuracy": "79.605", "train_wer": "20.381", "train_wps": "45.4", "train_ups": "0.36", "train_wpb": "126.9", "train_bsz": "8", "train_num_updates": "30000", "train_lr": "2.5e-05", "train_gnorm": "6.477", "train_loss_scale": "1024", "train_train_wall": "26590", "train_gb_free": "6.5", "train_wall": "84102"}
[2024-07-01 22:53:50,240][fairseq_cli.train][INFO] - done training in 84101.3 seconds