sf_log.txt

[2024-11-12 19:52:21,455][4080366] Saving configuration to /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/config.json...
[2024-11-12 19:52:21,455][4080366] Rollout worker 0 uses device cpu
[2024-11-12 19:52:21,455][4080366] Rollout worker 1 uses device cpu
[2024-11-12 19:52:21,456][4080366] Rollout worker 2 uses device cpu
[2024-11-12 19:52:21,456][4080366] Rollout worker 3 uses device cpu
[2024-11-12 19:52:21,456][4080366] Rollout worker 4 uses device cpu
[2024-11-12 19:52:21,456][4080366] Rollout worker 5 uses device cpu
[2024-11-12 19:52:21,457][4080366] Rollout worker 6 uses device cpu
[2024-11-12 19:52:21,457][4080366] Rollout worker 7 uses device cpu
[2024-11-12 19:52:21,557][4080366] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-11-12 19:52:21,558][4080366] InferenceWorker_p0-w0: min num requests: 2
[2024-11-12 19:52:21,577][4080366] Starting all processes...
[2024-11-12 19:52:21,578][4080366] Starting process learner_proc0
[2024-11-12 19:52:22,191][4080366] Starting all processes...
[2024-11-12 19:52:22,216][4080366] Starting process inference_proc0-0
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc0
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc1
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc2
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc3
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc4
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc5
[2024-11-12 19:52:22,217][4080366] Starting process rollout_proc6
[2024-11-12 19:52:22,218][4080366] Starting process rollout_proc7
[2024-11-12 19:52:23,438][4080811] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-11-12 19:52:23,438][4080811] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2024-11-12 19:52:23,447][4080816] Worker 5 uses CPU cores [15, 16, 17]
[2024-11-12 19:52:23,467][4080812] Worker 1 uses CPU cores [3, 4, 5]
[2024-11-12 19:52:23,474][4080814] Worker 2 uses CPU cores [6, 7, 8]
[2024-11-12 19:52:23,474][4080815] Worker 3 uses CPU cores [9, 10, 11]
[2024-11-12 19:52:23,476][4080817] Worker 4 uses CPU cores [12, 13, 14]
[2024-11-12 19:52:23,477][4080798] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-11-12 19:52:23,477][4080798] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2024-11-12 19:52:23,481][4080811] Num visible devices: 1
[2024-11-12 19:52:23,527][4080798] Num visible devices: 1
[2024-11-12 19:52:23,558][4080798] Starting seed is not provided
[2024-11-12 19:52:23,558][4080798] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-11-12 19:52:23,558][4080798] Initializing actor-critic model on device cuda:0
[2024-11-12 19:52:23,559][4080798] RunningMeanStd input shape: (3, 72, 128)
[2024-11-12 19:52:23,559][4080798] RunningMeanStd input shape: (1,)
[2024-11-12 19:52:23,570][4080798] ConvEncoder: input_channels=3
[2024-11-12 19:52:23,588][4080818] Worker 6 uses CPU cores [18, 19, 20]
[2024-11-12 19:52:23,633][4080813] Worker 0 uses CPU cores [0, 1, 2]
[2024-11-12 19:52:23,722][4080819] Worker 7 uses CPU cores [21, 22, 23]
[2024-11-12 19:52:24,116][4080798] Conv encoder output size: 512
[2024-11-12 19:52:24,116][4080798] Policy head output size: 512
[2024-11-12 19:52:24,123][4080798] Created Actor Critic model with architecture:
[2024-11-12 19:52:24,123][4080798] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ELU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ELU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ELU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ELU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): GRU(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2024-11-12 19:52:24,454][4080798] Using optimizer <class 'torch.optim.adam.Adam'>
[2024-11-12 19:52:24,811][4080798] No checkpoints found
[2024-11-12 19:52:24,811][4080798] Did not load from checkpoint, starting from scratch!
[2024-11-12 19:52:24,811][4080798] Initialized policy 0 weights for model version 0
[2024-11-12 19:52:24,812][4080798] LearnerWorker_p0 finished initialization!
[2024-11-12 19:52:24,812][4080798] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2024-11-12 19:52:24,883][4080811] RunningMeanStd input shape: (3, 72, 128)
[2024-11-12 19:52:24,884][4080811] RunningMeanStd input shape: (1,)
[2024-11-12 19:52:24,890][4080811] ConvEncoder: input_channels=3
[2024-11-12 19:52:24,936][4080811] Conv encoder output size: 512
[2024-11-12 19:52:24,936][4080811] Policy head output size: 512
[2024-11-12 19:52:24,964][4080366] Inference worker 0-0 is ready!
[2024-11-12 19:52:24,965][4080366] All inference workers are ready! Signal rollout workers to start!
[2024-11-12 19:52:25,000][4080814] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,000][4080813] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,000][4080817] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,000][4080815] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,000][4080812] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,001][4080816] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,008][4080819] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,011][4080818] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:52:25,322][4080366] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2024-11-12 19:52:25,444][4080817] Decorrelating experience for 0 frames...
[2024-11-12 19:52:25,446][4080815] Decorrelating experience for 0 frames...
[2024-11-12 19:52:25,448][4080816] Decorrelating experience for 0 frames...
[2024-11-12 19:52:25,449][4080819] Decorrelating experience for 0 frames...
[2024-11-12 19:52:25,581][4080815] Decorrelating experience for 32 frames...
[2024-11-12 19:52:25,586][4080812] Decorrelating experience for 0 frames...
[2024-11-12 19:52:25,662][4080816] Decorrelating experience for 32 frames...
[2024-11-12 19:52:25,723][4080812] Decorrelating experience for 32 frames...
[2024-11-12 19:52:25,732][4080817] Decorrelating experience for 32 frames...
[2024-11-12 19:52:25,897][4080817] Decorrelating experience for 64 frames...
[2024-11-12 19:52:25,919][4080819] Decorrelating experience for 32 frames...
[2024-11-12 19:52:25,924][4080816] Decorrelating experience for 64 frames...
[2024-11-12 19:52:25,978][4080818] Decorrelating experience for 0 frames...
[2024-11-12 19:52:26,046][4080817] Decorrelating experience for 96 frames...
[2024-11-12 19:52:26,094][4080812] Decorrelating experience for 64 frames...
[2024-11-12 19:52:26,150][4080816] Decorrelating experience for 96 frames...
[2024-11-12 19:52:26,151][4080815] Decorrelating experience for 64 frames...
[2024-11-12 19:52:26,193][4080818] Decorrelating experience for 32 frames...
[2024-11-12 19:52:26,288][4080819] Decorrelating experience for 64 frames...
[2024-11-12 19:52:26,290][4080812] Decorrelating experience for 96 frames...
[2024-11-12 19:52:26,291][4080815] Decorrelating experience for 96 frames...
[2024-11-12 19:52:26,455][4080818] Decorrelating experience for 64 frames...
[2024-11-12 19:52:26,519][4080819] Decorrelating experience for 96 frames...
[2024-11-12 19:52:26,681][4080818] Decorrelating experience for 96 frames...
[2024-11-12 19:52:29,589][4080798] Signal inference workers to stop experience collection...
[2024-11-12 19:52:29,592][4080811] InferenceWorker_p0-w0: stopping experience collection
[2024-11-12 19:52:30,322][4080366] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 5.6. Samples: 28. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
[2024-11-12 19:52:30,325][4080366] Avg episode reward: [(0, '2.638')]
[2024-11-12 19:52:34,741][4080798] Signal inference workers to resume experience collection...
[2024-11-12 19:52:34,741][4080811] InferenceWorker_p0-w0: resuming experience collection
[2024-11-12 19:52:35,321][4080366] Fps is (10 sec: 1638.5, 60 sec: 1638.5, 300 sec: 1638.5). Total num frames: 16384. Throughput: 0: 238.2. Samples: 2382. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
[2024-11-12 19:52:35,322][4080366] Avg episode reward: [(0, '3.726')]
[2024-11-12 19:52:35,990][4080811] Updated weights for policy 0, policy_version 10 (0.0186)
[2024-11-12 19:52:37,286][4080811] Updated weights for policy 0, policy_version 20 (0.0007)
[2024-11-12 19:52:38,492][4080811] Updated weights for policy 0, policy_version 30 (0.0005)
[2024-11-12 19:52:39,775][4080811] Updated weights for policy 0, policy_version 40 (0.0006)
[2024-11-12 19:52:40,321][4080366] Fps is (10 sec: 18023.6, 60 sec: 12015.4, 300 sec: 12015.4). Total num frames: 180224. Throughput: 0: 2887.3. Samples: 43308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:52:40,322][4080366] Avg episode reward: [(0, '4.426')]
[2024-11-12 19:52:40,343][4080798] Saving new best policy, reward=4.426!
[2024-11-12 19:52:40,999][4080811] Updated weights for policy 0, policy_version 50 (0.0006)
[2024-11-12 19:52:41,553][4080366] Heartbeat connected on LearnerWorker_p0
[2024-11-12 19:52:41,557][4080366] Heartbeat connected on Batcher_0
[2024-11-12 19:52:41,563][4080366] Heartbeat connected on InferenceWorker_p0-w0
[2024-11-12 19:52:41,566][4080366] Heartbeat connected on RolloutWorker_w1
[2024-11-12 19:52:41,571][4080366] Heartbeat connected on RolloutWorker_w3
[2024-11-12 19:52:41,571][4080366] Heartbeat connected on RolloutWorker_w4
[2024-11-12 19:52:41,573][4080366] Heartbeat connected on RolloutWorker_w5
[2024-11-12 19:52:41,575][4080366] Heartbeat connected on RolloutWorker_w6
[2024-11-12 19:52:41,580][4080366] Heartbeat connected on RolloutWorker_w7
[2024-11-12 19:52:42,204][4080811] Updated weights for policy 0, policy_version 60 (0.0006)
[2024-11-12 19:52:43,423][4080811] Updated weights for policy 0, policy_version 70 (0.0006)
[2024-11-12 19:52:44,641][4080811] Updated weights for policy 0, policy_version 80 (0.0005)
[2024-11-12 19:52:45,321][4080366] Fps is (10 sec: 33177.5, 60 sec: 17408.4, 300 sec: 17408.4). Total num frames: 348160. Throughput: 0: 3410.1. Samples: 68200. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:52:45,322][4080366] Avg episode reward: [(0, '4.315')]
[2024-11-12 19:52:45,826][4080811] Updated weights for policy 0, policy_version 90 (0.0005)
[2024-11-12 19:52:47,006][4080811] Updated weights for policy 0, policy_version 100 (0.0005)
[2024-11-12 19:52:48,164][4080811] Updated weights for policy 0, policy_version 110 (0.0006)
[2024-11-12 19:52:49,316][4080811] Updated weights for policy 0, policy_version 120 (0.0005)
[2024-11-12 19:52:50,321][4080366] Fps is (10 sec: 34405.3, 60 sec: 20971.7, 300 sec: 20971.7). Total num frames: 524288. Throughput: 0: 4811.7. Samples: 120292. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-11-12 19:52:50,322][4080366] Avg episode reward: [(0, '4.520')]
[2024-11-12 19:52:50,326][4080798] Saving new best policy, reward=4.520!
[2024-11-12 19:52:50,476][4080811] Updated weights for policy 0, policy_version 130 (0.0005)
[2024-11-12 19:52:51,623][4080811] Updated weights for policy 0, policy_version 140 (0.0005)
[2024-11-12 19:52:52,757][4080811] Updated weights for policy 0, policy_version 150 (0.0005)
[2024-11-12 19:52:53,932][4080811] Updated weights for policy 0, policy_version 160 (0.0006)
[2024-11-12 19:52:55,125][4080811] Updated weights for policy 0, policy_version 170 (0.0006)
[2024-11-12 19:52:55,321][4080366] Fps is (10 sec: 35225.6, 60 sec: 23347.6, 300 sec: 23347.6). Total num frames: 700416. Throughput: 0: 5779.0. Samples: 173368. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-11-12 19:52:55,322][4080366] Avg episode reward: [(0, '4.636')]
[2024-11-12 19:52:55,323][4080798] Saving new best policy, reward=4.636!
[2024-11-12 19:52:56,359][4080811] Updated weights for policy 0, policy_version 180 (0.0006)
[2024-11-12 19:52:57,539][4080811] Updated weights for policy 0, policy_version 190 (0.0005)
[2024-11-12 19:52:58,856][4080811] Updated weights for policy 0, policy_version 200 (0.0005)
[2024-11-12 19:53:00,042][4080811] Updated weights for policy 0, policy_version 210 (0.0005)
[2024-11-12 19:53:00,321][4080366] Fps is (10 sec: 34407.1, 60 sec: 24810.4, 300 sec: 24810.4). Total num frames: 868352. Throughput: 0: 5668.5. Samples: 198394. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-11-12 19:53:00,322][4080366] Avg episode reward: [(0, '4.573')]
[2024-11-12 19:53:01,307][4080811] Updated weights for policy 0, policy_version 220 (0.0005)
[2024-11-12 19:53:02,524][4080811] Updated weights for policy 0, policy_version 230 (0.0006)
[2024-11-12 19:53:03,789][4080811] Updated weights for policy 0, policy_version 240 (0.0006)
[2024-11-12 19:53:04,964][4080811] Updated weights for policy 0, policy_version 250 (0.0005)
[2024-11-12 19:53:05,321][4080366] Fps is (10 sec: 33177.9, 60 sec: 25805.2, 300 sec: 25805.2). Total num frames: 1032192. Throughput: 0: 6195.6. Samples: 247822. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:05,322][4080366] Avg episode reward: [(0, '4.831')]
[2024-11-12 19:53:05,339][4080798] Saving new best policy, reward=4.831!
[2024-11-12 19:53:06,199][4080811] Updated weights for policy 0, policy_version 260 (0.0005)
[2024-11-12 19:53:07,374][4080811] Updated weights for policy 0, policy_version 270 (0.0005)
[2024-11-12 19:53:08,535][4080811] Updated weights for policy 0, policy_version 280 (0.0006)
[2024-11-12 19:53:09,725][4080811] Updated weights for policy 0, policy_version 290 (0.0005)
[2024-11-12 19:53:10,321][4080366] Fps is (10 sec: 33587.4, 60 sec: 26760.8, 300 sec: 26760.8). Total num frames: 1204224. Throughput: 0: 6656.6. Samples: 299542. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:53:10,322][4080366] Avg episode reward: [(0, '5.176')]
[2024-11-12 19:53:10,338][4080798] Saving new best policy, reward=5.176!
[2024-11-12 19:53:10,967][4080811] Updated weights for policy 0, policy_version 300 (0.0006)
[2024-11-12 19:53:12,166][4080811] Updated weights for policy 0, policy_version 310 (0.0006)
[2024-11-12 19:53:13,332][4080811] Updated weights for policy 0, policy_version 320 (0.0006)
[2024-11-12 19:53:14,540][4080811] Updated weights for policy 0, policy_version 330 (0.0005)
[2024-11-12 19:53:15,321][4080366] Fps is (10 sec: 34406.4, 60 sec: 27525.4, 300 sec: 27525.4). Total num frames: 1376256. Throughput: 0: 7217.0. Samples: 324790. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
[2024-11-12 19:53:15,322][4080366] Avg episode reward: [(0, '5.563')]
[2024-11-12 19:53:15,323][4080798] Saving new best policy, reward=5.563!
[2024-11-12 19:53:15,772][4080811] Updated weights for policy 0, policy_version 340 (0.0005)
[2024-11-12 19:53:16,953][4080811] Updated weights for policy 0, policy_version 350 (0.0006)
[2024-11-12 19:53:18,355][4080811] Updated weights for policy 0, policy_version 360 (0.0007)
[2024-11-12 19:53:19,675][4080811] Updated weights for policy 0, policy_version 370 (0.0006)
[2024-11-12 19:53:20,321][4080366] Fps is (10 sec: 32768.3, 60 sec: 27853.1, 300 sec: 27853.1). Total num frames: 1531904. Throughput: 0: 8255.6. Samples: 373882. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:20,322][4080366] Avg episode reward: [(0, '6.977')]
[2024-11-12 19:53:20,325][4080798] Saving new best policy, reward=6.977!
[2024-11-12 19:53:20,994][4080811] Updated weights for policy 0, policy_version 380 (0.0006)
[2024-11-12 19:53:22,301][4080811] Updated weights for policy 0, policy_version 390 (0.0006)
[2024-11-12 19:53:23,657][4080811] Updated weights for policy 0, policy_version 400 (0.0007)
[2024-11-12 19:53:24,971][4080811] Updated weights for policy 0, policy_version 410 (0.0007)
[2024-11-12 19:53:25,321][4080366] Fps is (10 sec: 31129.4, 60 sec: 28126.1, 300 sec: 28126.1). Total num frames: 1687552. Throughput: 0: 8380.0. Samples: 420410. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-11-12 19:53:25,322][4080366] Avg episode reward: [(0, '8.040')]
[2024-11-12 19:53:25,323][4080798] Saving new best policy, reward=8.040!
[2024-11-12 19:53:26,341][4080811] Updated weights for policy 0, policy_version 420 (0.0006)
[2024-11-12 19:53:27,629][4080811] Updated weights for policy 0, policy_version 430 (0.0007)
[2024-11-12 19:53:28,946][4080811] Updated weights for policy 0, policy_version 440 (0.0006)
[2024-11-12 19:53:30,242][4080811] Updated weights for policy 0, policy_version 450 (0.0006)
[2024-11-12 19:53:30,321][4080366] Fps is (10 sec: 31129.3, 60 sec: 30720.3, 300 sec: 28357.1). Total num frames: 1843200. Throughput: 0: 8339.3. Samples: 443470. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:30,322][4080366] Avg episode reward: [(0, '9.563')]
[2024-11-12 19:53:30,324][4080798] Saving new best policy, reward=9.563!
[2024-11-12 19:53:31,542][4080811] Updated weights for policy 0, policy_version 460 (0.0008)
[2024-11-12 19:53:32,828][4080811] Updated weights for policy 0, policy_version 470 (0.0007)
[2024-11-12 19:53:34,122][4080811] Updated weights for policy 0, policy_version 480 (0.0007)
[2024-11-12 19:53:35,321][4080366] Fps is (10 sec: 31538.8, 60 sec: 33109.2, 300 sec: 28613.6). Total num frames: 2002944. Throughput: 0: 8235.5. Samples: 490890. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:35,322][4080366] Avg episode reward: [(0, '13.531')]
[2024-11-12 19:53:35,323][4080798] Saving new best policy, reward=13.531!
[2024-11-12 19:53:35,451][4080811] Updated weights for policy 0, policy_version 490 (0.0008)
[2024-11-12 19:53:36,761][4080811] Updated weights for policy 0, policy_version 500 (0.0008)
[2024-11-12 19:53:38,096][4080811] Updated weights for policy 0, policy_version 510 (0.0007)
[2024-11-12 19:53:39,399][4080811] Updated weights for policy 0, policy_version 520 (0.0007)
[2024-11-12 19:53:40,321][4080366] Fps is (10 sec: 31128.9, 60 sec: 32904.4, 300 sec: 28726.7). Total num frames: 2154496. Throughput: 0: 8089.3. Samples: 537386. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:40,322][4080366] Avg episode reward: [(0, '11.571')]
[2024-11-12 19:53:40,732][4080811] Updated weights for policy 0, policy_version 530 (0.0008)
[2024-11-12 19:53:42,064][4080811] Updated weights for policy 0, policy_version 540 (0.0007)
[2024-11-12 19:53:43,380][4080811] Updated weights for policy 0, policy_version 550 (0.0007)
[2024-11-12 19:53:44,668][4080811] Updated weights for policy 0, policy_version 560 (0.0006)
[2024-11-12 19:53:45,321][4080366] Fps is (10 sec: 31129.4, 60 sec: 32767.9, 300 sec: 28928.1). Total num frames: 2314240. Throughput: 0: 8048.8. Samples: 560592. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:53:45,323][4080366] Avg episode reward: [(0, '14.270')]
[2024-11-12 19:53:45,324][4080798] Saving new best policy, reward=14.270!
[2024-11-12 19:53:45,962][4080811] Updated weights for policy 0, policy_version 570 (0.0007)
[2024-11-12 19:53:47,236][4080811] Updated weights for policy 0, policy_version 580 (0.0007)
[2024-11-12 19:53:48,505][4080811] Updated weights for policy 0, policy_version 590 (0.0006)
[2024-11-12 19:53:49,793][4080811] Updated weights for policy 0, policy_version 600 (0.0007)
[2024-11-12 19:53:50,321][4080366] Fps is (10 sec: 31949.1, 60 sec: 32495.0, 300 sec: 29105.8). Total num frames: 2473984. Throughput: 0: 8007.6. Samples: 608166. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:53:50,322][4080366] Avg episode reward: [(0, '18.421')]
[2024-11-12 19:53:50,324][4080798] Saving new best policy, reward=18.421!
[2024-11-12 19:53:51,090][4080811] Updated weights for policy 0, policy_version 610 (0.0007)
[2024-11-12 19:53:52,345][4080811] Updated weights for policy 0, policy_version 620 (0.0006)
[2024-11-12 19:53:53,607][4080811] Updated weights for policy 0, policy_version 630 (0.0007)
[2024-11-12 19:53:54,903][4080811] Updated weights for policy 0, policy_version 640 (0.0007)
[2024-11-12 19:53:55,321][4080366] Fps is (10 sec: 31948.9, 60 sec: 32221.8, 300 sec: 29263.7). Total num frames: 2633728. Throughput: 0: 7931.1. Samples: 656444. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-11-12 19:53:55,322][4080366] Avg episode reward: [(0, '17.658')]
[2024-11-12 19:53:56,195][4080811] Updated weights for policy 0, policy_version 650 (0.0006)
[2024-11-12 19:53:57,483][4080811] Updated weights for policy 0, policy_version 660 (0.0007)
[2024-11-12 19:53:58,751][4080811] Updated weights for policy 0, policy_version 670 (0.0007)
[2024-11-12 19:54:00,013][4080811] Updated weights for policy 0, policy_version 680 (0.0006)
[2024-11-12 19:54:00,321][4080366] Fps is (10 sec: 31948.8, 60 sec: 32085.3, 300 sec: 29405.1). Total num frames: 2793472. Throughput: 0: 7900.4. Samples: 680308. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:54:00,322][4080366] Avg episode reward: [(0, '19.218')]
[2024-11-12 19:54:00,324][4080798] Saving new best policy, reward=19.218!
[2024-11-12 19:54:01,308][4080811] Updated weights for policy 0, policy_version 690 (0.0007)
[2024-11-12 19:54:02,563][4080811] Updated weights for policy 0, policy_version 700 (0.0007)
[2024-11-12 19:54:03,873][4080811] Updated weights for policy 0, policy_version 710 (0.0007)
[2024-11-12 19:54:05,163][4080811] Updated weights for policy 0, policy_version 720 (0.0007)
[2024-11-12 19:54:05,321][4080366] Fps is (10 sec: 31949.5, 60 sec: 32017.0, 300 sec: 29532.3). Total num frames: 2953216. Throughput: 0: 7872.6. Samples: 728150. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:54:05,322][4080366] Avg episode reward: [(0, '20.431')]
[2024-11-12 19:54:05,322][4080798] Saving new best policy, reward=20.431!
[2024-11-12 19:54:06,499][4080811] Updated weights for policy 0, policy_version 730 (0.0007)
[2024-11-12 19:54:07,813][4080811] Updated weights for policy 0, policy_version 740 (0.0007)
[2024-11-12 19:54:09,097][4080811] Updated weights for policy 0, policy_version 750 (0.0007)
[2024-11-12 19:54:10,321][4080366] Fps is (10 sec: 31539.3, 60 sec: 31744.0, 300 sec: 29608.4). Total num frames: 3108864. Throughput: 0: 7883.9. Samples: 775188. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:54:10,322][4080366] Avg episode reward: [(0, '21.010')]
[2024-11-12 19:54:10,325][4080798] Saving new best policy, reward=21.010!
[2024-11-12 19:54:10,442][4080811] Updated weights for policy 0, policy_version 760 (0.0006)
[2024-11-12 19:54:11,730][4080811] Updated weights for policy 0, policy_version 770 (0.0007)
[2024-11-12 19:54:13,003][4080811] Updated weights for policy 0, policy_version 780 (0.0007)
[2024-11-12 19:54:14,292][4080811] Updated weights for policy 0, policy_version 790 (0.0006)
[2024-11-12 19:54:15,321][4080366] Fps is (10 sec: 31538.6, 60 sec: 31539.1, 300 sec: 29714.7). Total num frames: 3268608. Throughput: 0: 7896.5. Samples: 798812. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:54:15,322][4080366] Avg episode reward: [(0, '19.712')]
[2024-11-12 19:54:15,591][4080811] Updated weights for policy 0, policy_version 800 (0.0007)
[2024-11-12 19:54:16,927][4080811] Updated weights for policy 0, policy_version 810 (0.0008)
[2024-11-12 19:54:18,182][4080811] Updated weights for policy 0, policy_version 820 (0.0007)
[2024-11-12 19:54:19,484][4080811] Updated weights for policy 0, policy_version 830 (0.0007)
[2024-11-12 19:54:20,321][4080366] Fps is (10 sec: 31539.1, 60 sec: 31539.1, 300 sec: 29776.2). Total num frames: 3424256. Throughput: 0: 7894.8. Samples: 846156. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:54:20,322][4080366] Avg episode reward: [(0, '20.980')]
[2024-11-12 19:54:20,325][4080798] Saving /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000836_3424256.pth...
[2024-11-12 19:54:20,821][4080811] Updated weights for policy 0, policy_version 840 (0.0007)
[2024-11-12 19:54:22,091][4080811] Updated weights for policy 0, policy_version 850 (0.0006)
[2024-11-12 19:54:23,347][4080811] Updated weights for policy 0, policy_version 860 (0.0007)
[2024-11-12 19:54:24,660][4080811] Updated weights for policy 0, policy_version 870 (0.0007)
[2024-11-12 19:54:25,321][4080366] Fps is (10 sec: 31539.4, 60 sec: 31607.4, 300 sec: 29866.8). Total num frames: 3584000. Throughput: 0: 7924.6. Samples: 893990. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-11-12 19:54:25,322][4080366] Avg episode reward: [(0, '24.097')]
[2024-11-12 19:54:25,323][4080798] Saving new best policy, reward=24.097!
[2024-11-12 19:54:25,927][4080811] Updated weights for policy 0, policy_version 880 (0.0007)
[2024-11-12 19:54:27,263][4080811] Updated weights for policy 0, policy_version 890 (0.0007)
[2024-11-12 19:54:28,511][4080811] Updated weights for policy 0, policy_version 900 (0.0006)
[2024-11-12 19:54:29,754][4080811] Updated weights for policy 0, policy_version 910 (0.0007)
[2024-11-12 19:54:30,321][4080366] Fps is (10 sec: 31949.2, 60 sec: 31675.7, 300 sec: 29950.1). Total num frames: 3743744. Throughput: 0: 7933.9. Samples: 917618. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:54:30,322][4080366] Avg episode reward: [(0, '19.775')]
[2024-11-12 19:54:31,050][4080811] Updated weights for policy 0, policy_version 920 (0.0006)
[2024-11-12 19:54:32,328][4080811] Updated weights for policy 0, policy_version 930 (0.0006)
[2024-11-12 19:54:33,644][4080811] Updated weights for policy 0, policy_version 940 (0.0007)
[2024-11-12 19:54:34,921][4080811] Updated weights for policy 0, policy_version 950 (0.0007)
[2024-11-12 19:54:35,321][4080366] Fps is (10 sec: 31949.1, 60 sec: 31675.8, 300 sec: 30026.9). Total num frames: 3903488. Throughput: 0: 7943.9. Samples: 965640. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-11-12 19:54:35,322][4080366] Avg episode reward: [(0, '24.386')]
[2024-11-12 19:54:35,323][4080798] Saving new best policy, reward=24.386!
[2024-11-12 19:54:36,212][4080811] Updated weights for policy 0, policy_version 960 (0.0007)
[2024-11-12 19:54:37,485][4080811] Updated weights for policy 0, policy_version 970 (0.0006)
[2024-11-12 19:54:38,791][4080811] Updated weights for policy 0, policy_version 980 (0.0006)
[2024-11-12 19:54:40,162][4080811] Updated weights for policy 0, policy_version 990 (0.0008)
[2024-11-12 19:54:40,321][4080366] Fps is (10 sec: 31129.4, 60 sec: 31675.8, 300 sec: 30037.4). Total num frames: 4055040. Throughput: 0: 7919.0. Samples: 1012800. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:54:40,323][4080366] Avg episode reward: [(0, '20.459')]
[2024-11-12 19:54:41,559][4080811] Updated weights for policy 0, policy_version 1000 (0.0007)
[2024-11-12 19:54:42,863][4080811] Updated weights for policy 0, policy_version 1010 (0.0006)
[2024-11-12 19:54:44,129][4080811] Updated weights for policy 0, policy_version 1020 (0.0006)
[2024-11-12 19:54:45,321][4080366] Fps is (10 sec: 31129.7, 60 sec: 31675.9, 300 sec: 30105.7). Total num frames: 4214784. Throughput: 0: 7895.7. Samples: 1035614. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0)
[2024-11-12 19:54:45,322][4080366] Avg episode reward: [(0, '23.194')]
[2024-11-12 19:54:45,414][4080811] Updated weights for policy 0, policy_version 1030 (0.0007)
[2024-11-12 19:54:46,708][4080811] Updated weights for policy 0, policy_version 1040 (0.0007)
[2024-11-12 19:54:47,990][4080811] Updated weights for policy 0, policy_version 1050 (0.0007)
[2024-11-12 19:54:49,290][4080811] Updated weights for policy 0, policy_version 1060 (0.0006)
[2024-11-12 19:54:50,321][4080366] Fps is (10 sec: 31539.1, 60 sec: 31607.5, 300 sec: 30141.0). Total num frames: 4370432. Throughput: 0: 7895.0. Samples: 1083428. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:54:50,323][4080366] Avg episode reward: [(0, '22.905')]
[2024-11-12 19:54:50,575][4080811] Updated weights for policy 0, policy_version 1070 (0.0007)
[2024-11-12 19:54:51,844][4080811] Updated weights for policy 0, policy_version 1080 (0.0007)
[2024-11-12 19:54:53,146][4080811] Updated weights for policy 0, policy_version 1090 (0.0007)
[2024-11-12 19:54:54,425][4080811] Updated weights for policy 0, policy_version 1100 (0.0007)
[2024-11-12 19:54:55,321][4080366] Fps is (10 sec: 31948.2, 60 sec: 31675.7, 300 sec: 30228.6). Total num frames: 4534272. Throughput: 0: 7909.7. Samples: 1131126. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:54:55,322][4080366] Avg episode reward: [(0, '22.847')]
[2024-11-12 19:54:55,731][4080811] Updated weights for policy 0, policy_version 1110 (0.0006)
[2024-11-12 19:54:57,043][4080811] Updated weights for policy 0, policy_version 1120 (0.0006)
[2024-11-12 19:54:58,317][4080811] Updated weights for policy 0, policy_version 1130 (0.0007)
[2024-11-12 19:54:59,597][4080811] Updated weights for policy 0, policy_version 1140 (0.0007)
[2024-11-12 19:55:00,321][4080366] Fps is (10 sec: 31948.6, 60 sec: 31607.5, 300 sec: 30257.6). Total num frames: 4689920. Throughput: 0: 7910.7. Samples: 1154792. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:00,322][4080366] Avg episode reward: [(0, '22.796')]
[2024-11-12 19:55:00,902][4080811] Updated weights for policy 0, policy_version 1150 (0.0008)
[2024-11-12 19:55:02,163][4080811] Updated weights for policy 0, policy_version 1160 (0.0007)
[2024-11-12 19:55:03,458][4080811] Updated weights for policy 0, policy_version 1170 (0.0006)
[2024-11-12 19:55:04,733][4080811] Updated weights for policy 0, policy_version 1180 (0.0007)
[2024-11-12 19:55:05,321][4080366] Fps is (10 sec: 31539.2, 60 sec: 31607.4, 300 sec: 30310.5). Total num frames: 4849664. Throughput: 0: 7921.5. Samples: 1202622. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:05,322][4080366] Avg episode reward: [(0, '21.007')]
[2024-11-12 19:55:06,037][4080811] Updated weights for policy 0, policy_version 1190 (0.0007)
[2024-11-12 19:55:07,375][4080811] Updated weights for policy 0, policy_version 1200 (0.0007)
[2024-11-12 19:55:08,671][4080811] Updated weights for policy 0, policy_version 1210 (0.0007)
[2024-11-12 19:55:10,005][4080811] Updated weights for policy 0, policy_version 1220 (0.0007)
[2024-11-12 19:55:10,321][4080366] Fps is (10 sec: 31539.8, 60 sec: 31607.5, 300 sec: 30335.3). Total num frames: 5005312. Throughput: 0: 7902.2. Samples: 1249590. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:10,322][4080366] Avg episode reward: [(0, '21.686')]
[2024-11-12 19:55:11,313][4080811] Updated weights for policy 0, policy_version 1230 (0.0006)
[2024-11-12 19:55:12,624][4080811] Updated weights for policy 0, policy_version 1240 (0.0007)
[2024-11-12 19:55:13,925][4080811] Updated weights for policy 0, policy_version 1250 (0.0006)
[2024-11-12 19:55:15,236][4080811] Updated weights for policy 0, policy_version 1260 (0.0007)
[2024-11-12 19:55:15,321][4080366] Fps is (10 sec: 31128.7, 60 sec: 31539.1, 300 sec: 30358.6). Total num frames: 5160960. Throughput: 0: 7895.0. Samples: 1272896. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-11-12 19:55:15,323][4080366] Avg episode reward: [(0, '21.382')]
[2024-11-12 19:55:16,526][4080811] Updated weights for policy 0, policy_version 1270 (0.0007)
[2024-11-12 19:55:17,820][4080811] Updated weights for policy 0, policy_version 1280 (0.0007)
[2024-11-12 19:55:19,091][4080811] Updated weights for policy 0, policy_version 1290 (0.0006)
[2024-11-12 19:55:20,321][4080366] Fps is (10 sec: 31539.2, 60 sec: 31607.6, 300 sec: 30404.1). Total num frames: 5320704. Throughput: 0: 7884.1. Samples: 1320426. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:20,322][4080366] Avg episode reward: [(0, '21.192')]
[2024-11-12 19:55:20,389][4080811] Updated weights for policy 0, policy_version 1300 (0.0007)
[2024-11-12 19:55:21,712][4080811] Updated weights for policy 0, policy_version 1310 (0.0006)
[2024-11-12 19:55:22,986][4080811] Updated weights for policy 0, policy_version 1320 (0.0007)
[2024-11-12 19:55:24,274][4080811] Updated weights for policy 0, policy_version 1330 (0.0007)
[2024-11-12 19:55:25,321][4080366] Fps is (10 sec: 31950.0, 60 sec: 31607.5, 300 sec: 30447.0). Total num frames: 5480448. Throughput: 0: 7889.9. Samples: 1367846. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:25,322][4080366] Avg episode reward: [(0, '23.688')]
[2024-11-12 19:55:25,596][4080811] Updated weights for policy 0, policy_version 1340 (0.0006)
[2024-11-12 19:55:26,883][4080811] Updated weights for policy 0, policy_version 1350 (0.0007)
[2024-11-12 19:55:28,176][4080811] Updated weights for policy 0, policy_version 1360 (0.0007)
[2024-11-12 19:55:29,498][4080811] Updated weights for policy 0, policy_version 1370 (0.0006)
[2024-11-12 19:55:30,321][4080366] Fps is (10 sec: 31539.2, 60 sec: 31539.2, 300 sec: 30465.5). Total num frames: 5636096. Throughput: 0: 7906.9. Samples: 1391426. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:55:30,322][4080366] Avg episode reward: [(0, '24.751')]
[2024-11-12 19:55:30,324][4080798] Saving new best policy, reward=24.751!
[2024-11-12 19:55:30,798][4080811] Updated weights for policy 0, policy_version 1380 (0.0006)
[2024-11-12 19:55:32,108][4080811] Updated weights for policy 0, policy_version 1390 (0.0007)
[2024-11-12 19:55:33,403][4080811] Updated weights for policy 0, policy_version 1400 (0.0007)
[2024-11-12 19:55:34,729][4080811] Updated weights for policy 0, policy_version 1410 (0.0007)
[2024-11-12 19:55:35,321][4080366] Fps is (10 sec: 31130.0, 60 sec: 31471.0, 300 sec: 30483.0). Total num frames: 5791744. Throughput: 0: 7888.5. Samples: 1438410. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-11-12 19:55:35,322][4080366] Avg episode reward: [(0, '24.579')]
[2024-11-12 19:55:36,035][4080811] Updated weights for policy 0, policy_version 1420 (0.0006)
[2024-11-12 19:55:37,363][4080811] Updated weights for policy 0, policy_version 1430 (0.0008)
[2024-11-12 19:55:38,694][4080811] Updated weights for policy 0, policy_version 1440 (0.0006)
[2024-11-12 19:55:40,009][4080811] Updated weights for policy 0, policy_version 1450 (0.0007)
[2024-11-12 19:55:40,321][4080366] Fps is (10 sec: 31129.0, 60 sec: 31539.2, 300 sec: 30499.5). Total num frames: 5947392. Throughput: 0: 7865.7. Samples: 1485082. Policy #0 lag: (min: 0.0, avg: 0.3, max: 1.0)
[2024-11-12 19:55:40,322][4080366] Avg episode reward: [(0, '23.786')]
[2024-11-12 19:55:41,320][4080811] Updated weights for policy 0, policy_version 1460 (0.0007)
[2024-11-12 19:55:42,606][4080811] Updated weights for policy 0, policy_version 1470 (0.0006)
[2024-11-12 19:55:43,928][4080811] Updated weights for policy 0, policy_version 1480 (0.0006)
[2024-11-12 19:55:45,255][4080811] Updated weights for policy 0, policy_version 1490 (0.0006)
[2024-11-12 19:55:45,321][4080366] Fps is (10 sec: 31129.5, 60 sec: 31470.9, 300 sec: 30515.3). Total num frames: 6103040. Throughput: 0: 7862.3. Samples: 1508594. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:55:45,322][4080366] Avg episode reward: [(0, '28.179')]
[2024-11-12 19:55:45,322][4080798] Saving new best policy, reward=28.179!
[2024-11-12 19:55:46,529][4080811] Updated weights for policy 0, policy_version 1500 (0.0007)
[2024-11-12 19:55:47,851][4080811] Updated weights for policy 0, policy_version 1510 (0.0007)
[2024-11-12 19:55:49,121][4080811] Updated weights for policy 0, policy_version 1520 (0.0007)
[2024-11-12 19:55:50,321][4080366] Fps is (10 sec: 31539.8, 60 sec: 31539.3, 300 sec: 30550.2). Total num frames: 6262784. Throughput: 0: 7844.0. Samples: 1555602. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:55:50,322][4080366] Avg episode reward: [(0, '21.492')]
[2024-11-12 19:55:50,440][4080811] Updated weights for policy 0, policy_version 1530 (0.0007)
[2024-11-12 19:55:51,748][4080811] Updated weights for policy 0, policy_version 1540 (0.0007)
[2024-11-12 19:55:53,031][4080811] Updated weights for policy 0, policy_version 1550 (0.0007)
[2024-11-12 19:55:54,350][4080811] Updated weights for policy 0, policy_version 1560 (0.0008)
[2024-11-12 19:55:55,321][4080366] Fps is (10 sec: 31538.7, 60 sec: 31402.7, 300 sec: 30564.0). Total num frames: 6418432. Throughput: 0: 7852.9. Samples: 1602972. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0)
[2024-11-12 19:55:55,322][4080366] Avg episode reward: [(0, '24.670')]
[2024-11-12 19:55:55,614][4080811] Updated weights for policy 0, policy_version 1570 (0.0007)
[2024-11-12 19:55:56,939][4080811] Updated weights for policy 0, policy_version 1580 (0.0006)
[2024-11-12 19:55:58,251][4080811] Updated weights for policy 0, policy_version 1590 (0.0008)
[2024-11-12 19:55:59,514][4080811] Updated weights for policy 0, policy_version 1600 (0.0006)
[2024-11-12 19:56:00,321][4080366] Fps is (10 sec: 31539.0, 60 sec: 31471.0, 300 sec: 30596.2). Total num frames: 6578176. Throughput: 0: 7860.6. Samples: 1626618. Policy #0 lag: (min: 0.0, avg: 0.3, max: 2.0)
[2024-11-12 19:56:00,322][4080366] Avg episode reward: [(0, '25.765')]
[2024-11-12 19:56:00,806][4080811] Updated weights for policy 0, policy_version 1610 (0.0007)
[2024-11-12 19:56:02,064][4080811] Updated weights for policy 0, policy_version 1620 (0.0007)
[2024-11-12 19:56:03,347][4080811] Updated weights for policy 0, policy_version 1630 (0.0007)
[2024-11-12 19:56:04,649][4080811] Updated weights for policy 0, policy_version 1640 (0.0007)
[2024-11-12 19:56:05,321][4080366] Fps is (10 sec: 31948.1, 60 sec: 31470.8, 300 sec: 30626.9). Total num frames: 6737920. Throughput: 0: 7869.8. Samples: 1674572. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:56:05,323][4080366] Avg episode reward: [(0, '25.263')]
[2024-11-12 19:56:05,945][4080811] Updated weights for policy 0, policy_version 1650 (0.0008)
[2024-11-12 19:56:07,254][4080811] Updated weights for policy 0, policy_version 1660 (0.0007)
[2024-11-12 19:56:08,561][4080811] Updated weights for policy 0, policy_version 1670 (0.0006)
[2024-11-12 19:56:09,851][4080811] Updated weights for policy 0, policy_version 1680 (0.0007)
[2024-11-12 19:56:10,321][4080366] Fps is (10 sec: 31539.4, 60 sec: 31470.9, 300 sec: 30638.2). Total num frames: 6893568. Throughput: 0: 7862.1. Samples: 1721640. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:56:10,322][4080366] Avg episode reward: [(0, '22.771')]
[2024-11-12 19:56:11,208][4080811] Updated weights for policy 0, policy_version 1690 (0.0007)
[2024-11-12 19:56:12,536][4080811] Updated weights for policy 0, policy_version 1700 (0.0007)
[2024-11-12 19:56:13,858][4080811] Updated weights for policy 0, policy_version 1710 (0.0007)
[2024-11-12 19:56:15,150][4080811] Updated weights for policy 0, policy_version 1720 (0.0007)
[2024-11-12 19:56:15,321][4080366] Fps is (10 sec: 31130.5, 60 sec: 31471.1, 300 sec: 30648.8). Total num frames: 7049216. Throughput: 0: 7850.2. Samples: 1744686. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:56:15,322][4080366] Avg episode reward: [(0, '26.287')]
[2024-11-12 19:56:16,466][4080811] Updated weights for policy 0, policy_version 1730 (0.0007)
[2024-11-12 19:56:17,754][4080811] Updated weights for policy 0, policy_version 1740 (0.0007)
[2024-11-12 19:56:19,056][4080811] Updated weights for policy 0, policy_version 1750 (0.0007)
[2024-11-12 19:56:20,321][4080366] Fps is (10 sec: 30719.3, 60 sec: 31334.3, 300 sec: 30641.6). Total num frames: 7200768. Throughput: 0: 7853.4. Samples: 1791816. Policy #0 lag: (min: 0.0, avg: 0.2, max: 1.0)
[2024-11-12 19:56:20,323][4080366] Avg episode reward: [(0, '22.495')]
[2024-11-12 19:56:20,327][4080798] Saving /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001758_7200768.pth...
[2024-11-12 19:56:20,568][4080811] Updated weights for policy 0, policy_version 1760 (0.0008)
[2024-11-12 19:56:21,938][4080811] Updated weights for policy 0, policy_version 1770 (0.0006)
[2024-11-12 19:56:23,176][4080811] Updated weights for policy 0, policy_version 1780 (0.0005)
[2024-11-12 19:56:24,457][4080811] Updated weights for policy 0, policy_version 1790 (0.0006)
[2024-11-12 19:56:25,321][4080366] Fps is (10 sec: 31128.9, 60 sec: 31334.3, 300 sec: 30668.8). Total num frames: 7360512. Throughput: 0: 7845.4. Samples: 1838128. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0)
[2024-11-12 19:56:25,323][4080366] Avg episode reward: [(0, '22.629')]
[2024-11-12 19:56:25,604][4080811] Updated weights for policy 0, policy_version 1800 (0.0005)
[2024-11-12 19:56:26,790][4080811] Updated weights for policy 0, policy_version 1810 (0.0005)
[2024-11-12 19:56:27,987][4080811] Updated weights for policy 0, policy_version 1820 (0.0005)
[2024-11-12 19:56:29,154][4080811] Updated weights for policy 0, policy_version 1830 (0.0005)
[2024-11-12 19:56:30,298][4080811] Updated weights for policy 0, policy_version 1840 (0.0005)
[2024-11-12 19:56:30,321][4080366] Fps is (10 sec: 33586.9, 60 sec: 31675.6, 300 sec: 30761.8). Total num frames: 7536640. Throughput: 0: 7897.6. Samples: 1863986. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:56:30,323][4080366] Avg episode reward: [(0, '24.512')]
[2024-11-12 19:56:31,516][4080811] Updated weights for policy 0, policy_version 1850 (0.0006)
[2024-11-12 19:56:32,677][4080811] Updated weights for policy 0, policy_version 1860 (0.0005)
[2024-11-12 19:56:33,829][4080811] Updated weights for policy 0, policy_version 1870 (0.0005)
[2024-11-12 19:56:35,067][4080811] Updated weights for policy 0, policy_version 1880 (0.0005)
[2024-11-12 19:56:35,321][4080366] Fps is (10 sec: 34816.2, 60 sec: 31948.7, 300 sec: 30834.7). Total num frames: 7708672. Throughput: 0: 8019.2. Samples: 1916466. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0)
[2024-11-12 19:56:35,322][4080366] Avg episode reward: [(0, '21.457')]
[2024-11-12 19:56:36,268][4080811] Updated weights for policy 0, policy_version 1890 (0.0005)
[2024-11-12 19:56:37,412][4080811] Updated weights for policy 0, policy_version 1900 (0.0005)
[2024-11-12 19:56:38,596][4080811] Updated weights for policy 0, policy_version 1910 (0.0006)
[2024-11-12 19:56:39,803][4080811] Updated weights for policy 0, policy_version 1920 (0.0006)
[2024-11-12 19:56:40,321][4080366] Fps is (10 sec: 34407.6, 60 sec: 32222.0, 300 sec: 30904.8). Total num frames: 7880704. Throughput: 0: 8111.2. Samples: 1967976. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0)
[2024-11-12 19:56:40,322][4080366] Avg episode reward: [(0, '26.030')]
[2024-11-12 19:56:40,995][4080811] Updated weights for policy 0, policy_version 1930 (0.0006)
[2024-11-12 19:56:42,161][4080811] Updated weights for policy 0, policy_version 1940 (0.0005)
[2024-11-12 19:56:43,354][4080811] Updated weights for policy 0, policy_version 1950 (0.0006)
[2024-11-12 19:56:43,960][4080798] Stopping Batcher_0...
[2024-11-12 19:56:43,961][4080798] Loop batcher_evt_loop terminating...
[2024-11-12 19:56:43,960][4080366] Component Batcher_0 stopped!
[2024-11-12 19:56:43,961][4080798] Saving /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth...
[2024-11-12 19:56:43,962][4080366] Component RolloutWorker_w0 process died already! Don't wait for it.
[2024-11-12 19:56:43,962][4080366] Component RolloutWorker_w2 process died already! Don't wait for it.
[2024-11-12 19:56:43,987][4080811] Weights refcount: 2 0
[2024-11-12 19:56:43,989][4080811] Stopping InferenceWorker_p0-w0...
[2024-11-12 19:56:43,990][4080811] Loop inference_proc0-0_evt_loop terminating...
[2024-11-12 19:56:43,990][4080366] Component InferenceWorker_p0-w0 stopped!
[2024-11-12 19:56:43,999][4080815] Stopping RolloutWorker_w3...
[2024-11-12 19:56:43,999][4080812] Stopping RolloutWorker_w1...
[2024-11-12 19:56:44,000][4080815] Loop rollout_proc3_evt_loop terminating...
[2024-11-12 19:56:44,000][4080812] Loop rollout_proc1_evt_loop terminating...
[2024-11-12 19:56:44,000][4080817] Stopping RolloutWorker_w4...
[2024-11-12 19:56:44,000][4080819] Stopping RolloutWorker_w7...
[2024-11-12 19:56:44,000][4080818] Stopping RolloutWorker_w6...
[2024-11-12 19:56:44,001][4080819] Loop rollout_proc7_evt_loop terminating...
[2024-11-12 19:56:44,001][4080817] Loop rollout_proc4_evt_loop terminating...
[2024-11-12 19:56:44,001][4080816] Stopping RolloutWorker_w5...
[2024-11-12 19:56:44,001][4080818] Loop rollout_proc6_evt_loop terminating...
[2024-11-12 19:56:44,001][4080816] Loop rollout_proc5_evt_loop terminating...
[2024-11-12 19:56:43,999][4080366] Component RolloutWorker_w3 stopped!
[2024-11-12 19:56:44,002][4080366] Component RolloutWorker_w1 stopped!
[2024-11-12 19:56:44,003][4080366] Component RolloutWorker_w4 stopped!
[2024-11-12 19:56:44,003][4080366] Component RolloutWorker_w7 stopped!
[2024-11-12 19:56:44,004][4080366] Component RolloutWorker_w6 stopped!
[2024-11-12 19:56:44,005][4080366] Component RolloutWorker_w5 stopped!
[2024-11-12 19:56:44,046][4080798] Removing /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000000836_3424256.pth
[2024-11-12 19:56:44,057][4080798] Saving /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth...
[2024-11-12 19:56:44,488][4080798] Stopping LearnerWorker_p0...
[2024-11-12 19:56:44,489][4080798] Loop learner_proc0_evt_loop terminating...
[2024-11-12 19:56:44,489][4080366] Component LearnerWorker_p0 stopped!
[2024-11-12 19:56:44,490][4080366] Waiting for process learner_proc0 to stop...
[2024-11-12 19:56:45,109][4080366] Waiting for process inference_proc0-0 to join...
[2024-11-12 19:56:45,110][4080366] Waiting for process rollout_proc0 to join...
[2024-11-12 19:56:45,110][4080366] Waiting for process rollout_proc1 to join...
[2024-11-12 19:56:45,111][4080366] Waiting for process rollout_proc2 to join...
[2024-11-12 19:56:45,111][4080366] Waiting for process rollout_proc3 to join...
[2024-11-12 19:56:45,111][4080366] Waiting for process rollout_proc4 to join...
[2024-11-12 19:56:45,111][4080366] Waiting for process rollout_proc5 to join...
[2024-11-12 19:56:45,111][4080366] Waiting for process rollout_proc6 to join...
[2024-11-12 19:56:45,112][4080366] Waiting for process rollout_proc7 to join...
[2024-11-12 19:56:45,112][4080366] Batcher 0 profile tree view:
batching: 26.9027, releasing_batches: 0.0357
[2024-11-12 19:56:45,112][4080366] InferenceWorker_p0-w0 profile tree view:
wait_policy: 0.0000
  wait_policy_total: 3.7839
update_model: 3.9104
  weight_update: 0.0005
one_step: 0.0020
  handle_policy_step: 233.9468
    deserialize: 12.5786, stack: 1.3074, obs_to_device_normalize: 59.5991, forward: 114.8215, send_messages: 10.9577
    prepare_outputs: 25.3487
      to_cpu: 14.6066
[2024-11-12 19:56:45,112][4080366] Learner 0 profile tree view:
misc: 0.0097, prepare_batch: 9.7770
train: 26.0776
  epoch_init: 0.0065, minibatch_init: 0.0074, losses_postprocess: 0.6240, kl_divergence: 0.5407, after_optimizer: 6.5999
  calculate_losses: 9.5364
    losses_init: 0.0028, forward_head: 0.7589, bptt_initial: 5.6054, tail: 0.6684, advantages_returns: 0.1629, losses: 0.9611
    bptt: 1.2044
      bptt_forward_core: 1.1543
  update: 8.3829
    clip: 0.6208
[2024-11-12 19:56:45,113][4080366] RolloutWorker_w7 profile tree view:
wait_for_trajectories: 0.2070, enqueue_policy_requests: 10.1650, env_step: 142.2907, overhead: 12.2160, complete_rollouts: 0.3404
save_policy_outputs: 10.5431
  split_output_tensors: 4.9354
[2024-11-12 19:56:45,113][4080366] Loop Runner_EvtLoop terminating...
[2024-11-12 19:56:45,113][4080366] Runner profile tree view:
main_loop: 263.5364
[2024-11-12 19:56:45,113][4080366] Collected {0: 8007680}, FPS: 30385.5
[2024-11-12 19:57:53,988][4080366] Loading existing experiment configuration from /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/config.json
[2024-11-12 19:57:53,988][4080366] Overriding arg 'num_workers' with value 1 passed from command line
[2024-11-12 19:57:53,989][4080366] Adding new argument 'no_render'=True that is not in the saved config file!
[2024-11-12 19:57:53,989][4080366] Adding new argument 'save_video'=True that is not in the saved config file!
[2024-11-12 19:57:53,989][4080366] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2024-11-12 19:57:53,989][4080366] Adding new argument 'video_name'=None that is not in the saved config file!
[2024-11-12 19:57:53,990][4080366] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
[2024-11-12 19:57:53,990][4080366] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2024-11-12 19:57:53,990][4080366] Adding new argument 'push_to_hub'=False that is not in the saved config file!
[2024-11-12 19:57:53,990][4080366] Adding new argument 'hf_repository'=None that is not in the saved config file!
[2024-11-12 19:57:53,990][4080366] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2024-11-12 19:57:53,991][4080366] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2024-11-12 19:57:53,991][4080366] Adding new argument 'train_script'=None that is not in the saved config file!
[2024-11-12 19:57:53,991][4080366] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2024-11-12 19:57:53,991][4080366] Using frameskip 1 and render_action_repeat=4 for evaluation
[2024-11-12 19:57:54,005][4080366] Doom resolution: 160x120, resize resolution: (128, 72)
[2024-11-12 19:57:54,007][4080366] RunningMeanStd input shape: (3, 72, 128)
[2024-11-12 19:57:54,008][4080366] RunningMeanStd input shape: (1,)
[2024-11-12 19:57:54,013][4080366] ConvEncoder: input_channels=3
[2024-11-12 19:57:54,079][4080366] Conv encoder output size: 512
[2024-11-12 19:57:54,079][4080366] Policy head output size: 512
[2024-11-12 19:57:54,176][4080366] Loading state from checkpoint /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth...
[2024-11-12 19:57:54,892][4080366] Num frames 100...
[2024-11-12 19:57:54,939][4080366] Num frames 200...
[2024-11-12 19:57:54,997][4080366] Num frames 300...
[2024-11-12 19:57:55,045][4080366] Num frames 400...
[2024-11-12 19:57:55,096][4080366] Num frames 500...
[2024-11-12 19:57:55,141][4080366] Num frames 600...
[2024-11-12 19:57:55,183][4080366] Num frames 700...
[2024-11-12 19:57:55,228][4080366] Num frames 800...
[2024-11-12 19:57:55,308][4080366] Avg episode rewards: #0: 24.640, true rewards: #0: 8.640
[2024-11-12 19:57:55,310][4080366] Avg episode reward: 24.640, avg true_objective: 8.640
[2024-11-12 19:57:55,370][4080366] Num frames 900...
[2024-11-12 19:57:55,417][4080366] Num frames 1000...
[2024-11-12 19:57:55,461][4080366] Num frames 1100...
[2024-11-12 19:57:55,505][4080366] Num frames 1200...
[2024-11-12 19:57:55,549][4080366] Num frames 1300...
[2024-11-12 19:57:55,591][4080366] Num frames 1400...
[2024-11-12 19:57:55,632][4080366] Num frames 1500...
[2024-11-12 19:57:55,674][4080366] Num frames 1600...
[2024-11-12 19:57:55,716][4080366] Num frames 1700...
[2024-11-12 19:57:55,763][4080366] Num frames 1800...
[2024-11-12 19:57:55,806][4080366] Num frames 1900...
[2024-11-12 19:57:55,849][4080366] Num frames 2000...
[2024-11-12 19:57:55,895][4080366] Num frames 2100...
[2024-11-12 19:57:55,941][4080366] Num frames 2200...
[2024-11-12 19:57:55,992][4080366] Num frames 2300...
[2024-11-12 19:57:56,071][4080366] Num frames 2400...
[2024-11-12 19:57:56,122][4080366] Avg episode rewards: #0: 28.000, true rewards: #0: 12.000
[2024-11-12 19:57:56,124][4080366] Avg episode reward: 28.000, avg true_objective: 12.000
[2024-11-12 19:57:56,195][4080366] Num frames 2500...
[2024-11-12 19:57:56,242][4080366] Num frames 2600...
[2024-11-12 19:57:56,307][4080366] Num frames 2700...
[2024-11-12 19:57:56,377][4080366] Num frames 2800...
[2024-11-12 19:57:56,424][4080366] Num frames 2900...
[2024-11-12 19:57:56,474][4080366] Num frames 3000...
[2024-11-12 19:57:56,520][4080366] Num frames 3100...
[2024-11-12 19:57:56,566][4080366] Num frames 3200...
[2024-11-12 19:57:56,644][4080366] Avg episode rewards: #0: 25.107, true rewards: #0: 10.773
[2024-11-12 19:57:56,645][4080366] Avg episode reward: 25.107, avg true_objective: 10.773
[2024-11-12 19:57:56,688][4080366] Num frames 3300...
[2024-11-12 19:57:56,741][4080366] Num frames 3400...
[2024-11-12 19:57:56,786][4080366] Num frames 3500...
[2024-11-12 19:57:56,840][4080366] Num frames 3600...
[2024-11-12 19:57:56,888][4080366] Num frames 3700...
[2024-11-12 19:57:56,936][4080366] Num frames 3800...
[2024-11-12 19:57:56,982][4080366] Num frames 3900...
[2024-11-12 19:57:57,039][4080366] Avg episode rewards: #0: 23.010, true rewards: #0: 9.760
[2024-11-12 19:57:57,041][4080366] Avg episode reward: 23.010, avg true_objective: 9.760
[2024-11-12 19:57:57,144][4080366] Num frames 4000...
[2024-11-12 19:57:57,197][4080366] Num frames 4100...
[2024-11-12 19:57:57,243][4080366] Num frames 4200...
[2024-11-12 19:57:57,303][4080366] Num frames 4300...
[2024-11-12 19:57:57,354][4080366] Num frames 4400...
[2024-11-12 19:57:57,398][4080366] Num frames 4500...
[2024-11-12 19:57:57,443][4080366] Num frames 4600...
[2024-11-12 19:57:57,491][4080366] Num frames 4700...
[2024-11-12 19:57:57,535][4080366] Num frames 4800...
[2024-11-12 19:57:57,579][4080366] Num frames 4900...
[2024-11-12 19:57:57,623][4080366] Num frames 5000...
[2024-11-12 19:57:57,673][4080366] Num frames 5100...
[2024-11-12 19:57:57,734][4080366] Avg episode rewards: #0: 23.640, true rewards: #0: 10.240
[2024-11-12 19:57:57,739][4080366] Avg episode reward: 23.640, avg true_objective: 10.240
[2024-11-12 19:57:57,797][4080366] Num frames 5200...
[2024-11-12 19:57:57,838][4080366] Num frames 5300...
[2024-11-12 19:57:57,884][4080366] Num frames 5400...
[2024-11-12 19:57:57,963][4080366] Num frames 5500...
[2024-11-12 19:57:58,016][4080366] Num frames 5600...
[2024-11-12 19:57:58,096][4080366] Avg episode rewards: #0: 21.607, true rewards: #0: 9.440
[2024-11-12 19:57:58,099][4080366] Avg episode reward: 21.607, avg true_objective: 9.440
[2024-11-12 19:57:58,136][4080366] Num frames 5700...
[2024-11-12 19:57:58,181][4080366] Num frames 5800...
[2024-11-12 19:57:58,230][4080366] Num frames 5900...
[2024-11-12 19:57:58,273][4080366] Num frames 6000...
[2024-11-12 19:57:58,317][4080366] Num frames 6100...
[2024-11-12 19:57:58,361][4080366] Num frames 6200...
[2024-11-12 19:57:58,406][4080366] Num frames 6300...
[2024-11-12 19:57:58,450][4080366] Num frames 6400...
[2024-11-12 19:57:58,497][4080366] Num frames 6500...
[2024-11-12 19:57:58,544][4080366] Num frames 6600...
[2024-11-12 19:57:58,590][4080366] Num frames 6700...
[2024-11-12 19:57:58,635][4080366] Num frames 6800...
[2024-11-12 19:57:58,679][4080366] Num frames 6900...
[2024-11-12 19:57:58,724][4080366] Num frames 7000...
[2024-11-12 19:57:58,766][4080366] Num frames 7100...
[2024-11-12 19:57:58,807][4080366] Num frames 7200...
[2024-11-12 19:57:58,875][4080366] Avg episode rewards: #0: 24.046, true rewards: #0: 10.331
[2024-11-12 19:57:58,877][4080366] Avg episode reward: 24.046, avg true_objective: 10.331
[2024-11-12 19:57:58,932][4080366] Num frames 7300...
[2024-11-12 19:57:58,981][4080366] Num frames 7400...
[2024-11-12 19:57:59,038][4080366] Num frames 7500...
[2024-11-12 19:57:59,082][4080366] Num frames 7600...
[2024-11-12 19:57:59,125][4080366] Num frames 7700...
[2024-11-12 19:57:59,168][4080366] Num frames 7800...
[2024-11-12 19:57:59,228][4080366] Num frames 7900...
[2024-11-12 19:57:59,270][4080366] Num frames 8000...
[2024-11-12 19:57:59,314][4080366] Num frames 8100...
[2024-11-12 19:57:59,360][4080366] Num frames 8200...
[2024-11-12 19:57:59,406][4080366] Num frames 8300...
[2024-11-12 19:57:59,451][4080366] Num frames 8400...
[2024-11-12 19:57:59,524][4080366] Avg episode rewards: #0: 24.935, true rewards: #0: 10.560
[2024-11-12 19:57:59,526][4080366] Avg episode reward: 24.935, avg true_objective: 10.560
[2024-11-12 19:57:59,571][4080366] Num frames 8500...
[2024-11-12 19:57:59,615][4080366] Num frames 8600...
[2024-11-12 19:57:59,659][4080366] Num frames 8700...
[2024-11-12 19:57:59,702][4080366] Num frames 8800...
[2024-11-12 19:57:59,750][4080366] Num frames 8900...
[2024-11-12 19:57:59,796][4080366] Num frames 9000...
[2024-11-12 19:57:59,841][4080366] Num frames 9100...
[2024-11-12 19:57:59,884][4080366] Num frames 9200...
[2024-11-12 19:57:59,932][4080366] Num frames 9300...
[2024-11-12 19:57:59,993][4080366] Num frames 9400...
[2024-11-12 19:58:00,057][4080366] Num frames 9500...
[2024-11-12 19:58:00,105][4080366] Num frames 9600...
[2024-11-12 19:58:00,185][4080366] Avg episode rewards: #0: 25.288, true rewards: #0: 10.732
[2024-11-12 19:58:00,187][4080366] Avg episode reward: 25.288, avg true_objective: 10.732
[2024-11-12 19:58:00,224][4080366] Num frames 9700...
[2024-11-12 19:58:00,268][4080366] Num frames 9800...
[2024-11-12 19:58:00,329][4080366] Avg episode rewards: #0: 23.019, true rewards: #0: 9.819
[2024-11-12 19:58:00,331][4080366] Avg episode reward: 23.019, avg true_objective: 9.819
[2024-11-12 19:58:10,093][4080366] Replay video saved to /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/replay.mp4!
[2024-11-12 20:00:58,329][4080366] Loading existing experiment configuration from /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/config.json
[2024-11-12 20:00:58,330][4080366] Overriding arg 'num_workers' with value 1 passed from command line
[2024-11-12 20:00:58,331][4080366] Adding new argument 'no_render'=True that is not in the saved config file!
[2024-11-12 20:00:58,332][4080366] Adding new argument 'save_video'=True that is not in the saved config file!
[2024-11-12 20:00:58,332][4080366] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
[2024-11-12 20:00:58,333][4080366] Adding new argument 'video_name'=None that is not in the saved config file!
[2024-11-12 20:00:58,333][4080366] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
[2024-11-12 20:00:58,334][4080366] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
[2024-11-12 20:00:58,334][4080366] Adding new argument 'push_to_hub'=True that is not in the saved config file!
[2024-11-12 20:00:58,335][4080366] Adding new argument 'hf_repository'='sun-s/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
[2024-11-12 20:00:58,336][4080366] Adding new argument 'policy_index'=0 that is not in the saved config file!
[2024-11-12 20:00:58,336][4080366] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
[2024-11-12 20:00:58,337][4080366] Adding new argument 'train_script'=None that is not in the saved config file!
[2024-11-12 20:00:58,338][4080366] Adding new argument 'enjoy_script'=None that is not in the saved config file!
[2024-11-12 20:00:58,338][4080366] Using frameskip 1 and render_action_repeat=4 for evaluation
[2024-11-12 20:00:58,358][4080366] RunningMeanStd input shape: (3, 72, 128)
[2024-11-12 20:00:58,359][4080366] RunningMeanStd input shape: (1,)
[2024-11-12 20:00:58,365][4080366] ConvEncoder: input_channels=3
[2024-11-12 20:00:58,395][4080366] Conv encoder output size: 512
[2024-11-12 20:00:58,396][4080366] Policy head output size: 512
[2024-11-12 20:00:58,425][4080366] Loading state from checkpoint /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth...
[2024-11-12 20:00:58,917][4080366] Num frames 100...
[2024-11-12 20:00:58,973][4080366] Num frames 200...
[2024-11-12 20:00:59,030][4080366] Num frames 300...
[2024-11-12 20:00:59,082][4080366] Num frames 400...
[2024-11-12 20:00:59,140][4080366] Num frames 500...
[2024-11-12 20:00:59,197][4080366] Num frames 600...
[2024-11-12 20:00:59,249][4080366] Num frames 700...
[2024-11-12 20:00:59,306][4080366] Num frames 800...
[2024-11-12 20:00:59,362][4080366] Num frames 900...
[2024-11-12 20:00:59,415][4080366] Num frames 1000...
[2024-11-12 20:00:59,469][4080366] Num frames 1100...
[2024-11-12 20:00:59,518][4080366] Num frames 1200...
[2024-11-12 20:00:59,567][4080366] Num frames 1300...
[2024-11-12 20:00:59,619][4080366] Num frames 1400...
[2024-11-12 20:00:59,709][4080366] Avg episode rewards: #0: 40.760, true rewards: #0: 14.760
[2024-11-12 20:00:59,710][4080366] Avg episode reward: 40.760, avg true_objective: 14.760
[2024-11-12 20:00:59,721][4080366] Num frames 1500...
[2024-11-12 20:00:59,771][4080366] Num frames 1600...
[2024-11-12 20:00:59,815][4080366] Num frames 1700...
[2024-11-12 20:00:59,858][4080366] Num frames 1800...
[2024-11-12 20:00:59,923][4080366] Avg episode rewards: #0: 22.640, true rewards: #0: 9.140
[2024-11-12 20:00:59,925][4080366] Avg episode reward: 22.640, avg true_objective: 9.140
[2024-11-12 20:00:59,974][4080366] Num frames 1900...
[2024-11-12 20:01:00,020][4080366] Num frames 2000...
[2024-11-12 20:01:00,066][4080366] Num frames 2100...
[2024-11-12 20:01:00,108][4080366] Num frames 2200...
[2024-11-12 20:01:00,155][4080366] Num frames 2300...
[2024-11-12 20:01:00,205][4080366] Num frames 2400...
[2024-11-12 20:01:00,259][4080366] Num frames 2500...
[2024-11-12 20:01:00,303][4080366] Num frames 2600...
[2024-11-12 20:01:00,347][4080366] Num frames 2700...
[2024-11-12 20:01:00,391][4080366] Num frames 2800...
[2024-11-12 20:01:00,434][4080366] Num frames 2900...
[2024-11-12 20:01:00,477][4080366] Num frames 3000...
[2024-11-12 20:01:00,521][4080366] Num frames 3100...
[2024-11-12 20:01:00,572][4080366] Num frames 3200...
[2024-11-12 20:01:00,613][4080366] Num frames 3300...
[2024-11-12 20:01:00,654][4080366] Num frames 3400...
[2024-11-12 20:01:00,695][4080366] Num frames 3500...
[2024-11-12 20:01:00,737][4080366] Num frames 3600...
[2024-11-12 20:01:00,814][4080366] Avg episode rewards: #0: 30.160, true rewards: #0: 12.160
[2024-11-12 20:01:00,817][4080366] Avg episode reward: 30.160, avg true_objective: 12.160
[2024-11-12 20:01:00,892][4080366] Num frames 3700...
[2024-11-12 20:01:00,939][4080366] Num frames 3800...
[2024-11-12 20:01:00,985][4080366] Num frames 3900...
[2024-11-12 20:01:01,026][4080366] Num frames 4000...
[2024-11-12 20:01:01,095][4080366] Num frames 4100...
[2024-11-12 20:01:01,142][4080366] Num frames 4200...
[2024-11-12 20:01:01,189][4080366] Num frames 4300...
[2024-11-12 20:01:01,232][4080366] Num frames 4400...
[2024-11-12 20:01:01,315][4080366] Num frames 4500...
[2024-11-12 20:01:01,358][4080366] Num frames 4600...
[2024-11-12 20:01:01,400][4080366] Num frames 4700...
[2024-11-12 20:01:01,441][4080366] Num frames 4800...
[2024-11-12 20:01:01,507][4080366] Avg episode rewards: #0: 30.080, true rewards: #0: 12.080
[2024-11-12 20:01:01,509][4080366] Avg episode reward: 30.080, avg true_objective: 12.080
[2024-11-12 20:01:01,569][4080366] Num frames 4900...
[2024-11-12 20:01:01,611][4080366] Num frames 5000...
[2024-11-12 20:01:01,651][4080366] Num frames 5100...
[2024-11-12 20:01:01,693][4080366] Num frames 5200...
[2024-11-12 20:01:01,737][4080366] Num frames 5300...
[2024-11-12 20:01:01,779][4080366] Num frames 5400...
[2024-11-12 20:01:01,828][4080366] Num frames 5500...
[2024-11-12 20:01:01,895][4080366] Avg episode rewards: #0: 26.472, true rewards: #0: 11.072
[2024-11-12 20:01:01,896][4080366] Avg episode reward: 26.472, avg true_objective: 11.072
[2024-11-12 20:01:01,936][4080366] Num frames 5600...
[2024-11-12 20:01:01,977][4080366] Num frames 5700...
[2024-11-12 20:01:02,019][4080366] Num frames 5800...
[2024-11-12 20:01:02,061][4080366] Num frames 5900...
[2024-11-12 20:01:02,102][4080366] Num frames 6000...
[2024-11-12 20:01:02,149][4080366] Num frames 6100...
[2024-11-12 20:01:02,199][4080366] Num frames 6200...
[2024-11-12 20:01:02,259][4080366] Num frames 6300...
[2024-11-12 20:01:02,321][4080366] Avg episode rewards: #0: 24.703, true rewards: #0: 10.537
[2024-11-12 20:01:02,323][4080366] Avg episode reward: 24.703, avg true_objective: 10.537
[2024-11-12 20:01:02,369][4080366] Num frames 6400...
[2024-11-12 20:01:02,414][4080366] Num frames 6500...
[2024-11-12 20:01:02,456][4080366] Num frames 6600...
[2024-11-12 20:01:02,497][4080366] Num frames 6700...
[2024-11-12 20:01:02,538][4080366] Num frames 6800...
[2024-11-12 20:01:02,626][4080366] Avg episode rewards: #0: 22.551, true rewards: #0: 9.837
[2024-11-12 20:01:02,628][4080366] Avg episode reward: 22.551, avg true_objective: 9.837
[2024-11-12 20:01:02,653][4080366] Num frames 6900...
[2024-11-12 20:01:02,703][4080366] Num frames 7000...
[2024-11-12 20:01:02,746][4080366] Num frames 7100...
[2024-11-12 20:01:02,791][4080366] Num frames 7200...
[2024-11-12 20:01:02,835][4080366] Num frames 7300...
[2024-11-12 20:01:02,880][4080366] Num frames 7400...
[2024-11-12 20:01:02,925][4080366] Num frames 7500...
[2024-11-12 20:01:02,968][4080366] Num frames 7600...
[2024-11-12 20:01:03,015][4080366] Num frames 7700...
[2024-11-12 20:01:03,062][4080366] Num frames 7800...
[2024-11-12 20:01:03,109][4080366] Num frames 7900...
[2024-11-12 20:01:03,150][4080366] Num frames 8000...
[2024-11-12 20:01:03,194][4080366] Num frames 8100...
[2024-11-12 20:01:03,237][4080366] Num frames 8200...
[2024-11-12 20:01:03,281][4080366] Num frames 8300...
[2024-11-12 20:01:03,325][4080366] Num frames 8400...
[2024-11-12 20:01:03,372][4080366] Num frames 8500...
[2024-11-12 20:01:03,425][4080366] Num frames 8600...
[2024-11-12 20:01:03,497][4080366] Avg episode rewards: #0: 24.932, true rewards: #0: 10.807
[2024-11-12 20:01:03,499][4080366] Avg episode reward: 24.932, avg true_objective: 10.807
[2024-11-12 20:01:03,543][4080366] Num frames 8700...
[2024-11-12 20:01:03,601][4080366] Num frames 8800...
[2024-11-12 20:01:03,644][4080366] Num frames 8900...
[2024-11-12 20:01:03,686][4080366] Num frames 9000...
[2024-11-12 20:01:03,732][4080366] Num frames 9100...
[2024-11-12 20:01:03,776][4080366] Num frames 9200...
[2024-11-12 20:01:03,847][4080366] Num frames 9300...
[2024-11-12 20:01:03,895][4080366] Num frames 9400...
[2024-11-12 20:01:03,943][4080366] Num frames 9500...
[2024-11-12 20:01:03,986][4080366] Num frames 9600...
[2024-11-12 20:01:04,031][4080366] Num frames 9700...
[2024-11-12 20:01:04,075][4080366] Num frames 9800...
[2024-11-12 20:01:04,148][4080366] Avg episode rewards: #0: 25.379, true rewards: #0: 10.934
[2024-11-12 20:01:04,151][4080366] Avg episode reward: 25.379, avg true_objective: 10.934
[2024-11-12 20:01:04,226][4080366] Num frames 9900...
[2024-11-12 20:01:04,273][4080366] Num frames 10000...
[2024-11-12 20:01:04,316][4080366] Num frames 10100...
[2024-11-12 20:01:04,358][4080366] Num frames 10200...
[2024-11-12 20:01:04,400][4080366] Num frames 10300...
[2024-11-12 20:01:04,443][4080366] Num frames 10400...
[2024-11-12 20:01:04,485][4080366] Num frames 10500...
[2024-11-12 20:01:04,529][4080366] Num frames 10600...
[2024-11-12 20:01:04,570][4080366] Num frames 10700...
[2024-11-12 20:01:04,613][4080366] Num frames 10800...
[2024-11-12 20:01:04,680][4080366] Avg episode rewards: #0: 24.633, true rewards: #0: 10.833
[2024-11-12 20:01:04,682][4080366] Avg episode reward: 24.633, avg true_objective: 10.833
[2024-11-12 20:01:15,228][4080366] Replay video saved to /media/kemove/17a64fef-179c-4620-a965-511df0584b38/home/yunpeng/transformers/deep-rl-class-main/notebooks/unit8/train_dir/default_experiment/replay.mp4!