| [2023-09-05 10:58:35,050][272918] Saving configuration to /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/config.json... | |
| [2023-09-05 10:58:35,052][272918] Rollout worker 0 uses device cpu | |
| [2023-09-05 10:58:35,052][272918] Rollout worker 1 uses device cpu | |
| [2023-09-05 10:58:35,052][272918] Rollout worker 2 uses device cpu | |
| [2023-09-05 10:58:35,052][272918] Rollout worker 3 uses device cpu | |
| [2023-09-05 10:58:35,053][272918] Rollout worker 4 uses device cpu | |
| [2023-09-05 10:58:35,053][272918] Rollout worker 5 uses device cpu | |
| [2023-09-05 10:58:35,053][272918] Rollout worker 6 uses device cpu | |
| [2023-09-05 10:58:35,054][272918] Rollout worker 7 uses device cpu | |
| [2023-09-05 10:58:35,130][272918] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-09-05 10:58:35,130][272918] InferenceWorker_p0-w0: min num requests: 2 | |
| [2023-09-05 10:58:35,163][272918] Starting all processes... | |
| [2023-09-05 10:58:35,163][272918] Starting process learner_proc0 | |
| [2023-09-05 10:58:37,092][272918] Starting all processes... | |
| [2023-09-05 10:58:37,106][272918] Starting process inference_proc0-0 | |
| [2023-09-05 10:58:37,107][272918] Starting process rollout_proc0 | |
| [2023-09-05 10:58:37,108][273075] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-09-05 10:58:37,109][273075] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2023-09-05 10:58:37,107][272918] Starting process rollout_proc1 | |
| [2023-09-05 10:58:37,108][272918] Starting process rollout_proc2 | |
| [2023-09-05 10:58:37,111][272918] Starting process rollout_proc3 | |
| [2023-09-05 10:58:37,118][273075] Num visible devices: 1 | |
| [2023-09-05 10:58:37,111][272918] Starting process rollout_proc4 | |
| [2023-09-05 10:58:37,112][272918] Starting process rollout_proc5 | |
| [2023-09-05 10:58:37,114][272918] Starting process rollout_proc6 | |
| [2023-09-05 10:58:37,114][272918] Starting process rollout_proc7 | |
| [2023-09-05 10:58:37,198][273075] Starting seed is not provided | |
| [2023-09-05 10:58:37,199][273075] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-09-05 10:58:37,200][273075] Initializing actor-critic model on device cuda:0 | |
| [2023-09-05 10:58:37,201][273075] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-09-05 10:58:37,204][273075] RunningMeanStd input shape: (1,) | |
| [2023-09-05 10:58:37,245][273075] ConvEncoder: input_channels=3 | |
| [2023-09-05 10:58:37,545][273075] Conv encoder output size: 512 | |
| [2023-09-05 10:58:37,546][273075] Policy head output size: 512 | |
| [2023-09-05 10:58:37,568][273075] Created Actor Critic model with architecture: | |
| [2023-09-05 10:58:37,568][273075] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2023-09-05 10:58:40,269][273148] Worker 2 uses CPU cores [2] | |
| [2023-09-05 10:58:40,288][273075] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2023-09-05 10:58:40,291][273075] No checkpoints found | |
| [2023-09-05 10:58:40,292][273075] Did not load from checkpoint, starting from scratch! | |
| [2023-09-05 10:58:40,293][273075] Initialized policy 0 weights for model version 0 | |
| [2023-09-05 10:58:40,301][273075] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-09-05 10:58:40,325][273075] LearnerWorker_p0 finished initialization! | |
| [2023-09-05 10:58:40,825][273157] Worker 4 uses CPU cores [4] | |
| [2023-09-05 10:58:41,270][273146] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2023-09-05 10:58:41,271][273146] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2023-09-05 10:58:41,280][273146] Num visible devices: 1 | |
| [2023-09-05 10:58:41,504][273146] RunningMeanStd input shape: (3, 72, 128) | |
| [2023-09-05 10:58:41,507][273146] RunningMeanStd input shape: (1,) | |
| [2023-09-05 10:58:41,573][273146] ConvEncoder: input_channels=3 | |
| [2023-09-05 10:58:41,784][273146] Conv encoder output size: 512 | |
| [2023-09-05 10:58:41,786][273146] Policy head output size: 512 | |
| [2023-09-05 10:58:41,847][273147] Worker 0 uses CPU cores [0] | |
| [2023-09-05 10:58:42,376][273149] Worker 1 uses CPU cores [1] | |
| [2023-09-05 10:58:42,765][273165] Worker 6 uses CPU cores [6] | |
| [2023-09-05 10:58:43,101][273160] Worker 7 uses CPU cores [7] | |
| [2023-09-05 10:58:43,329][273162] Worker 3 uses CPU cores [3] | |
| [2023-09-05 10:58:43,474][272918] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-09-05 10:58:43,484][273164] Worker 5 uses CPU cores [5] | |
| [2023-09-05 10:58:44,287][272918] Inference worker 0-0 is ready! | |
| [2023-09-05 10:58:44,287][272918] All inference workers are ready! Signal rollout workers to start! | |
| [2023-09-05 10:58:44,335][272918] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2023-09-05 10:58:44,366][273160] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,367][273148] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,370][273147] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,376][273149] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,386][273165] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,389][273162] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,392][273164] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,409][273157] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2023-09-05 10:58:44,936][273148] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:44,936][273147] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:44,939][273164] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:44,939][273160] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:44,941][273149] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:44,941][273162] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:45,312][273162] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,313][273147] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,314][273148] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,320][273164] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,325][273149] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,375][273165] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:45,388][273157] Decorrelating experience for 0 frames... | |
| [2023-09-05 10:58:45,644][273160] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,731][273147] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:45,745][273164] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:45,754][273165] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:45,838][273148] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:46,055][273147] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,072][273160] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:46,156][273162] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:46,164][273149] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:46,167][273164] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,202][273165] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:46,485][273160] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,537][273148] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,585][273157] Decorrelating experience for 32 frames... | |
| [2023-09-05 10:58:46,609][273149] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,636][273162] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:46,821][273165] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:47,141][273157] Decorrelating experience for 64 frames... | |
| [2023-09-05 10:58:47,523][273157] Decorrelating experience for 96 frames... | |
| [2023-09-05 10:58:47,915][273075] Signal inference workers to stop experience collection... | |
| [2023-09-05 10:58:47,920][273146] InferenceWorker_p0-w0: stopping experience collection | |
| [2023-09-05 10:58:48,900][273075] Signal inference workers to resume experience collection... | |
| [2023-09-05 10:58:48,901][273146] InferenceWorker_p0-w0: resuming experience collection | |
| [2023-09-05 10:58:49,335][272918] Fps is (10 sec: 698.8, 60 sec: 698.8, 300 sec: 698.8). Total num frames: 4096. Throughput: 0: 174.0. Samples: 1020. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2023-09-05 10:58:49,335][272918] Avg episode reward: [(0, '2.753')] | |
| [2023-09-05 10:58:51,929][273146] Updated weights for policy 0, policy_version 10 (0.0297) | |
| [2023-09-05 10:58:54,335][272918] Fps is (10 sec: 6963.3, 60 sec: 6411.3, 300 sec: 6411.3). Total num frames: 69632. Throughput: 0: 1441.0. Samples: 15650. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-09-05 10:58:54,335][272918] Avg episode reward: [(0, '4.479')] | |
| [2023-09-05 10:58:54,955][273146] Updated weights for policy 0, policy_version 20 (0.0021) | |
| [2023-09-05 10:58:55,121][272918] Heartbeat connected on Batcher_0 | |
| [2023-09-05 10:58:55,125][272918] Heartbeat connected on LearnerWorker_p0 | |
| [2023-09-05 10:58:55,136][272918] Heartbeat connected on RolloutWorker_w0 | |
| [2023-09-05 10:58:55,137][272918] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2023-09-05 10:58:55,140][272918] Heartbeat connected on RolloutWorker_w1 | |
| [2023-09-05 10:58:55,142][272918] Heartbeat connected on RolloutWorker_w2 | |
| [2023-09-05 10:58:55,149][272918] Heartbeat connected on RolloutWorker_w3 | |
| [2023-09-05 10:58:55,156][272918] Heartbeat connected on RolloutWorker_w5 | |
| [2023-09-05 10:58:55,157][272918] Heartbeat connected on RolloutWorker_w6 | |
| [2023-09-05 10:58:55,169][272918] Heartbeat connected on RolloutWorker_w7 | |
| [2023-09-05 10:58:55,177][272918] Heartbeat connected on RolloutWorker_w4 | |
| [2023-09-05 10:58:57,912][273146] Updated weights for policy 0, policy_version 30 (0.0023) | |
| [2023-09-05 10:58:59,335][272918] Fps is (10 sec: 13516.9, 60 sec: 8780.3, 300 sec: 8780.3). Total num frames: 139264. Throughput: 0: 2262.8. Samples: 35890. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-09-05 10:58:59,335][272918] Avg episode reward: [(0, '4.458')] | |
| [2023-09-05 10:58:59,341][273075] Saving new best policy, reward=4.458! | |
| [2023-09-05 10:59:01,040][273146] Updated weights for policy 0, policy_version 40 (0.0024) | |
| [2023-09-05 10:59:04,120][273146] Updated weights for policy 0, policy_version 50 (0.0023) | |
| [2023-09-05 10:59:04,334][272918] Fps is (10 sec: 13516.9, 60 sec: 9817.5, 300 sec: 9817.5). Total num frames: 204800. Throughput: 0: 2193.2. Samples: 45752. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 10:59:04,335][272918] Avg episode reward: [(0, '4.259')] | |
| [2023-09-05 10:59:07,182][273146] Updated weights for policy 0, policy_version 60 (0.0022) | |
| [2023-09-05 10:59:09,335][272918] Fps is (10 sec: 13516.9, 60 sec: 10611.8, 300 sec: 10611.8). Total num frames: 274432. Throughput: 0: 2555.9. Samples: 66098. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-09-05 10:59:09,335][272918] Avg episode reward: [(0, '4.337')] | |
| [2023-09-05 10:59:10,137][273146] Updated weights for policy 0, policy_version 70 (0.0026) | |
| [2023-09-05 10:59:13,193][273146] Updated weights for policy 0, policy_version 80 (0.0021) | |
| [2023-09-05 10:59:14,335][272918] Fps is (10 sec: 13516.7, 60 sec: 11016.2, 300 sec: 11016.2). Total num frames: 339968. Throughput: 0: 2790.7. Samples: 86124. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-09-05 10:59:14,335][272918] Avg episode reward: [(0, '4.386')] | |
| [2023-09-05 10:59:16,324][273146] Updated weights for policy 0, policy_version 90 (0.0021) | |
| [2023-09-05 10:59:19,335][272918] Fps is (10 sec: 13107.2, 60 sec: 11307.7, 300 sec: 11307.7). Total num frames: 405504. Throughput: 0: 2681.4. Samples: 96156. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 10:59:19,335][272918] Avg episode reward: [(0, '4.795')] | |
| [2023-09-05 10:59:19,341][273075] Saving new best policy, reward=4.795! | |
| [2023-09-05 10:59:19,499][273146] Updated weights for policy 0, policy_version 100 (0.0019) | |
| [2023-09-05 10:59:22,787][273146] Updated weights for policy 0, policy_version 110 (0.0022) | |
| [2023-09-05 10:59:24,335][272918] Fps is (10 sec: 12697.4, 60 sec: 11427.6, 300 sec: 11427.6). Total num frames: 466944. Throughput: 0: 2810.7. Samples: 114848. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 10:59:24,337][272918] Avg episode reward: [(0, '4.540')] | |
| [2023-09-05 10:59:26,017][273146] Updated weights for policy 0, policy_version 120 (0.0023) | |
| [2023-09-05 10:59:29,065][273146] Updated weights for policy 0, policy_version 130 (0.0021) | |
| [2023-09-05 10:59:29,335][272918] Fps is (10 sec: 12697.5, 60 sec: 11610.7, 300 sec: 11610.7). Total num frames: 532480. Throughput: 0: 2983.6. Samples: 134262. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-09-05 10:59:29,336][272918] Avg episode reward: [(0, '4.593')] | |
| [2023-09-05 10:59:32,211][273146] Updated weights for policy 0, policy_version 140 (0.0023) | |
| [2023-09-05 10:59:34,335][272918] Fps is (10 sec: 13107.4, 60 sec: 11757.9, 300 sec: 11757.9). Total num frames: 598016. Throughput: 0: 3181.7. Samples: 144196. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 10:59:34,335][272918] Avg episode reward: [(0, '4.554')] | |
| [2023-09-05 10:59:35,265][273146] Updated weights for policy 0, policy_version 150 (0.0025) | |
| [2023-09-05 10:59:38,077][273146] Updated weights for policy 0, policy_version 160 (0.0022) | |
| [2023-09-05 10:59:39,335][272918] Fps is (10 sec: 13516.9, 60 sec: 11952.0, 300 sec: 11952.0). Total num frames: 667648. Throughput: 0: 3317.8. Samples: 164952. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-09-05 10:59:39,335][272918] Avg episode reward: [(0, '4.551')] | |
| [2023-09-05 10:59:41,172][273146] Updated weights for policy 0, policy_version 170 (0.0023) | |
| [2023-09-05 10:59:44,159][273146] Updated weights for policy 0, policy_version 180 (0.0022) | |
| [2023-09-05 10:59:44,334][272918] Fps is (10 sec: 13926.5, 60 sec: 12288.1, 300 sec: 12114.2). Total num frames: 737280. Throughput: 0: 3316.6. Samples: 185138. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-09-05 10:59:44,335][272918] Avg episode reward: [(0, '5.000')] | |
| [2023-09-05 10:59:44,335][273075] Saving new best policy, reward=5.000! | |
| [2023-09-05 10:59:47,218][273146] Updated weights for policy 0, policy_version 190 (0.0022) | |
| [2023-09-05 10:59:49,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13312.0, 300 sec: 12189.6). Total num frames: 802816. Throughput: 0: 3323.1. Samples: 195294. Policy #0 lag: (min: 0.0, avg: 0.4, max: 1.0) | |
| [2023-09-05 10:59:49,335][272918] Avg episode reward: [(0, '5.285')] | |
| [2023-09-05 10:59:49,365][273075] Saving new best policy, reward=5.285! | |
| [2023-09-05 10:59:50,372][273146] Updated weights for policy 0, policy_version 200 (0.0024) | |
| [2023-09-05 10:59:53,644][273146] Updated weights for policy 0, policy_version 210 (0.0020) | |
| [2023-09-05 10:59:54,335][272918] Fps is (10 sec: 13107.1, 60 sec: 13312.0, 300 sec: 12254.3). Total num frames: 868352. Throughput: 0: 3302.3. Samples: 214700. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 10:59:54,335][272918] Avg episode reward: [(0, '4.984')] | |
| [2023-09-05 10:59:56,640][273146] Updated weights for policy 0, policy_version 220 (0.0021) | |
| [2023-09-05 10:59:59,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13243.7, 300 sec: 12310.5). Total num frames: 933888. Throughput: 0: 3306.0. Samples: 234894. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 10:59:59,335][272918] Avg episode reward: [(0, '5.508')] | |
| [2023-09-05 10:59:59,343][273075] Saving new best policy, reward=5.508! | |
| [2023-09-05 10:59:59,720][273146] Updated weights for policy 0, policy_version 230 (0.0024) | |
| [2023-09-05 11:00:02,687][273146] Updated weights for policy 0, policy_version 240 (0.0023) | |
| [2023-09-05 11:00:04,334][272918] Fps is (10 sec: 13107.3, 60 sec: 13243.7, 300 sec: 12359.8). Total num frames: 999424. Throughput: 0: 3307.4. Samples: 244988. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-09-05 11:00:04,335][272918] Avg episode reward: [(0, '5.247')] | |
| [2023-09-05 11:00:05,979][273146] Updated weights for policy 0, policy_version 250 (0.0019) | |
| [2023-09-05 11:00:08,952][273146] Updated weights for policy 0, policy_version 260 (0.0020) | |
| [2023-09-05 11:00:09,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13243.7, 300 sec: 12451.0). Total num frames: 1069056. Throughput: 0: 3330.0. Samples: 264696. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-09-05 11:00:09,335][272918] Avg episode reward: [(0, '5.880')] | |
| [2023-09-05 11:00:09,341][273075] Saving new best policy, reward=5.880! | |
| [2023-09-05 11:00:12,025][273146] Updated weights for policy 0, policy_version 270 (0.0028) | |
| [2023-09-05 11:00:14,335][272918] Fps is (10 sec: 13516.7, 60 sec: 13243.7, 300 sec: 12487.1). Total num frames: 1134592. Throughput: 0: 3347.6. Samples: 284902. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:00:14,335][272918] Avg episode reward: [(0, '6.638')] | |
| [2023-09-05 11:00:14,336][273075] Saving new best policy, reward=6.638! | |
| [2023-09-05 11:00:15,077][273146] Updated weights for policy 0, policy_version 280 (0.0022) | |
| [2023-09-05 11:00:18,087][273146] Updated weights for policy 0, policy_version 290 (0.0019) | |
| [2023-09-05 11:00:19,335][272918] Fps is (10 sec: 13106.8, 60 sec: 13243.7, 300 sec: 12519.4). Total num frames: 1200128. Throughput: 0: 3351.6. Samples: 295020. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2023-09-05 11:00:19,335][272918] Avg episode reward: [(0, '6.295')] | |
| [2023-09-05 11:00:21,249][273146] Updated weights for policy 0, policy_version 300 (0.0024) | |
| [2023-09-05 11:00:24,280][273146] Updated weights for policy 0, policy_version 310 (0.0026) | |
| [2023-09-05 11:00:24,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13380.3, 300 sec: 12589.2). Total num frames: 1269760. Throughput: 0: 3333.8. Samples: 314972. Policy #0 lag: (min: 0.0, avg: 0.7, max: 1.0) | |
| [2023-09-05 11:00:24,335][272918] Avg episode reward: [(0, '6.793')] | |
| [2023-09-05 11:00:24,336][273075] Saving new best policy, reward=6.793! | |
| [2023-09-05 11:00:27,263][273146] Updated weights for policy 0, policy_version 320 (0.0021) | |
| [2023-09-05 11:00:29,335][272918] Fps is (10 sec: 13517.1, 60 sec: 13380.3, 300 sec: 12613.7). Total num frames: 1335296. Throughput: 0: 3336.7. Samples: 335288. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:00:29,335][272918] Avg episode reward: [(0, '7.181')] | |
| [2023-09-05 11:00:29,354][273075] Saving /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/checkpoint_p0/checkpoint_000000327_1339392.pth... | |
| [2023-09-05 11:00:29,425][273075] Saving new best policy, reward=7.181! | |
| [2023-09-05 11:00:30,346][273146] Updated weights for policy 0, policy_version 330 (0.0024) | |
| [2023-09-05 11:00:33,428][273146] Updated weights for policy 0, policy_version 340 (0.0021) | |
| [2023-09-05 11:00:34,334][272918] Fps is (10 sec: 13107.3, 60 sec: 13380.3, 300 sec: 12636.0). Total num frames: 1400832. Throughput: 0: 3333.3. Samples: 345290. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:00:34,335][272918] Avg episode reward: [(0, '8.376')] | |
| [2023-09-05 11:00:34,355][273075] Saving new best policy, reward=8.376! | |
| [2023-09-05 11:00:36,545][273146] Updated weights for policy 0, policy_version 350 (0.0026) | |
| [2023-09-05 11:00:39,335][272918] Fps is (10 sec: 13516.7, 60 sec: 13380.2, 300 sec: 12691.6). Total num frames: 1470464. Throughput: 0: 3344.1. Samples: 365184. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:00:39,335][272918] Avg episode reward: [(0, '8.591')] | |
| [2023-09-05 11:00:39,342][273075] Saving new best policy, reward=8.591! | |
| [2023-09-05 11:00:39,554][273146] Updated weights for policy 0, policy_version 360 (0.0023) | |
| [2023-09-05 11:00:42,541][273146] Updated weights for policy 0, policy_version 370 (0.0022) | |
| [2023-09-05 11:00:44,335][272918] Fps is (10 sec: 13516.6, 60 sec: 13312.0, 300 sec: 12708.8). Total num frames: 1536000. Throughput: 0: 3345.4. Samples: 385438. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:00:44,335][272918] Avg episode reward: [(0, '9.990')] | |
| [2023-09-05 11:00:44,336][273075] Saving new best policy, reward=9.990! | |
| [2023-09-05 11:00:45,608][273146] Updated weights for policy 0, policy_version 380 (0.0025) | |
| [2023-09-05 11:00:48,766][273146] Updated weights for policy 0, policy_version 390 (0.0020) | |
| [2023-09-05 11:00:49,335][272918] Fps is (10 sec: 13107.3, 60 sec: 13312.0, 300 sec: 12724.6). Total num frames: 1601536. Throughput: 0: 3341.8. Samples: 395368. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:00:49,335][272918] Avg episode reward: [(0, '11.946')] | |
| [2023-09-05 11:00:49,340][273075] Saving new best policy, reward=11.946! | |
| [2023-09-05 11:00:51,861][273146] Updated weights for policy 0, policy_version 400 (0.0020) | |
| [2023-09-05 11:00:54,335][272918] Fps is (10 sec: 13516.7, 60 sec: 13380.2, 300 sec: 12770.6). Total num frames: 1671168. Throughput: 0: 3345.3. Samples: 415234. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:00:54,335][272918] Avg episode reward: [(0, '13.390')] | |
| [2023-09-05 11:00:54,336][273075] Saving new best policy, reward=13.390! | |
| [2023-09-05 11:00:54,888][273146] Updated weights for policy 0, policy_version 410 (0.0020) | |
| [2023-09-05 11:00:57,900][273146] Updated weights for policy 0, policy_version 420 (0.0021) | |
| [2023-09-05 11:00:59,335][272918] Fps is (10 sec: 13516.7, 60 sec: 13380.3, 300 sec: 12783.0). Total num frames: 1736704. Throughput: 0: 3346.1. Samples: 435478. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:00:59,335][272918] Avg episode reward: [(0, '14.793')] | |
| [2023-09-05 11:00:59,341][273075] Saving new best policy, reward=14.793! | |
| [2023-09-05 11:01:00,923][273146] Updated weights for policy 0, policy_version 430 (0.0022) | |
| [2023-09-05 11:01:04,200][273146] Updated weights for policy 0, policy_version 440 (0.0022) | |
| [2023-09-05 11:01:04,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13380.2, 300 sec: 12794.5). Total num frames: 1802240. Throughput: 0: 3341.1. Samples: 445370. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:04,335][272918] Avg episode reward: [(0, '13.747')] | |
| [2023-09-05 11:01:07,432][273146] Updated weights for policy 0, policy_version 450 (0.0024) | |
| [2023-09-05 11:01:09,335][272918] Fps is (10 sec: 13107.3, 60 sec: 13312.0, 300 sec: 12805.2). Total num frames: 1867776. Throughput: 0: 3317.2. Samples: 464246. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:09,335][272918] Avg episode reward: [(0, '15.855')] | |
| [2023-09-05 11:01:09,341][273075] Saving new best policy, reward=15.855! | |
| [2023-09-05 11:01:10,541][273146] Updated weights for policy 0, policy_version 460 (0.0021) | |
| [2023-09-05 11:01:13,527][273146] Updated weights for policy 0, policy_version 470 (0.0021) | |
| [2023-09-05 11:01:14,335][272918] Fps is (10 sec: 13107.4, 60 sec: 13312.0, 300 sec: 12815.2). Total num frames: 1933312. Throughput: 0: 3315.0. Samples: 484462. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-09-05 11:01:14,335][272918] Avg episode reward: [(0, '18.685')] | |
| [2023-09-05 11:01:14,336][273075] Saving new best policy, reward=18.685! | |
| [2023-09-05 11:01:16,623][273146] Updated weights for policy 0, policy_version 480 (0.0021) | |
| [2023-09-05 11:01:19,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13312.1, 300 sec: 12824.6). Total num frames: 1998848. Throughput: 0: 3315.1. Samples: 494468. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:19,335][272918] Avg episode reward: [(0, '20.522')] | |
| [2023-09-05 11:01:19,343][273075] Saving new best policy, reward=20.522! | |
| [2023-09-05 11:01:19,842][273146] Updated weights for policy 0, policy_version 490 (0.0022) | |
| [2023-09-05 11:01:22,834][273146] Updated weights for policy 0, policy_version 500 (0.0021) | |
| [2023-09-05 11:01:24,334][272918] Fps is (10 sec: 13516.9, 60 sec: 13312.0, 300 sec: 12858.8). Total num frames: 2068480. Throughput: 0: 3311.8. Samples: 514212. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:24,335][272918] Avg episode reward: [(0, '21.052')] | |
| [2023-09-05 11:01:24,336][273075] Saving new best policy, reward=21.052! | |
| [2023-09-05 11:01:25,844][273146] Updated weights for policy 0, policy_version 510 (0.0025) | |
| [2023-09-05 11:01:28,874][273146] Updated weights for policy 0, policy_version 520 (0.0025) | |
| [2023-09-05 11:01:29,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13312.0, 300 sec: 12866.3). Total num frames: 2134016. Throughput: 0: 3311.5. Samples: 534454. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:29,335][272918] Avg episode reward: [(0, '18.649')] | |
| [2023-09-05 11:01:31,946][273146] Updated weights for policy 0, policy_version 530 (0.0020) | |
| [2023-09-05 11:01:34,335][272918] Fps is (10 sec: 13107.1, 60 sec: 13312.0, 300 sec: 12873.4). Total num frames: 2199552. Throughput: 0: 3311.8. Samples: 544398. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-09-05 11:01:34,335][272918] Avg episode reward: [(0, '18.016')] | |
| [2023-09-05 11:01:35,077][273146] Updated weights for policy 0, policy_version 540 (0.0026) | |
| [2023-09-05 11:01:38,046][273146] Updated weights for policy 0, policy_version 550 (0.0020) | |
| [2023-09-05 11:01:39,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13312.0, 300 sec: 12903.3). Total num frames: 2269184. Throughput: 0: 3318.1. Samples: 564548. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-09-05 11:01:39,335][272918] Avg episode reward: [(0, '17.439')] | |
| [2023-09-05 11:01:41,126][273146] Updated weights for policy 0, policy_version 560 (0.0026) | |
| [2023-09-05 11:01:44,116][273146] Updated weights for policy 0, policy_version 570 (0.0022) | |
| [2023-09-05 11:01:44,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13312.0, 300 sec: 12908.9). Total num frames: 2334720. Throughput: 0: 3318.9. Samples: 584830. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:01:44,335][272918] Avg episode reward: [(0, '20.176')] | |
| [2023-09-05 11:01:47,095][273146] Updated weights for policy 0, policy_version 580 (0.0022) | |
| [2023-09-05 11:01:49,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13380.3, 300 sec: 12936.3). Total num frames: 2404352. Throughput: 0: 3330.1. Samples: 595226. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:49,335][272918] Avg episode reward: [(0, '22.372')] | |
| [2023-09-05 11:01:49,343][273075] Saving new best policy, reward=22.372! | |
| [2023-09-05 11:01:50,071][273146] Updated weights for policy 0, policy_version 590 (0.0024) | |
| [2023-09-05 11:01:53,092][273146] Updated weights for policy 0, policy_version 600 (0.0024) | |
| [2023-09-05 11:01:54,335][272918] Fps is (10 sec: 13926.5, 60 sec: 13380.3, 300 sec: 12962.2). Total num frames: 2473984. Throughput: 0: 3366.9. Samples: 615756. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-09-05 11:01:54,335][272918] Avg episode reward: [(0, '21.832')] | |
| [2023-09-05 11:01:56,097][273146] Updated weights for policy 0, policy_version 610 (0.0019) | |
| [2023-09-05 11:01:59,149][273146] Updated weights for policy 0, policy_version 620 (0.0022) | |
| [2023-09-05 11:01:59,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13380.3, 300 sec: 12965.9). Total num frames: 2539520. Throughput: 0: 3369.6. Samples: 636096. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:01:59,335][272918] Avg episode reward: [(0, '18.456')] | |
| [2023-09-05 11:02:02,099][273146] Updated weights for policy 0, policy_version 630 (0.0027) | |
| [2023-09-05 11:02:04,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13380.3, 300 sec: 12969.5). Total num frames: 2605056. Throughput: 0: 3374.4. Samples: 646314. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-09-05 11:02:04,335][272918] Avg episode reward: [(0, '18.894')] | |
| [2023-09-05 11:02:05,393][273146] Updated weights for policy 0, policy_version 640 (0.0025) | |
| [2023-09-05 11:02:08,452][273146] Updated weights for policy 0, policy_version 650 (0.0021) | |
| [2023-09-05 11:02:09,334][272918] Fps is (10 sec: 13107.5, 60 sec: 13380.3, 300 sec: 12972.8). Total num frames: 2670592. Throughput: 0: 3369.4. Samples: 665836. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:02:09,335][272918] Avg episode reward: [(0, '18.514')] | |
| [2023-09-05 11:02:11,476][273146] Updated weights for policy 0, policy_version 660 (0.0023) | |
| [2023-09-05 11:02:14,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13448.5, 300 sec: 12995.4). Total num frames: 2740224. Throughput: 0: 3369.8. Samples: 686096. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:02:14,335][272918] Avg episode reward: [(0, '20.559')] | |
| [2023-09-05 11:02:14,497][273146] Updated weights for policy 0, policy_version 670 (0.0022) | |
| [2023-09-05 11:02:17,491][273146] Updated weights for policy 0, policy_version 680 (0.0023) | |
| [2023-09-05 11:02:19,335][272918] Fps is (10 sec: 13926.1, 60 sec: 13516.8, 300 sec: 13017.0). Total num frames: 2809856. Throughput: 0: 3377.9. Samples: 696404. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:02:19,335][272918] Avg episode reward: [(0, '22.185')] | |
| [2023-09-05 11:02:20,481][273146] Updated weights for policy 0, policy_version 690 (0.0021) | |
| [2023-09-05 11:02:23,472][273146] Updated weights for policy 0, policy_version 700 (0.0022) | |
| [2023-09-05 11:02:24,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13448.5, 300 sec: 13019.0). Total num frames: 2875392. Throughput: 0: 3382.6. Samples: 716764. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:02:24,336][272918] Avg episode reward: [(0, '23.435')] | |
| [2023-09-05 11:02:24,337][273075] Saving new best policy, reward=23.435! | |
| [2023-09-05 11:02:26,545][273146] Updated weights for policy 0, policy_version 710 (0.0024) | |
| [2023-09-05 11:02:29,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13516.8, 300 sec: 13039.1). Total num frames: 2945024. Throughput: 0: 3381.4. Samples: 736992. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-09-05 11:02:29,336][272918] Avg episode reward: [(0, '24.806')] | |
| [2023-09-05 11:02:29,344][273075] Saving /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/checkpoint_p0/checkpoint_000000719_2945024.pth... | |
| [2023-09-05 11:02:29,414][273075] Saving new best policy, reward=24.806! | |
| [2023-09-05 11:02:29,608][273146] Updated weights for policy 0, policy_version 720 (0.0020) | |
| [2023-09-05 11:02:32,750][273146] Updated weights for policy 0, policy_version 730 (0.0020) | |
| [2023-09-05 11:02:34,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13516.8, 300 sec: 13040.6). Total num frames: 3010560. Throughput: 0: 3364.9. Samples: 746644. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:02:34,335][272918] Avg episode reward: [(0, '23.367')] | |
| [2023-09-05 11:02:35,757][273146] Updated weights for policy 0, policy_version 740 (0.0026) | |
| [2023-09-05 11:02:38,889][273146] Updated weights for policy 0, policy_version 750 (0.0024) | |
| [2023-09-05 11:02:39,335][272918] Fps is (10 sec: 13107.3, 60 sec: 13448.5, 300 sec: 13042.0). Total num frames: 3076096. Throughput: 0: 3356.6. Samples: 766804. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:02:39,335][272918] Avg episode reward: [(0, '23.194')] | |
| [2023-09-05 11:02:41,880][273146] Updated weights for policy 0, policy_version 760 (0.0022) | |
| [2023-09-05 11:02:44,334][272918] Fps is (10 sec: 13517.0, 60 sec: 13516.8, 300 sec: 13060.4). Total num frames: 3145728. Throughput: 0: 3356.3. Samples: 787128. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:02:44,335][272918] Avg episode reward: [(0, '24.675')] | |
| [2023-09-05 11:02:44,885][273146] Updated weights for policy 0, policy_version 770 (0.0024) | |
| [2023-09-05 11:02:47,776][273146] Updated weights for policy 0, policy_version 780 (0.0018) | |
| [2023-09-05 11:02:49,335][272918] Fps is (10 sec: 13926.4, 60 sec: 13516.8, 300 sec: 13078.0). Total num frames: 3215360. Throughput: 0: 3365.2. Samples: 797750. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:02:49,335][272918] Avg episode reward: [(0, '21.628')] | |
| [2023-09-05 11:02:50,793][273146] Updated weights for policy 0, policy_version 790 (0.0024) | |
| [2023-09-05 11:02:53,815][273146] Updated weights for policy 0, policy_version 800 (0.0021) | |
| [2023-09-05 11:02:54,335][272918] Fps is (10 sec: 13516.6, 60 sec: 13448.5, 300 sec: 13078.6). Total num frames: 3280896. Throughput: 0: 3384.3. Samples: 818128. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:02:54,335][272918] Avg episode reward: [(0, '23.257')] | |
| [2023-09-05 11:02:56,820][273146] Updated weights for policy 0, policy_version 810 (0.0028) | |
| [2023-09-05 11:02:59,335][272918] Fps is (10 sec: 13516.9, 60 sec: 13516.8, 300 sec: 13095.1). Total num frames: 3350528. Throughput: 0: 3381.9. Samples: 838280. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:02:59,335][272918] Avg episode reward: [(0, '25.138')] | |
| [2023-09-05 11:02:59,344][273075] Saving new best policy, reward=25.138! | |
| [2023-09-05 11:02:59,945][273146] Updated weights for policy 0, policy_version 820 (0.0021) | |
| [2023-09-05 11:03:03,134][273146] Updated weights for policy 0, policy_version 830 (0.0021) | |
| [2023-09-05 11:03:04,335][272918] Fps is (10 sec: 13107.1, 60 sec: 13448.5, 300 sec: 13079.6). Total num frames: 3411968. Throughput: 0: 3361.8. Samples: 847684. Policy #0 lag: (min: 0.0, avg: 0.5, max: 1.0) | |
| [2023-09-05 11:03:04,335][272918] Avg episode reward: [(0, '25.295')] | |
| [2023-09-05 11:03:04,336][273075] Saving new best policy, reward=25.295! | |
| [2023-09-05 11:03:06,301][273146] Updated weights for policy 0, policy_version 840 (0.0022) | |
| [2023-09-05 11:03:09,286][273146] Updated weights for policy 0, policy_version 850 (0.0023) | |
| [2023-09-05 11:03:09,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13516.8, 300 sec: 13095.6). Total num frames: 3481600. Throughput: 0: 3353.8. Samples: 867686. Policy #0 lag: (min: 0.0, avg: 0.6, max: 1.0) | |
| [2023-09-05 11:03:09,335][272918] Avg episode reward: [(0, '23.496')] | |
| [2023-09-05 11:03:12,386][273146] Updated weights for policy 0, policy_version 860 (0.0022) | |
| [2023-09-05 11:03:14,335][272918] Fps is (10 sec: 13107.3, 60 sec: 13380.3, 300 sec: 13080.7). Total num frames: 3543040. Throughput: 0: 3333.9. Samples: 887018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:03:14,335][272918] Avg episode reward: [(0, '23.233')] | |
| [2023-09-05 11:03:15,630][273146] Updated weights for policy 0, policy_version 870 (0.0024) | |
| [2023-09-05 11:03:18,835][273146] Updated weights for policy 0, policy_version 880 (0.0023) | |
| [2023-09-05 11:03:19,335][272918] Fps is (10 sec: 12697.5, 60 sec: 13312.0, 300 sec: 13081.1). Total num frames: 3608576. Throughput: 0: 3341.8. Samples: 897026. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:03:19,335][272918] Avg episode reward: [(0, '24.170')] | |
| [2023-09-05 11:03:21,975][273146] Updated weights for policy 0, policy_version 890 (0.0023) | |
| [2023-09-05 11:03:24,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13312.0, 300 sec: 13081.6). Total num frames: 3674112. Throughput: 0: 3324.8. Samples: 916418. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:03:24,335][272918] Avg episode reward: [(0, '24.026')] | |
| [2023-09-05 11:03:25,044][273146] Updated weights for policy 0, policy_version 900 (0.0021) | |
| [2023-09-05 11:03:28,101][273146] Updated weights for policy 0, policy_version 910 (0.0021) | |
| [2023-09-05 11:03:29,334][272918] Fps is (10 sec: 13517.0, 60 sec: 13312.0, 300 sec: 13096.4). Total num frames: 3743744. Throughput: 0: 3319.3. Samples: 936496. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:03:29,335][272918] Avg episode reward: [(0, '23.463')] | |
| [2023-09-05 11:03:31,228][273146] Updated weights for policy 0, policy_version 920 (0.0024) | |
| [2023-09-05 11:03:34,274][273146] Updated weights for policy 0, policy_version 930 (0.0026) | |
| [2023-09-05 11:03:34,335][272918] Fps is (10 sec: 13516.8, 60 sec: 13312.0, 300 sec: 13096.6). Total num frames: 3809280. Throughput: 0: 3301.7. Samples: 946328. Policy #0 lag: (min: 0.0, avg: 0.5, max: 2.0) | |
| [2023-09-05 11:03:34,335][272918] Avg episode reward: [(0, '25.828')] | |
| [2023-09-05 11:03:34,336][273075] Saving new best policy, reward=25.828! | |
| [2023-09-05 11:03:37,349][273146] Updated weights for policy 0, policy_version 940 (0.0022) | |
| [2023-09-05 11:03:39,335][272918] Fps is (10 sec: 13107.0, 60 sec: 13312.0, 300 sec: 13135.0). Total num frames: 3874816. Throughput: 0: 3297.1. Samples: 966496. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2023-09-05 11:03:39,335][272918] Avg episode reward: [(0, '25.149')] | |
| [2023-09-05 11:03:40,415][273146] Updated weights for policy 0, policy_version 950 (0.0020) | |
| [2023-09-05 11:03:43,469][273146] Updated weights for policy 0, policy_version 960 (0.0023) | |
| [2023-09-05 11:03:44,335][272918] Fps is (10 sec: 13107.2, 60 sec: 13243.7, 300 sec: 13343.2). Total num frames: 3940352. Throughput: 0: 3294.0. Samples: 986512. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2023-09-05 11:03:44,335][272918] Avg episode reward: [(0, '25.829')] | |
| [2023-09-05 11:03:44,374][273075] Saving new best policy, reward=25.829! | |
| [2023-09-05 11:03:46,544][273146] Updated weights for policy 0, policy_version 970 (0.0024) | |
| [2023-09-05 11:03:48,986][273075] Stopping Batcher_0... | |
| [2023-09-05 11:03:48,987][273075] Loop batcher_evt_loop terminating... | |
| [2023-09-05 11:03:48,986][272918] Component Batcher_0 stopped! | |
| [2023-09-05 11:03:48,989][273075] Saving /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-09-05 11:03:48,998][272918] Component RolloutWorker_w7 stopped! | |
| [2023-09-05 11:03:49,001][272918] Component RolloutWorker_w5 stopped! | |
| [2023-09-05 11:03:48,999][273160] Stopping RolloutWorker_w7... | |
| [2023-09-05 11:03:49,000][273164] Stopping RolloutWorker_w5... | |
| [2023-09-05 11:03:49,002][272918] Component RolloutWorker_w2 stopped! | |
| [2023-09-05 11:03:49,002][273148] Stopping RolloutWorker_w2... | |
| [2023-09-05 11:03:49,003][273160] Loop rollout_proc7_evt_loop terminating... | |
| [2023-09-05 11:03:49,005][272918] Component RolloutWorker_w3 stopped! | |
| [2023-09-05 11:03:49,002][273164] Loop rollout_proc5_evt_loop terminating... | |
| [2023-09-05 11:03:49,003][273148] Loop rollout_proc2_evt_loop terminating... | |
| [2023-09-05 11:03:49,005][273162] Stopping RolloutWorker_w3... | |
| [2023-09-05 11:03:49,007][273162] Loop rollout_proc3_evt_loop terminating... | |
| [2023-09-05 11:03:49,007][273146] Weights refcount: 2 0 | |
| [2023-09-05 11:03:49,010][272918] Component RolloutWorker_w1 stopped! | |
| [2023-09-05 11:03:49,010][273149] Stopping RolloutWorker_w1... | |
| [2023-09-05 11:03:49,011][273149] Loop rollout_proc1_evt_loop terminating... | |
| [2023-09-05 11:03:49,017][272918] Component InferenceWorker_p0-w0 stopped! | |
| [2023-09-05 11:03:49,017][273146] Stopping InferenceWorker_p0-w0... | |
| [2023-09-05 11:03:49,020][272918] Component RolloutWorker_w0 stopped! | |
| [2023-09-05 11:03:49,020][273147] Stopping RolloutWorker_w0... | |
| [2023-09-05 11:03:49,022][273147] Loop rollout_proc0_evt_loop terminating... | |
| [2023-09-05 11:03:49,024][273146] Loop inference_proc0-0_evt_loop terminating... | |
| [2023-09-05 11:03:49,026][273157] Stopping RolloutWorker_w4... | |
| [2023-09-05 11:03:49,026][272918] Component RolloutWorker_w4 stopped! | |
| [2023-09-05 11:03:49,027][273157] Loop rollout_proc4_evt_loop terminating... | |
| [2023-09-05 11:03:49,029][272918] Component RolloutWorker_w6 stopped! | |
| [2023-09-05 11:03:49,029][273165] Stopping RolloutWorker_w6... | |
| [2023-09-05 11:03:49,030][273165] Loop rollout_proc6_evt_loop terminating... | |
| [2023-09-05 11:03:49,049][273075] Removing /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/checkpoint_p0/checkpoint_000000327_1339392.pth | |
| [2023-09-05 11:03:49,054][273075] Saving new best policy, reward=26.360! | |
| [2023-09-05 11:03:49,113][273075] Saving /media/ml1/data/nogletrading/ppo_vizdoom/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2023-09-05 11:03:49,384][273075] Stopping LearnerWorker_p0... | |
| [2023-09-05 11:03:49,384][272918] Component LearnerWorker_p0 stopped! | |
| [2023-09-05 11:03:49,385][272918] Waiting for process learner_proc0 to stop... | |
| [2023-09-05 11:03:49,385][273075] Loop learner_proc0_evt_loop terminating... | |
| [2023-09-05 11:03:50,689][272918] Waiting for process inference_proc0-0 to join... | |
| [2023-09-05 11:03:50,689][272918] Waiting for process rollout_proc0 to join... | |
| [2023-09-05 11:03:50,690][272918] Waiting for process rollout_proc1 to join... | |
| [2023-09-05 11:03:50,690][272918] Waiting for process rollout_proc2 to join... | |
| [2023-09-05 11:03:50,690][272918] Waiting for process rollout_proc3 to join... | |
| [2023-09-05 11:03:50,690][272918] Waiting for process rollout_proc4 to join... | |
| [2023-09-05 11:03:50,690][272918] Waiting for process rollout_proc5 to join... | |
| [2023-09-05 11:03:50,691][272918] Waiting for process rollout_proc6 to join... | |
| [2023-09-05 11:03:50,691][272918] Waiting for process rollout_proc7 to join... | |
| [2023-09-05 11:03:50,691][272918] Batcher 0 profile tree view: | |
| batching: 12.1773, releasing_batches: 0.0370 | |
| [2023-09-05 11:03:50,692][272918] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0002 | |
| wait_policy_total: 8.1322 | |
| update_model: 6.1108 | |
| weight_update: 0.0024 | |
| one_step: 0.0040 | |
| handle_policy_step: 259.8800 | |
| deserialize: 12.6613, stack: 2.4922, obs_to_device_normalize: 69.9636, forward: 100.9362, send_messages: 22.0939 | |
| prepare_outputs: 30.9768 | |
| to_cpu: 17.8297 | |
| [2023-09-05 11:03:50,693][272918] Learner 0 profile tree view: | |
| misc: 0.0108, prepare_batch: 8.5503 | |
| train: 30.1955 | |
| epoch_init: 0.0128, minibatch_init: 0.0083, losses_postprocess: 0.2211, kl_divergence: 0.2575, after_optimizer: 10.6754 | |
| calculate_losses: 10.3571 | |
| losses_init: 0.0088, forward_head: 0.7368, bptt_initial: 6.7256, tail: 0.5483, advantages_returns: 0.1471, losses: 0.8372 | |
| bptt: 1.0653 | |
| bptt_forward_core: 0.9959 | |
| update: 8.1721 | |
| clip: 1.5875 | |
| [2023-09-05 11:03:50,693][272918] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 0.3348, enqueue_policy_requests: 11.6962, env_step: 124.9929, overhead: 10.9930, complete_rollouts: 0.7431 | |
| save_policy_outputs: 23.2284 | |
| split_output_tensors: 9.0382 | |
| [2023-09-05 11:03:50,693][272918] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 0.3340, enqueue_policy_requests: 11.9348, env_step: 126.8656, overhead: 11.2031, complete_rollouts: 0.8436 | |
| save_policy_outputs: 22.9981 | |
| split_output_tensors: 8.8397 | |
| [2023-09-05 11:03:50,694][272918] Loop Runner_EvtLoop terminating... | |
| [2023-09-05 11:03:50,694][272918] Runner profile tree view: | |
| main_loop: 315.5323 | |
| [2023-09-05 11:03:50,695][272918] Collected {0: 4005888}, FPS: 12695.6 | |