[2023-07-08 20:44:45,202][17004] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 20:44:45,221][17004] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 20:44:45,272][17004] Num visible devices: 1
[2023-07-08 20:44:45,422][17004] Setting fixed seed 42
[2023-07-08 20:44:45,422][17004] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 20:44:45,422][17004] Initializing actor-critic model on device cuda:0
[2023-07-08 20:44:45,423][17004] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 20:44:45,423][17004] RunningMeanStd input shape: (1,)
[2023-07-08 20:44:45,429][17004] ConvEncoder: input_channels=3
[2023-07-08 20:44:45,842][17004] Conv encoder output size: 512
[2023-07-08 20:44:45,842][17004] Policy head output size: 512
[2023-07-08 20:44:45,888][17004] Created Actor Critic model with architecture:
[2023-07-08 20:44:45,888][17004] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 20:44:46,131][17028] Worker 2 uses CPU cores [2]
[2023-07-08 20:44:46,218][17025] Worker 0 uses CPU cores [0]
[2023-07-08 20:44:46,362][17024] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 20:44:46,362][17024] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 20:44:46,378][17040] Worker 15 uses CPU cores [3]
[2023-07-08 20:44:46,419][17026] Worker 4 uses CPU cores [0]
[2023-07-08 20:44:46,426][17024] Num visible devices: 1
[2023-07-08 20:44:46,461][17027] Worker 3 uses CPU cores [3]
[2023-07-08 20:44:46,470][17029] Worker 6 uses CPU cores [2]
[2023-07-08 20:44:46,484][17042] Worker 17 uses CPU cores [1]
[2023-07-08 20:44:46,539][17038] Worker 13 uses CPU cores [1]
[2023-07-08 20:44:46,540][17031] Worker 1 uses CPU cores [1]
[2023-07-08 20:44:46,598][17043] Worker 18 uses CPU cores [2]
[2023-07-08 20:44:46,600][17035] Worker 9 uses CPU cores [1]
[2023-07-08 20:44:46,625][17030] Worker 7 uses CPU cores [3]
[2023-07-08 20:44:46,633][17034] Worker 16 uses CPU cores [0]
[2023-07-08 20:44:46,679][17041] Worker 10 uses CPU cores [2]
[2023-07-08 20:44:46,686][17039] Worker 14 uses CPU cores [2]
[2023-07-08 20:44:46,691][17037] Worker 12 uses CPU cores [0]
[2023-07-08 20:44:46,701][17044] Worker 19 uses CPU cores [3]
[2023-07-08 20:44:46,722][17036] Worker 11 uses CPU cores [3]
[2023-07-08 20:44:46,782][17032] Worker 5 uses CPU cores [1]
[2023-07-08 20:44:46,870][17033] Worker 8 uses CPU cores [0]
[2023-07-08 20:44:48,363][17004] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 20:44:48,364][17004] No checkpoints found
[2023-07-08 20:44:48,364][17004] Did not load from checkpoint, starting from scratch!
[2023-07-08 20:44:48,364][17004] Initialized policy 0 weights for model version 0
[2023-07-08 20:44:48,367][17004] LearnerWorker_p0 finished initialization!
[2023-07-08 20:44:48,367][17004] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 20:44:48,523][17024] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 20:46:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:48:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:50:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:52:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:54:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:56:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:58:38,984][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:59:38,986][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:59:38,987][17044] Stopping RolloutWorker_w19...
[2023-07-08 20:59:38,987][17044] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 20:59:38,987][17025] Stopping RolloutWorker_w0...
[2023-07-08 20:59:38,987][17025] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 20:59:38,988][17026] Stopping RolloutWorker_w4...
[2023-07-08 20:59:38,986][17027] Stopping RolloutWorker_w3...
[2023-07-08 20:59:38,988][17026] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 20:59:38,988][17036] Stopping RolloutWorker_w11...
[2023-07-08 20:59:38,988][17027] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 20:59:38,988][17030] Stopping RolloutWorker_w7...
[2023-07-08 20:59:38,989][17030] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 20:59:38,988][17043] Stopping RolloutWorker_w18...
[2023-07-08 20:59:38,986][17040] Stopping RolloutWorker_w15...
[2023-07-08 20:59:38,988][17039] Stopping RolloutWorker_w14...
[2023-07-08 20:59:38,989][17036] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 20:59:38,988][17041] Stopping RolloutWorker_w10...
[2023-07-08 20:59:38,990][17040] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 20:59:38,989][17029] Stopping RolloutWorker_w6...
[2023-07-08 20:59:38,989][17028] Stopping RolloutWorker_w2...
[2023-07-08 20:59:38,989][17043] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 20:59:38,990][17041] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 20:59:38,992][17037] Stopping RolloutWorker_w12...
[2023-07-08 20:59:38,992][17035] Stopping RolloutWorker_w9...
[2023-07-08 20:59:38,990][17039] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 20:59:38,992][17037] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 20:59:38,992][17035] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 20:59:38,990][17029] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 20:59:38,990][17028] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 20:59:39,001][17034] Stopping RolloutWorker_w16...
[2023-07-08 20:59:39,001][17034] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 20:59:39,002][17031] Stopping RolloutWorker_w1...
[2023-07-08 20:59:39,002][17031] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 20:59:38,986][17033] Stopping RolloutWorker_w8...
[2023-07-08 20:59:39,012][17042] Stopping RolloutWorker_w17...
[2023-07-08 20:59:39,012][17033] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 20:59:39,012][17004] Stopping Batcher_0...
[2023-07-08 20:59:39,012][17042] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 20:59:39,012][17004] Loop batcher_evt_loop terminating...
[2023-07-08 20:59:39,022][17038] Stopping RolloutWorker_w13...
[2023-07-08 20:59:39,022][17038] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 20:59:39,032][17032] Stopping RolloutWorker_w5...
[2023-07-08 20:59:39,032][17032] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 20:59:39,043][17004] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 20:59:39,075][17004] Stopping LearnerWorker_p0...
[2023-07-08 20:59:39,075][17004] Loop learner_proc0_evt_loop terminating...
[2023-07-08 21:13:45,642][17306] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:13:45,642][17306] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 21:13:45,688][17306] Num visible devices: 1
[2023-07-08 21:13:45,807][17306] Setting fixed seed 42
[2023-07-08 21:13:45,808][17306] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:13:45,808][17306] Initializing actor-critic model on device cuda:0
[2023-07-08 21:13:45,808][17306] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 21:13:45,809][17306] RunningMeanStd input shape: (1,)
[2023-07-08 21:13:45,816][17306] ConvEncoder: input_channels=3
[2023-07-08 21:13:45,919][17326] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:13:45,920][17326] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 21:13:45,976][17326] Num visible devices: 1
[2023-07-08 21:13:46,140][17327] Worker 0 uses CPU cores [0]
[2023-07-08 21:13:46,130][17306] Conv encoder output size: 512
[2023-07-08 21:13:46,172][17306] Policy head output size: 512
[2023-07-08 21:13:46,223][17306] Created Actor Critic model with architecture:
[2023-07-08 21:13:46,242][17306] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 21:13:46,351][17329] Worker 2 uses CPU cores [2]
[2023-07-08 21:13:46,421][17330] Worker 3 uses CPU cores [3]
[2023-07-08 21:13:46,561][17331] Worker 4 uses CPU cores [0]
[2023-07-08 21:13:46,608][17334] Worker 5 uses CPU cores [1]
[2023-07-08 21:13:46,689][17338] Worker 11 uses CPU cores [3]
[2023-07-08 21:13:46,697][17336] Worker 9 uses CPU cores [1]
[2023-07-08 21:13:46,698][17339] Worker 12 uses CPU cores [0]
[2023-07-08 21:13:46,704][17328] Worker 1 uses CPU cores [1]
[2023-07-08 21:13:46,711][17345] Worker 18 uses CPU cores [2]
[2023-07-08 21:13:46,732][17346] Worker 19 uses CPU cores [3]
[2023-07-08 21:13:46,743][17341] Worker 15 uses CPU cores [3]
[2023-07-08 21:13:46,746][17340] Worker 13 uses CPU cores [1]
[2023-07-08 21:13:46,746][17344] Worker 17 uses CPU cores [1]
[2023-07-08 21:13:46,771][17337] Worker 10 uses CPU cores [2]
[2023-07-08 21:13:46,789][17333] Worker 7 uses CPU cores [3]
[2023-07-08 21:13:46,791][17332] Worker 6 uses CPU cores [2]
[2023-07-08 21:13:46,793][17343] Worker 16 uses CPU cores [0]
[2023-07-08 21:13:46,801][17335] Worker 8 uses CPU cores [0]
[2023-07-08 21:13:46,841][17342] Worker 14 uses CPU cores [2]
[2023-07-08 21:13:47,146][17306] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 21:13:47,147][17306] Loading state from checkpoint /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:13:47,155][17306] Loading model from checkpoint
[2023-07-08 21:13:47,156][17306] Loaded experiment state at self.train_step=0, self.env_steps=0
[2023-07-08 21:13:47,156][17306] Initialized policy 0 weights for model version 0
[2023-07-08 21:13:47,159][17306] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:13:47,161][17306] LearnerWorker_p0 finished initialization!
[2023-07-08 21:13:47,303][17326] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 21:15:39,049][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:17:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:19:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:21:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:23:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:25:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:27:39,050][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:28:39,052][17337] Stopping RolloutWorker_w10...
[2023-07-08 21:28:39,052][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:28:39,052][17337] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 21:28:39,052][17335] Stopping RolloutWorker_w8...
[2023-07-08 21:28:39,053][17339] Stopping RolloutWorker_w12...
[2023-07-08 21:28:39,054][17333] Stopping RolloutWorker_w7...
[2023-07-08 21:28:39,053][17343] Stopping RolloutWorker_w16...
[2023-07-08 21:28:39,055][17341] Stopping RolloutWorker_w15...
[2023-07-08 21:28:39,055][17343] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 21:28:39,054][17346] Stopping RolloutWorker_w19...
[2023-07-08 21:28:39,055][17338] Stopping RolloutWorker_w11...
[2023-07-08 21:28:39,055][17330] Stopping RolloutWorker_w3...
[2023-07-08 21:28:39,056][17341] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 21:28:39,053][17331] Stopping RolloutWorker_w4...
[2023-07-08 21:28:39,056][17333] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 21:28:39,057][17331] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 21:28:39,056][17338] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 21:28:39,061][17345] Stopping RolloutWorker_w18...
[2023-07-08 21:28:39,061][17306] Stopping Batcher_0...
[2023-07-08 21:28:39,061][17345] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 21:28:39,061][17306] Loop batcher_evt_loop terminating...
[2023-07-08 21:28:39,061][17335] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 21:28:39,057][17346] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 21:28:39,062][17340] Stopping RolloutWorker_w13...
[2023-07-08 21:28:39,057][17330] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 21:28:39,062][17340] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 21:28:39,071][17329] Stopping RolloutWorker_w2...
[2023-07-08 21:28:39,071][17329] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 21:28:39,052][17327] Stopping RolloutWorker_w0...
[2023-07-08 21:28:39,072][17344] Stopping RolloutWorker_w17...
[2023-07-08 21:28:39,072][17327] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 21:28:39,072][17344] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 21:28:39,081][17332] Stopping RolloutWorker_w6...
[2023-07-08 21:28:39,081][17332] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 21:28:39,071][17339] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 21:28:39,082][17328] Stopping RolloutWorker_w1...
[2023-07-08 21:28:39,082][17328] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 21:28:39,091][17342] Stopping RolloutWorker_w14...
[2023-07-08 21:28:39,091][17342] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 21:28:39,092][17334] Stopping RolloutWorker_w5...
[2023-07-08 21:28:39,092][17334] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 21:28:39,094][17306] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:28:39,102][17336] Stopping RolloutWorker_w9...
[2023-07-08 21:28:39,102][17336] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 21:28:39,125][17306] Stopping LearnerWorker_p0...
[2023-07-08 21:28:39,125][17306] Loop learner_proc0_evt_loop terminating...
[2023-07-08 21:39:33,872][17857] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:39:33,882][17857] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 21:39:33,928][17857] Num visible devices: 1
[2023-07-08 21:39:34,048][17857] Setting fixed seed 42
[2023-07-08 21:39:34,049][17857] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:39:34,049][17857] Initializing actor-critic model on device cuda:0
[2023-07-08 21:39:34,049][17857] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 21:39:34,049][17857] RunningMeanStd input shape: (1,)
[2023-07-08 21:39:34,056][17857] ConvEncoder: input_channels=3
[2023-07-08 21:39:34,442][17857] Conv encoder output size: 512
[2023-07-08 21:39:34,443][17857] Policy head output size: 512
[2023-07-08 21:39:34,452][17857] Created Actor Critic model with architecture:
[2023-07-08 21:39:34,492][17857] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 21:39:34,999][17857] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 21:39:35,000][17857] Loading state from checkpoint /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:39:35,072][17857] Loading model from checkpoint
[2023-07-08 21:39:35,073][17857] Loaded experiment state at self.train_step=0, self.env_steps=0
[2023-07-08 21:39:35,074][17857] Initialized policy 0 weights for model version 0
[2023-07-08 21:39:35,084][17857] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:39:35,092][17857] LearnerWorker_p0 finished initialization!
[2023-07-08 21:39:35,100][17884] Worker 6 uses CPU cores [2]
[2023-07-08 21:39:35,221][17877] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 21:39:35,222][17877] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 21:39:35,223][17878] Worker 0 uses CPU cores [0]
[2023-07-08 21:39:35,231][17879] Worker 1 uses CPU cores [1]
[2023-07-08 21:39:35,286][17877] Num visible devices: 1
[2023-07-08 21:39:35,371][17880] Worker 2 uses CPU cores [2]
[2023-07-08 21:39:35,398][17886] Worker 8 uses CPU cores [0]
[2023-07-08 21:39:35,408][17888] Worker 10 uses CPU cores [2]
[2023-07-08 21:39:35,492][17885] Worker 7 uses CPU cores [3]
[2023-07-08 21:39:35,492][17891] Worker 12 uses CPU cores [0]
[2023-07-08 21:39:35,511][17887] Worker 9 uses CPU cores [1]
[2023-07-08 21:39:35,532][17881] Worker 4 uses CPU cores [0]
[2023-07-08 21:39:35,541][17882] Worker 3 uses CPU cores [3]
[2023-07-08 21:39:35,576][17893] Worker 15 uses CPU cores [3]
[2023-07-08 21:39:35,601][17890] Worker 13 uses CPU cores [1]
[2023-07-08 21:39:35,613][17892] Worker 14 uses CPU cores [2]
[2023-07-08 21:39:35,621][17894] Worker 16 uses CPU cores [0]
[2023-07-08 21:39:35,624][17895] Worker 17 uses CPU cores [1]
[2023-07-08 21:39:35,672][17896] Worker 19 uses CPU cores [3]
[2023-07-08 21:39:35,679][17883] Worker 5 uses CPU cores [1]
[2023-07-08 21:39:35,686][17897] Worker 18 uses CPU cores [2]
[2023-07-08 21:39:35,725][17889] Worker 11 uses CPU cores [3]
[2023-07-08 21:39:35,840][17877] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 21:41:27,794][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:43:27,795][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:45:27,793][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:47:27,793][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:49:27,793][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:51:27,793][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:53:27,795][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:54:27,795][17894] Stopping RolloutWorker_w16...
[2023-07-08 21:54:27,795][17890] Stopping RolloutWorker_w13...
[2023-07-08 21:54:27,795][17894] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 21:54:27,795][17890] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 21:54:27,796][17888] Stopping RolloutWorker_w10...
[2023-07-08 21:54:27,796][17888] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 21:54:27,802][17879] Stopping RolloutWorker_w1...
[2023-07-08 21:54:27,802][17892] Stopping RolloutWorker_w14...
[2023-07-08 21:54:27,802][17891] Stopping RolloutWorker_w12...
[2023-07-08 21:54:27,802][17896] Stopping RolloutWorker_w19...
[2023-07-08 21:54:27,802][17879] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 21:54:27,802][17892] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 21:54:27,802][17891] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 21:54:27,802][17896] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 21:54:27,808][17897] Stopping RolloutWorker_w18...
[2023-07-08 21:54:27,808][17897] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 21:54:27,812][17885] Stopping RolloutWorker_w7...
[2023-07-08 21:54:27,812][17881] Stopping RolloutWorker_w4...
[2023-07-08 21:54:27,812][17880] Stopping RolloutWorker_w2...
[2023-07-08 21:54:27,812][17885] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 21:54:27,812][17895] Stopping RolloutWorker_w17...
[2023-07-08 21:54:27,812][17881] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 21:54:27,812][17880] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 21:54:27,812][17895] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 21:54:27,818][17884] Stopping RolloutWorker_w6...
[2023-07-08 21:54:27,819][17884] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 21:54:27,822][17893] Stopping RolloutWorker_w15...
[2023-07-08 21:54:27,822][17883] Stopping RolloutWorker_w5...
[2023-07-08 21:54:27,822][17878] Stopping RolloutWorker_w0...
[2023-07-08 21:54:27,822][17893] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 21:54:27,822][17883] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 21:54:27,822][17878] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 21:54:27,832][17882] Stopping RolloutWorker_w3...
[2023-07-08 21:54:27,832][17882] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 21:54:27,832][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:54:27,833][17886] Stopping RolloutWorker_w8...
[2023-07-08 21:54:27,833][17886] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 21:54:27,833][17887] Stopping RolloutWorker_w9...
[2023-07-08 21:54:27,833][17887] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 21:54:27,839][17857] Stopping Batcher_0...
[2023-07-08 21:54:27,839][17857] Loop batcher_evt_loop terminating...
[2023-07-08 21:54:27,842][17889] Stopping RolloutWorker_w11...
[2023-07-08 21:54:27,842][17889] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 21:54:27,855][17857] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 21:54:27,905][17857] Stopping LearnerWorker_p0...
[2023-07-08 21:54:27,905][17857] Loop learner_proc0_evt_loop terminating...
[2023-07-08 22:10:38,828][18235] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:10:38,832][18235] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 22:10:38,894][18235] Num visible devices: 1
[2023-07-08 22:10:39,062][18235] Setting fixed seed 42
[2023-07-08 22:10:39,063][18235] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:10:39,063][18235] Initializing actor-critic model on device cuda:0
[2023-07-08 22:10:39,063][18235] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 22:10:39,063][18235] RunningMeanStd input shape: (1,)
[2023-07-08 22:10:39,070][18235] ConvEncoder: input_channels=3
[2023-07-08 22:10:39,386][18235] Conv encoder output size: 512
[2023-07-08 22:10:39,428][18235] Policy head output size: 512
[2023-07-08 22:10:39,448][18235] Created Actor Critic model with architecture:
[2023-07-08 22:10:39,481][18235] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 22:10:40,063][18235] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 22:10:40,063][18235] Loading state from checkpoint /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:10:40,149][18255] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:10:40,150][18255] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 22:10:40,226][18255] Num visible devices: 1
[2023-07-08 22:10:40,229][18235] Loading model from checkpoint
[2023-07-08 22:10:40,231][18235] Loaded experiment state at self.train_step=539850, self.env_steps=4422451200
[2023-07-08 22:10:40,232][18235] Initialized policy 0 weights for model version 539850
[2023-07-08 22:10:40,252][18235] LearnerWorker_p0 finished initialization!
[2023-07-08 22:10:40,253][18235] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:10:40,461][18260] Worker 3 uses CPU cores [3]
[2023-07-08 22:10:40,672][18256] Worker 0 uses CPU cores [0]
[2023-07-08 22:10:40,741][18261] Worker 5 uses CPU cores [1]
[2023-07-08 22:10:40,772][18257] Worker 1 uses CPU cores [1]
[2023-07-08 22:10:40,774][18258] Worker 2 uses CPU cores [2]
[2023-07-08 22:10:40,917][18263] Worker 7 uses CPU cores [3]
[2023-07-08 22:10:40,931][18264] Worker 8 uses CPU cores [0]
[2023-07-08 22:10:40,941][18269] Worker 13 uses CPU cores [1]
[2023-07-08 22:10:40,977][18259] Worker 4 uses CPU cores [0]
[2023-07-08 22:10:41,112][18274] Worker 18 uses CPU cores [2]
[2023-07-08 22:10:41,119][18272] Worker 16 uses CPU cores [0]
[2023-07-08 22:10:41,137][18262] Worker 6 uses CPU cores [2]
[2023-07-08 22:10:41,156][18270] Worker 14 uses CPU cores [2]
[2023-07-08 22:10:41,172][18266] Worker 11 uses CPU cores [3]
[2023-07-08 22:10:41,191][18275] Worker 19 uses CPU cores [3]
[2023-07-08 22:10:41,201][18268] Worker 9 uses CPU cores [1]
[2023-07-08 22:10:41,214][18273] Worker 17 uses CPU cores [1]
[2023-07-08 22:10:41,218][18267] Worker 12 uses CPU cores [0]
[2023-07-08 22:10:41,218][18271] Worker 15 uses CPU cores [3]
[2023-07-08 22:10:41,227][18265] Worker 10 uses CPU cores [2]
[2023-07-08 22:10:41,377][18255] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 22:12:33,306][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:14:33,308][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:16:33,307][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:18:33,307][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:20:33,307][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:22:33,307][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:24:33,306][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:25:33,309][18274] Stopping RolloutWorker_w18...
[2023-07-08 22:25:33,308][18264] Stopping RolloutWorker_w8...
[2023-07-08 22:25:33,309][18264] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 22:25:33,309][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:25:33,309][18267] Stopping RolloutWorker_w12...
[2023-07-08 22:25:33,309][18267] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 22:25:33,308][18270] Stopping RolloutWorker_w14...
[2023-07-08 22:25:33,310][18258] Stopping RolloutWorker_w2...
[2023-07-08 22:25:33,309][18262] Stopping RolloutWorker_w6...
[2023-07-08 22:25:33,311][18266] Stopping RolloutWorker_w11...
[2023-07-08 22:25:33,309][18265] Stopping RolloutWorker_w10...
[2023-07-08 22:25:33,311][18266] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 22:25:33,310][18270] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 22:25:33,312][18260] Stopping RolloutWorker_w3...
[2023-07-08 22:25:33,312][18257] Stopping RolloutWorker_w1...
[2023-07-08 22:25:33,312][18260] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 22:25:33,312][18257] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 22:25:33,310][18258] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 22:25:33,316][18275] Stopping RolloutWorker_w19...
[2023-07-08 22:25:33,316][18275] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 22:25:33,311][18262] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 22:25:33,311][18274] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 22:25:33,321][18271] Stopping RolloutWorker_w15...
[2023-07-08 22:25:33,311][18265] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 22:25:33,321][18271] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 22:25:33,322][18268] Stopping RolloutWorker_w9...
[2023-07-08 22:25:33,322][18263] Stopping RolloutWorker_w7...
[2023-07-08 22:25:33,322][18256] Stopping RolloutWorker_w0...
[2023-07-08 22:25:33,322][18263] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 22:25:33,322][18268] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 22:25:33,322][18256] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 22:25:33,326][18272] Stopping RolloutWorker_w16...
[2023-07-08 22:25:33,326][18272] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 22:25:33,332][18273] Stopping RolloutWorker_w17...
[2023-07-08 22:25:33,332][18273] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 22:25:33,332][18259] Stopping RolloutWorker_w4...
[2023-07-08 22:25:33,332][18259] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 22:25:33,342][18261] Stopping RolloutWorker_w5...
[2023-07-08 22:25:33,342][18261] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 22:25:33,347][18235] Stopping Batcher_0...
[2023-07-08 22:25:33,348][18235] Loop batcher_evt_loop terminating...
[2023-07-08 22:25:33,352][18269] Stopping RolloutWorker_w13...
[2023-07-08 22:25:33,352][18269] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 22:25:33,428][18235] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000539850_4422451200.pth...
[2023-07-08 22:25:33,539][18235] Stopping LearnerWorker_p0...
[2023-07-08 22:25:33,539][18235] Loop learner_proc0_evt_loop terminating...
[2023-07-08 22:34:04,498][18621] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:34:04,498][18621] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 22:34:04,582][18621] Num visible devices: 1
[2023-07-08 22:34:04,692][18621] Setting fixed seed 42
[2023-07-08 22:34:04,692][18641] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:34:04,692][18641] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 22:34:04,692][18621] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:34:04,693][18621] Initializing actor-critic model on device cuda:0
[2023-07-08 22:34:04,693][18621] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 22:34:04,694][18621] RunningMeanStd input shape: (1,)
[2023-07-08 22:34:04,700][18621] ConvEncoder: input_channels=3
[2023-07-08 22:34:04,744][18641] Num visible devices: 1
[2023-07-08 22:34:04,771][18645] Worker 4 uses CPU cores [0]
[2023-07-08 22:34:04,781][18646] Worker 5 uses CPU cores [1]
[2023-07-08 22:34:04,822][18643] Worker 0 uses CPU cores [0]
[2023-07-08 22:34:04,872][18642] Worker 1 uses CPU cores [1]
[2023-07-08 22:34:05,084][18621] Conv encoder output size: 512
[2023-07-08 22:34:05,102][18621] Policy head output size: 512
[2023-07-08 22:34:05,142][18648] Worker 6 uses CPU cores [2]
[2023-07-08 22:34:05,152][18621] Created Actor Critic model with architecture:
[2023-07-08 22:34:05,152][18621] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 22:34:05,191][18658] Worker 16 uses CPU cores [0]
[2023-07-08 22:34:05,221][18644] Worker 2 uses CPU cores [2]
[2023-07-08 22:34:05,271][18651] Worker 9 uses CPU cores [1]
[2023-07-08 22:34:05,275][18647] Worker 3 uses CPU cores [3]
[2023-07-08 22:34:05,321][18652] Worker 10 uses CPU cores [2]
[2023-07-08 22:34:05,371][18649] Worker 7 uses CPU cores [3]
[2023-07-08 22:34:05,401][18650] Worker 8 uses CPU cores [0]
[2023-07-08 22:34:05,406][18657] Worker 15 uses CPU cores [3]
[2023-07-08 22:34:05,408][18654] Worker 12 uses CPU cores [0]
[2023-07-08 22:34:05,411][18655] Worker 13 uses CPU cores [1]
[2023-07-08 22:34:05,421][18659] Worker 17 uses CPU cores [1]
[2023-07-08 22:34:05,431][18656] Worker 14 uses CPU cores [2]
[2023-07-08 22:34:05,431][18653] Worker 11 uses CPU cores [3]
[2023-07-08 22:34:05,531][18661] Worker 19 uses CPU cores [3]
[2023-07-08 22:34:05,555][18660] Worker 18 uses CPU cores [2]
[2023-07-08 22:34:05,639][18621] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 22:34:05,639][18621] No checkpoints found
[2023-07-08 22:34:05,640][18621] Did not load from checkpoint, starting from scratch!
[2023-07-08 22:34:05,640][18621] Initialized policy 0 weights for model version 0
[2023-07-08 22:34:05,642][18621] LearnerWorker_p0 finished initialization!
[2023-07-08 22:34:05,642][18621] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 22:34:05,765][18641] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 22:35:57,062][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:37:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:39:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:41:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:43:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:45:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:47:57,063][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:48:57,065][18652] Stopping RolloutWorker_w10...
[2023-07-08 22:48:57,065][18621] Stopping Batcher_0...
[2023-07-08 22:48:57,065][18621] Loop batcher_evt_loop terminating...
[2023-07-08 22:48:57,065][18652] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 22:48:57,065][18661] Stopping RolloutWorker_w19...
[2023-07-08 22:48:57,065][18661] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 22:48:57,066][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:48:57,067][18654] Stopping RolloutWorker_w12...
[2023-07-08 22:48:57,067][18654] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 22:48:57,071][18646] Stopping RolloutWorker_w5...
[2023-07-08 22:48:57,071][18648] Stopping RolloutWorker_w6...
[2023-07-08 22:48:57,072][18648] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 22:48:57,072][18646] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 22:48:57,072][18643] Stopping RolloutWorker_w0...
[2023-07-08 22:48:57,072][18650] Stopping RolloutWorker_w8...
[2023-07-08 22:48:57,065][18647] Stopping RolloutWorker_w3...
[2023-07-08 22:48:57,073][18650] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 22:48:57,072][18649] Stopping RolloutWorker_w7...
[2023-07-08 22:48:57,072][18653] Stopping RolloutWorker_w11...
[2023-07-08 22:48:57,072][18657] Stopping RolloutWorker_w15...
[2023-07-08 22:48:57,073][18647] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 22:48:57,073][18649] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 22:48:57,074][18653] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 22:48:57,074][18657] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 22:48:57,082][18660] Stopping RolloutWorker_w18...
[2023-07-08 22:48:57,082][18642] Stopping RolloutWorker_w1...
[2023-07-08 22:48:57,082][18660] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 22:48:57,082][18642] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 22:48:57,083][18645] Stopping RolloutWorker_w4...
[2023-07-08 22:48:57,083][18645] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 22:48:57,092][18656] Stopping RolloutWorker_w14...
[2023-07-08 22:48:57,092][18651] Stopping RolloutWorker_w9...
[2023-07-08 22:48:57,092][18651] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 22:48:57,092][18656] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 22:48:57,082][18658] Stopping RolloutWorker_w16...
[2023-07-08 22:48:57,095][18658] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 22:48:57,098][18643] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 22:48:57,102][18644] Stopping RolloutWorker_w2...
[2023-07-08 22:48:57,102][18655] Stopping RolloutWorker_w13...
[2023-07-08 22:48:57,102][18655] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 22:48:57,102][18644] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 22:48:57,112][18659] Stopping RolloutWorker_w17...
[2023-07-08 22:48:57,112][18659] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 22:48:57,115][18621] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 22:48:57,152][18621] Stopping LearnerWorker_p0...
[2023-07-08 22:48:57,152][18621] Loop learner_proc0_evt_loop terminating...
[2023-07-08 23:01:01,233][19220] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:01:01,234][19220] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 23:01:01,277][19220] Num visible devices: 1
[2023-07-08 23:01:01,412][19220] Setting fixed seed 42
[2023-07-08 23:01:01,412][19220] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:01:01,412][19220] Initializing actor-critic model on device cuda:0
[2023-07-08 23:01:01,413][19220] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 23:01:01,413][19220] RunningMeanStd input shape: (1,)
[2023-07-08 23:01:01,420][19220] ConvEncoder: input_channels=3
[2023-07-08 23:01:01,502][19241] Worker 0 uses CPU cores [0]
[2023-07-08 23:01:01,550][19242] Worker 2 uses CPU cores [2]
[2023-07-08 23:01:01,609][19243] Worker 1 uses CPU cores [1]
[2023-07-08 23:01:01,648][19244] Worker 3 uses CPU cores [3]
[2023-07-08 23:01:01,649][19240] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:01:01,649][19240] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 23:01:01,737][19240] Num visible devices: 1
[2023-07-08 23:01:01,782][19246] Worker 4 uses CPU cores [0]
[2023-07-08 23:01:01,741][19220] Conv encoder output size: 512
[2023-07-08 23:01:01,787][19220] Policy head output size: 512
[2023-07-08 23:01:01,794][19252] Worker 11 uses CPU cores [3]
[2023-07-08 23:01:01,800][19250] Worker 9 uses CPU cores [1]
[2023-07-08 23:01:01,807][19220] Created Actor Critic model with architecture:
[2023-07-08 23:01:01,831][19248] Worker 7 uses CPU cores [3]
[2023-07-08 23:01:01,849][19249] Worker 8 uses CPU cores [0]
[2023-07-08 23:01:01,862][19251] Worker 10 uses CPU cores [2]
[2023-07-08 23:01:01,891][19220] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 23:01:01,903][19253] Worker 12 uses CPU cores [0]
[2023-07-08 23:01:01,925][19247] Worker 6 uses CPU cores [2]
[2023-07-08 23:01:01,943][19254] Worker 13 uses CPU cores [1]
[2023-07-08 23:01:01,967][19256] Worker 15 uses CPU cores [3]
[2023-07-08 23:01:01,975][19259] Worker 18 uses CPU cores [2]
[2023-07-08 23:01:01,991][19255] Worker 14 uses CPU cores [2]
[2023-07-08 23:01:02,001][19257] Worker 16 uses CPU cores [0]
[2023-07-08 23:01:02,021][19260] Worker 19 uses CPU cores [3]
[2023-07-08 23:01:02,041][19245] Worker 5 uses CPU cores [1]
[2023-07-08 23:01:02,052][19258] Worker 17 uses CPU cores [1]
[2023-07-08 23:01:02,219][19220] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 23:01:02,220][19220] No checkpoints found
[2023-07-08 23:01:02,220][19220] Did not load from checkpoint, starting from scratch!
[2023-07-08 23:01:02,220][19220] Initialized policy 0 weights for model version 0
[2023-07-08 23:01:02,223][19220] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:01:02,236][19220] LearnerWorker_p0 finished initialization!
[2023-07-08 23:01:02,356][19240] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 23:02:53,735][19220] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 23:04:53,736][19220] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 23:05:29,863][19259] Stopping RolloutWorker_w18...
[2023-07-08 23:05:29,864][19259] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 23:05:29,864][19260] Stopping RolloutWorker_w19...
[2023-07-08 23:05:29,864][19260] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 23:05:29,872][19252] Stopping RolloutWorker_w11...
[2023-07-08 23:05:29,872][19249] Stopping RolloutWorker_w8...
[2023-07-08 23:05:29,872][19251] Stopping RolloutWorker_w10...
[2023-07-08 23:05:29,872][19252] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 23:05:29,872][19249] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 23:05:29,872][19251] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 23:05:29,882][19255] Stopping RolloutWorker_w14...
[2023-07-08 23:05:29,882][19248] Stopping RolloutWorker_w7...
[2023-07-08 23:05:29,882][19257] Stopping RolloutWorker_w16...
[2023-07-08 23:05:29,882][19255] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 23:05:29,882][19248] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 23:05:29,882][19257] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 23:05:29,882][19220] Stopping Batcher_0...
[2023-07-08 23:05:29,883][19220] Loop batcher_evt_loop terminating...
[2023-07-08 23:05:29,892][19244] Stopping RolloutWorker_w3...
[2023-07-08 23:05:29,892][19241] Stopping RolloutWorker_w0...
[2023-07-08 23:05:29,892][19244] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 23:05:29,892][19241] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 23:05:29,892][19220] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 23:05:29,895][19256] Stopping RolloutWorker_w15...
[2023-07-08 23:05:29,895][19247] Stopping RolloutWorker_w6...
[2023-07-08 23:05:29,895][19256] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 23:05:29,896][19253] Stopping RolloutWorker_w12...
[2023-07-08 23:05:29,896][19247] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 23:05:29,896][19253] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 23:05:29,901][19246] Stopping RolloutWorker_w4...
[2023-07-08 23:05:29,902][19246] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 23:05:29,903][19242] Stopping RolloutWorker_w2...
[2023-07-08 23:05:29,905][19242] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 23:05:29,956][19220] Stopping LearnerWorker_p0...
[2023-07-08 23:05:29,956][19220] Loop learner_proc0_evt_loop terminating...
[2023-07-08 23:05:30,040][19250] Stopping RolloutWorker_w9...
[2023-07-08 23:05:30,041][19250] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 23:05:30,042][19254] Stopping RolloutWorker_w13...
[2023-07-08 23:05:30,042][19254] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 23:05:30,052][19245] Stopping RolloutWorker_w5...
[2023-07-08 23:05:30,052][19245] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 23:05:30,062][19258] Stopping RolloutWorker_w17...
[2023-07-08 23:05:30,062][19258] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 23:05:30,072][19243] Stopping RolloutWorker_w1...
[2023-07-08 23:05:30,072][19243] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 23:06:39,079][19475] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:06:39,079][19475] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
[2023-07-08 23:06:39,116][19475] Num visible devices: 1
[2023-07-08 23:06:39,240][19475] Setting fixed seed 42
[2023-07-08 23:06:39,241][19475] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:06:39,241][19475] Initializing actor-critic model on device cuda:0
[2023-07-08 23:06:39,241][19475] RunningMeanStd input shape: (3, 72, 128)
[2023-07-08 23:06:39,242][19475] RunningMeanStd input shape: (1,)
[2023-07-08 23:06:39,248][19475] ConvEncoder: input_channels=3
[2023-07-08 23:06:39,280][19499] Worker 3 uses CPU cores [3]
[2023-07-08 23:06:39,412][19496] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:06:39,412][19496] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
[2023-07-08 23:06:39,470][19496] Num visible devices: 1
[2023-07-08 23:06:39,462][19500] Worker 4 uses CPU cores [0]
[2023-07-08 23:06:39,527][19495] Worker 0 uses CPU cores [0]
[2023-07-08 23:06:39,541][19497] Worker 1 uses CPU cores [1]
[2023-07-08 23:06:39,541][19502] Worker 6 uses CPU cores [2]
[2023-07-08 23:06:39,604][19475] Conv encoder output size: 512
[2023-07-08 23:06:39,605][19475] Policy head output size: 512
[2023-07-08 23:06:39,608][19507] Worker 12 uses CPU cores [0]
[2023-07-08 23:06:39,646][19475] Created Actor Critic model with architecture:
[2023-07-08 23:06:39,646][19475] ActorCriticSharedWeights(
  (obs_normalizer): ObservationNormalizer(
    (running_mean_std): RunningMeanStdDictInPlace(
      (running_mean_std): ModuleDict(
        (obs): RunningMeanStdInPlace()
      )
    )
  )
  (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
  (encoder): VizdoomEncoder(
    (basic_encoder): ConvEncoder(
      (enc): RecursiveScriptModule(
        original_name=ConvEncoderImpl
        (conv_head): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Conv2d)
          (1): RecursiveScriptModule(original_name=ReLU)
          (2): RecursiveScriptModule(original_name=Conv2d)
          (3): RecursiveScriptModule(original_name=ReLU)
          (4): RecursiveScriptModule(original_name=Conv2d)
          (5): RecursiveScriptModule(original_name=ReLU)
        )
        (mlp_layers): RecursiveScriptModule(
          original_name=Sequential
          (0): RecursiveScriptModule(original_name=Linear)
          (1): RecursiveScriptModule(original_name=ReLU)
        )
      )
    )
  )
  (core): ModelCoreRNN(
    (core): LSTM(512, 512)
  )
  (decoder): MlpDecoder(
    (mlp): Identity()
  )
  (critic_linear): Linear(in_features=512, out_features=1, bias=True)
  (action_parameterization): ActionParameterizationDefault(
    (distribution_linear): Linear(in_features=512, out_features=5, bias=True)
  )
)
[2023-07-08 23:06:39,648][19498] Worker 2 uses CPU cores [2]
[2023-07-08 23:06:39,753][19513] Worker 15 uses CPU cores [3]
[2023-07-08 23:06:39,774][19503] Worker 7 uses CPU cores [3]
[2023-07-08 23:06:39,781][19510] Worker 14 uses CPU cores [2]
[2023-07-08 23:06:39,800][19511] Worker 16 uses CPU cores [0]
[2023-07-08 23:06:39,801][19506] Worker 10 uses CPU cores [2]
[2023-07-08 23:06:39,804][19508] Worker 13 uses CPU cores [1]
[2023-07-08 23:06:39,805][19505] Worker 9 uses CPU cores [1]
[2023-07-08 23:06:39,811][19501] Worker 5 uses CPU cores [1]
[2023-07-08 23:06:39,821][19509] Worker 11 uses CPU cores [3]
[2023-07-08 23:06:39,888][19514] Worker 17 uses CPU cores [1]
[2023-07-08 23:06:39,899][19515] Worker 19 uses CPU cores [3]
[2023-07-08 23:06:39,901][19512] Worker 18 uses CPU cores [2]
[2023-07-08 23:06:39,968][19504] Worker 8 uses CPU cores [0]
[2023-07-08 23:06:40,230][19475] Using optimizer <class 'torch.optim.adam.Adam'>
[2023-07-08 23:06:40,230][19475] Loading state from checkpoint /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000000000_0.pth...
[2023-07-08 23:06:40,254][19475] Loading model from checkpoint
[2023-07-08 23:06:40,257][19475] Loaded experiment state at self.train_step=466273, self.env_steps=3819708416
[2023-07-08 23:06:40,257][19475] Initialized policy 0 weights for model version 466273
[2023-07-08 23:06:40,260][19475] LearnerWorker_p0 finished initialization!
[2023-07-08 23:06:40,260][19475] Using GPUs [0] for process 0 (actually maps to GPUs [0])
[2023-07-08 23:06:40,402][19496] Unhandled exception CUDA error: OS call failed or operation not supported on this OS
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
 in evt loop inference_proc0-0_evt_loop
[2023-07-08 23:08:32,108][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:10:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:12:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:14:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:16:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:18:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:20:32,106][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:21:32,107][19507] Stopping RolloutWorker_w12...
[2023-07-08 23:21:32,107][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:21:32,107][19513] Stopping RolloutWorker_w15...
[2023-07-08 23:21:32,107][19507] Loop rollout_proc12_evt_loop terminating...
[2023-07-08 23:21:32,108][19513] Loop rollout_proc15_evt_loop terminating...
[2023-07-08 23:21:32,108][19505] Stopping RolloutWorker_w9...
[2023-07-08 23:21:32,108][19508] Stopping RolloutWorker_w13...
[2023-07-08 23:21:32,109][19514] Stopping RolloutWorker_w17...
[2023-07-08 23:21:32,109][19501] Stopping RolloutWorker_w5...
[2023-07-08 23:21:32,109][19501] Loop rollout_proc5_evt_loop terminating...
[2023-07-08 23:21:32,109][19502] Stopping RolloutWorker_w6...
[2023-07-08 23:21:32,109][19502] Loop rollout_proc6_evt_loop terminating...
[2023-07-08 23:21:32,107][19497] Stopping RolloutWorker_w1...
[2023-07-08 23:21:32,112][19497] Loop rollout_proc1_evt_loop terminating...
[2023-07-08 23:21:32,112][19509] Stopping RolloutWorker_w11...
[2023-07-08 23:21:32,112][19495] Stopping RolloutWorker_w0...
[2023-07-08 23:21:32,112][19512] Stopping RolloutWorker_w18...
[2023-07-08 23:21:32,112][19495] Loop rollout_proc0_evt_loop terminating...
[2023-07-08 23:21:32,112][19509] Loop rollout_proc11_evt_loop terminating...
[2023-07-08 23:21:32,112][19512] Loop rollout_proc18_evt_loop terminating...
[2023-07-08 23:21:32,117][19499] Stopping RolloutWorker_w3...
[2023-07-08 23:21:32,117][19506] Stopping RolloutWorker_w10...
[2023-07-08 23:21:32,107][19515] Stopping RolloutWorker_w19...
[2023-07-08 23:21:32,116][19503] Stopping RolloutWorker_w7...
[2023-07-08 23:21:32,117][19506] Loop rollout_proc10_evt_loop terminating...
[2023-07-08 23:21:32,117][19499] Loop rollout_proc3_evt_loop terminating...
[2023-07-08 23:21:32,117][19503] Loop rollout_proc7_evt_loop terminating...
[2023-07-08 23:21:32,122][19505] Loop rollout_proc9_evt_loop terminating...
[2023-07-08 23:21:32,122][19498] Stopping RolloutWorker_w2...
[2023-07-08 23:21:32,122][19511] Stopping RolloutWorker_w16...
[2023-07-08 23:21:32,118][19515] Loop rollout_proc19_evt_loop terminating...
[2023-07-08 23:21:32,122][19511] Loop rollout_proc16_evt_loop terminating...
[2023-07-08 23:21:32,122][19498] Loop rollout_proc2_evt_loop terminating...
[2023-07-08 23:21:32,129][19510] Stopping RolloutWorker_w14...
[2023-07-08 23:21:32,129][19510] Loop rollout_proc14_evt_loop terminating...
[2023-07-08 23:21:32,132][19514] Loop rollout_proc17_evt_loop terminating...
[2023-07-08 23:21:32,132][19504] Stopping RolloutWorker_w8...
[2023-07-08 23:21:32,132][19504] Loop rollout_proc8_evt_loop terminating...
[2023-07-08 23:21:32,132][19475] Stopping Batcher_0...
[2023-07-08 23:21:32,132][19475] Loop batcher_evt_loop terminating...
[2023-07-08 23:21:32,136][19508] Loop rollout_proc13_evt_loop terminating...
[2023-07-08 23:21:32,142][19500] Stopping RolloutWorker_w4...
[2023-07-08 23:21:32,142][19500] Loop rollout_proc4_evt_loop terminating...
[2023-07-08 23:21:32,239][19475] Saving /home/raj/repos/HF-DeepRL/8-Proximal-Policy-Optimization/train_dir/default_experiment/checkpoint_p0/checkpoint_000466273_3819708416.pth...
[2023-07-08 23:21:32,373][19475] Stopping LearnerWorker_p0...
[2023-07-08 23:21:32,373][19475] Loop learner_proc0_evt_loop terminating...