[2025-01-15 20:40:31,019][18890] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-01-15 20:40:31,022][18890] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 [2025-01-15 20:40:31,093][18890] Num visible devices: 1 [2025-01-15 20:40:31,136][18890] Starting seed is not provided [2025-01-15 20:40:31,138][18890] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-01-15 20:40:31,138][18890] Initializing actor-critic model on device cuda:0 [2025-01-15 20:40:31,139][18890] RunningMeanStd input shape: (3, 72, 128) [2025-01-15 20:40:31,141][18890] RunningMeanStd input shape: (1,) [2025-01-15 20:40:31,228][18890] ConvEncoder: input_channels=3 [2025-01-15 20:40:31,267][18908] Worker 4 uses CPU cores [0] [2025-01-15 20:40:31,383][18910] Worker 7 uses CPU cores [1] [2025-01-15 20:40:31,596][18903] Worker 1 uses CPU cores [1] [2025-01-15 20:40:31,617][18904] Worker 0 uses CPU cores [0] [2025-01-15 20:40:31,658][18906] Worker 2 uses CPU cores [0] [2025-01-15 20:40:31,725][18905] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-01-15 20:40:31,725][18905] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 [2025-01-15 20:40:31,751][18909] Worker 5 uses CPU cores [1] [2025-01-15 20:40:31,763][18907] Worker 3 uses CPU cores [1] [2025-01-15 20:40:31,770][18890] Conv encoder output size: 512 [2025-01-15 20:40:31,771][18890] Policy head output size: 512 [2025-01-15 20:40:31,775][18911] Worker 6 uses CPU cores [0] [2025-01-15 20:40:31,779][18905] Num visible devices: 1 [2025-01-15 20:40:31,799][18890] Created Actor Critic model with architecture: [2025-01-15 20:40:31,800][18890] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2025-01-15 20:40:31,914][18890] Using optimizer [2025-01-15 20:40:32,713][18890] No checkpoints found [2025-01-15 20:40:32,713][18890] Did not load from checkpoint, starting from scratch! [2025-01-15 20:40:32,713][18890] Initialized policy 0 weights for model version 0 [2025-01-15 20:40:32,717][18890] Using GPUs [0] for process 0 (actually maps to GPUs [0]) [2025-01-15 20:40:32,724][18890] LearnerWorker_p0 finished initialization! [2025-01-15 20:40:32,811][18905] RunningMeanStd input shape: (3, 72, 128) [2025-01-15 20:40:32,812][18905] RunningMeanStd input shape: (1,) [2025-01-15 20:40:32,824][18905] ConvEncoder: input_channels=3 [2025-01-15 20:40:32,926][18905] Conv encoder output size: 512 [2025-01-15 20:40:32,927][18905] Policy head output size: 512 [2025-01-15 20:40:33,174][18904] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,177][18906] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,176][18907] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,180][18903] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,178][18909] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,176][18911] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,182][18910] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,181][18908] Doom resolution: 160x120, resize resolution: (128, 72) [2025-01-15 20:40:33,813][18904] Decorrelating experience for 0 frames... [2025-01-15 20:40:34,224][18909] Decorrelating experience for 0 frames... [2025-01-15 20:40:34,243][18903] Decorrelating experience for 0 frames... [2025-01-15 20:40:34,248][18907] Decorrelating experience for 0 frames... [2025-01-15 20:40:34,970][18909] Decorrelating experience for 32 frames... [2025-01-15 20:40:35,000][18903] Decorrelating experience for 32 frames... [2025-01-15 20:40:35,043][18906] Decorrelating experience for 0 frames... [2025-01-15 20:40:35,042][18908] Decorrelating experience for 0 frames... [2025-01-15 20:40:36,561][18904] Decorrelating experience for 32 frames... [2025-01-15 20:40:36,569][18906] Decorrelating experience for 32 frames... [2025-01-15 20:40:36,583][18908] Decorrelating experience for 32 frames... [2025-01-15 20:40:37,320][18907] Decorrelating experience for 32 frames... [2025-01-15 20:40:37,386][18910] Decorrelating experience for 0 frames... [2025-01-15 20:40:37,751][18909] Decorrelating experience for 64 frames... [2025-01-15 20:40:37,784][18903] Decorrelating experience for 64 frames... [2025-01-15 20:40:38,306][18911] Decorrelating experience for 0 frames... [2025-01-15 20:40:38,649][18904] Decorrelating experience for 64 frames... [2025-01-15 20:40:39,093][18906] Decorrelating experience for 64 frames... [2025-01-15 20:40:39,749][18908] Decorrelating experience for 64 frames... [2025-01-15 20:40:39,782][18910] Decorrelating experience for 32 frames... [2025-01-15 20:40:39,825][18907] Decorrelating experience for 64 frames... [2025-01-15 20:40:39,911][18903] Decorrelating experience for 96 frames... [2025-01-15 20:40:40,736][18909] Decorrelating experience for 96 frames... [2025-01-15 20:40:40,862][18904] Decorrelating experience for 96 frames... [2025-01-15 20:40:41,518][18911] Decorrelating experience for 32 frames... [2025-01-15 20:40:42,272][18910] Decorrelating experience for 64 frames... [2025-01-15 20:40:42,711][18907] Decorrelating experience for 96 frames... [2025-01-15 20:40:43,827][18906] Decorrelating experience for 96 frames... [2025-01-15 20:40:45,204][18911] Decorrelating experience for 64 frames... [2025-01-15 20:40:45,678][18890] Signal inference workers to stop experience collection... [2025-01-15 20:40:45,693][18905] InferenceWorker_p0-w0: stopping experience collection [2025-01-15 20:40:45,748][18910] Decorrelating experience for 96 frames... [2025-01-15 20:40:46,017][18908] Decorrelating experience for 96 frames... [2025-01-15 20:40:46,444][18911] Decorrelating experience for 96 frames... [2025-01-15 20:40:48,164][18890] Signal inference workers to resume experience collection... [2025-01-15 20:40:48,165][18905] InferenceWorker_p0-w0: resuming experience collection [2025-01-15 20:40:58,532][18905] Updated weights for policy 0, policy_version 10 (0.0151) [2025-01-15 20:41:07,752][18905] Updated weights for policy 0, policy_version 20 (0.0021) [2025-01-15 20:41:16,654][18890] Saving new best policy, reward=4.286! [2025-01-15 20:41:19,054][18905] Updated weights for policy 0, policy_version 30 (0.0027) [2025-01-15 20:41:21,730][18890] Saving new best policy, reward=4.421! [2025-01-15 20:41:26,657][18890] Saving new best policy, reward=4.544! [2025-01-15 20:41:28,296][18905] Updated weights for policy 0, policy_version 40 (0.0025) [2025-01-15 20:41:39,180][18905] Updated weights for policy 0, policy_version 50 (0.0013) [2025-01-15 20:41:49,606][18905] Updated weights for policy 0, policy_version 60 (0.0024) [2025-01-15 20:41:56,658][18890] Saving new best policy, reward=4.672! [2025-01-15 20:41:59,097][18905] Updated weights for policy 0, policy_version 70 (0.0014) [2025-01-15 20:42:10,377][18905] Updated weights for policy 0, policy_version 80 (0.0034) [2025-01-15 20:42:11,660][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000081_331776.pth... [2025-01-15 20:42:18,927][18905] Updated weights for policy 0, policy_version 90 (0.0022) [2025-01-15 20:42:30,370][18905] Updated weights for policy 0, policy_version 100 (0.0023) [2025-01-15 20:42:39,917][18905] Updated weights for policy 0, policy_version 110 (0.0017) [2025-01-15 20:42:46,656][18890] Saving new best policy, reward=4.700! [2025-01-15 20:42:50,317][18905] Updated weights for policy 0, policy_version 120 (0.0017) [2025-01-15 20:43:01,728][18905] Updated weights for policy 0, policy_version 130 (0.0031) [2025-01-15 20:43:09,954][18905] Updated weights for policy 0, policy_version 140 (0.0020) [2025-01-15 20:43:21,208][18905] Updated weights for policy 0, policy_version 150 (0.0033) [2025-01-15 20:43:21,663][18890] Saving new best policy, reward=4.805! [2025-01-15 20:43:26,658][18890] Saving new best policy, reward=4.912! [2025-01-15 20:43:30,742][18905] Updated weights for policy 0, policy_version 160 (0.0033) [2025-01-15 20:43:31,677][18890] Saving new best policy, reward=5.011! [2025-01-15 20:43:36,656][18890] Saving new best policy, reward=5.529! [2025-01-15 20:43:41,256][18905] Updated weights for policy 0, policy_version 170 (0.0020) [2025-01-15 20:43:51,796][18905] Updated weights for policy 0, policy_version 180 (0.0025) [2025-01-15 20:44:00,760][18905] Updated weights for policy 0, policy_version 190 (0.0016) [2025-01-15 20:44:01,661][18890] Saving new best policy, reward=5.920! [2025-01-15 20:44:06,656][18890] Saving new best policy, reward=6.300! [2025-01-15 20:44:11,662][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000199_815104.pth... [2025-01-15 20:44:12,157][18905] Updated weights for policy 0, policy_version 200 (0.0020) [2025-01-15 20:44:20,549][18905] Updated weights for policy 0, policy_version 210 (0.0016) [2025-01-15 20:44:21,663][18890] Saving new best policy, reward=6.683! [2025-01-15 20:44:31,929][18905] Updated weights for policy 0, policy_version 220 (0.0022) [2025-01-15 20:44:36,656][18890] Saving new best policy, reward=6.875! [2025-01-15 20:44:41,670][18890] Saving new best policy, reward=7.165! [2025-01-15 20:44:41,990][18905] Updated weights for policy 0, policy_version 230 (0.0020) [2025-01-15 20:44:46,657][18890] Saving new best policy, reward=7.461! [2025-01-15 20:44:51,665][18890] Saving new best policy, reward=7.797! [2025-01-15 20:44:51,887][18905] Updated weights for policy 0, policy_version 240 (0.0022) [2025-01-15 20:45:03,081][18905] Updated weights for policy 0, policy_version 250 (0.0027) [2025-01-15 20:45:11,328][18905] Updated weights for policy 0, policy_version 260 (0.0021) [2025-01-15 20:45:11,663][18890] Saving new best policy, reward=8.564! [2025-01-15 20:45:16,659][18890] Saving new best policy, reward=8.706! [2025-01-15 20:45:21,661][18890] Saving new best policy, reward=9.101! [2025-01-15 20:45:22,805][18905] Updated weights for policy 0, policy_version 270 (0.0018) [2025-01-15 20:45:26,658][18890] Saving new best policy, reward=9.152! [2025-01-15 20:45:32,452][18905] Updated weights for policy 0, policy_version 280 (0.0025) [2025-01-15 20:45:36,661][18890] Saving new best policy, reward=9.238! [2025-01-15 20:45:41,663][18890] Saving new best policy, reward=9.757! [2025-01-15 20:45:42,863][18905] Updated weights for policy 0, policy_version 290 (0.0032) [2025-01-15 20:45:46,655][18890] Saving new best policy, reward=10.477! [2025-01-15 20:45:53,611][18905] Updated weights for policy 0, policy_version 300 (0.0020) [2025-01-15 20:45:56,711][18890] Saving new best policy, reward=10.766! [2025-01-15 20:46:01,659][18890] Saving new best policy, reward=11.492! [2025-01-15 20:46:02,942][18905] Updated weights for policy 0, policy_version 310 (0.0014) [2025-01-15 20:46:06,656][18890] Saving new best policy, reward=12.060! [2025-01-15 20:46:11,663][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000316_1294336.pth... [2025-01-15 20:46:11,820][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000081_331776.pth [2025-01-15 20:46:11,836][18890] Saving new best policy, reward=12.845! [2025-01-15 20:46:14,719][18905] Updated weights for policy 0, policy_version 320 (0.0027) [2025-01-15 20:46:16,657][18890] Saving new best policy, reward=12.907! [2025-01-15 20:46:24,108][18905] Updated weights for policy 0, policy_version 330 (0.0028) [2025-01-15 20:46:31,663][18890] Saving new best policy, reward=13.885! [2025-01-15 20:46:35,142][18905] Updated weights for policy 0, policy_version 340 (0.0031) [2025-01-15 20:46:46,257][18905] Updated weights for policy 0, policy_version 350 (0.0030) [2025-01-15 20:46:46,664][18890] Saving new best policy, reward=14.050! [2025-01-15 20:46:51,667][18890] Saving new best policy, reward=14.325! [2025-01-15 20:46:55,987][18905] Updated weights for policy 0, policy_version 360 (0.0023) [2025-01-15 20:46:56,663][18890] Saving new best policy, reward=15.430! [2025-01-15 20:47:07,932][18905] Updated weights for policy 0, policy_version 370 (0.0028) [2025-01-15 20:47:17,492][18905] Updated weights for policy 0, policy_version 380 (0.0028) [2025-01-15 20:47:26,656][18890] Saving new best policy, reward=15.671! [2025-01-15 20:47:28,279][18905] Updated weights for policy 0, policy_version 390 (0.0017) [2025-01-15 20:47:31,661][18890] Saving new best policy, reward=16.523! [2025-01-15 20:47:39,939][18905] Updated weights for policy 0, policy_version 400 (0.0017) [2025-01-15 20:47:46,656][18890] Saving new best policy, reward=18.349! [2025-01-15 20:47:48,655][18905] Updated weights for policy 0, policy_version 410 (0.0024) [2025-01-15 20:47:51,668][18890] Saving new best policy, reward=19.337! [2025-01-15 20:48:00,447][18905] Updated weights for policy 0, policy_version 420 (0.0054) [2025-01-15 20:48:01,670][18890] Saving new best policy, reward=21.156! [2025-01-15 20:48:10,522][18905] Updated weights for policy 0, policy_version 430 (0.0024) [2025-01-15 20:48:11,672][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000430_1761280.pth... [2025-01-15 20:48:11,873][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000199_815104.pth [2025-01-15 20:48:16,653][18890] Saving new best policy, reward=21.622! [2025-01-15 20:48:20,712][18905] Updated weights for policy 0, policy_version 440 (0.0023) [2025-01-15 20:48:21,663][18890] Saving new best policy, reward=23.133! [2025-01-15 20:48:32,132][18905] Updated weights for policy 0, policy_version 450 (0.0033) [2025-01-15 20:48:40,538][18905] Updated weights for policy 0, policy_version 460 (0.0030) [2025-01-15 20:48:51,879][18905] Updated weights for policy 0, policy_version 470 (0.0042) [2025-01-15 20:49:01,033][18905] Updated weights for policy 0, policy_version 480 (0.0022) [2025-01-15 20:49:11,656][18905] Updated weights for policy 0, policy_version 490 (0.0024) [2025-01-15 20:49:22,367][18905] Updated weights for policy 0, policy_version 500 (0.0022) [2025-01-15 20:49:31,613][18905] Updated weights for policy 0, policy_version 510 (0.0031) [2025-01-15 20:49:42,685][18905] Updated weights for policy 0, policy_version 520 (0.0052) [2025-01-15 20:49:51,425][18905] Updated weights for policy 0, policy_version 530 (0.0042) [2025-01-15 20:50:02,716][18905] Updated weights for policy 0, policy_version 540 (0.0031) [2025-01-15 20:50:11,674][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000549_2248704.pth... [2025-01-15 20:50:11,854][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000316_1294336.pth [2025-01-15 20:50:12,874][18905] Updated weights for policy 0, policy_version 550 (0.0026) [2025-01-15 20:50:22,584][18905] Updated weights for policy 0, policy_version 560 (0.0047) [2025-01-15 20:50:33,989][18905] Updated weights for policy 0, policy_version 570 (0.0019) [2025-01-15 20:50:42,382][18905] Updated weights for policy 0, policy_version 580 (0.0013) [2025-01-15 20:50:53,839][18905] Updated weights for policy 0, policy_version 590 (0.0021) [2025-01-15 20:50:56,654][18890] Saving new best policy, reward=23.940! [2025-01-15 20:51:03,550][18905] Updated weights for policy 0, policy_version 600 (0.0031) [2025-01-15 20:51:13,866][18905] Updated weights for policy 0, policy_version 610 (0.0021) [2025-01-15 20:51:24,952][18905] Updated weights for policy 0, policy_version 620 (0.0030) [2025-01-15 20:51:33,904][18905] Updated weights for policy 0, policy_version 630 (0.0021) [2025-01-15 20:51:45,397][18905] Updated weights for policy 0, policy_version 640 (0.0030) [2025-01-15 20:51:54,896][18905] Updated weights for policy 0, policy_version 650 (0.0018) [2025-01-15 20:52:05,840][18905] Updated weights for policy 0, policy_version 660 (0.0030) [2025-01-15 20:52:11,666][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000666_2727936.pth... [2025-01-15 20:52:11,834][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000430_1761280.pth [2025-01-15 20:52:16,965][18905] Updated weights for policy 0, policy_version 670 (0.0029) [2025-01-15 20:52:26,215][18905] Updated weights for policy 0, policy_version 680 (0.0020) [2025-01-15 20:52:31,658][18890] Saving new best policy, reward=24.357! [2025-01-15 20:52:38,122][18905] Updated weights for policy 0, policy_version 690 (0.0028) [2025-01-15 20:52:41,670][18890] Saving new best policy, reward=26.488! [2025-01-15 20:52:46,669][18890] Saving new best policy, reward=26.503! [2025-01-15 20:52:47,883][18905] Updated weights for policy 0, policy_version 700 (0.0020) [2025-01-15 20:52:51,667][18890] Saving new best policy, reward=27.526! [2025-01-15 20:52:56,658][18890] Saving new best policy, reward=29.132! [2025-01-15 20:52:58,673][18905] Updated weights for policy 0, policy_version 710 (0.0026) [2025-01-15 20:53:09,768][18905] Updated weights for policy 0, policy_version 720 (0.0025) [2025-01-15 20:53:18,657][18905] Updated weights for policy 0, policy_version 730 (0.0014) [2025-01-15 20:53:30,135][18905] Updated weights for policy 0, policy_version 740 (0.0015) [2025-01-15 20:53:38,869][18905] Updated weights for policy 0, policy_version 750 (0.0014) [2025-01-15 20:53:49,919][18905] Updated weights for policy 0, policy_version 760 (0.0019) [2025-01-15 20:54:00,558][18905] Updated weights for policy 0, policy_version 770 (0.0022) [2025-01-15 20:54:09,799][18905] Updated weights for policy 0, policy_version 780 (0.0034) [2025-01-15 20:54:11,659][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000782_3203072.pth... [2025-01-15 20:54:11,783][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000549_2248704.pth [2025-01-15 20:54:21,517][18905] Updated weights for policy 0, policy_version 790 (0.0016) [2025-01-15 20:54:29,995][18905] Updated weights for policy 0, policy_version 800 (0.0030) [2025-01-15 20:54:41,458][18905] Updated weights for policy 0, policy_version 810 (0.0022) [2025-01-15 20:54:51,392][18905] Updated weights for policy 0, policy_version 820 (0.0022) [2025-01-15 20:55:01,548][18905] Updated weights for policy 0, policy_version 830 (0.0013) [2025-01-15 20:55:13,403][18905] Updated weights for policy 0, policy_version 840 (0.0018) [2025-01-15 20:55:22,095][18905] Updated weights for policy 0, policy_version 850 (0.0019) [2025-01-15 20:55:33,757][18905] Updated weights for policy 0, policy_version 860 (0.0018) [2025-01-15 20:55:44,201][18905] Updated weights for policy 0, policy_version 870 (0.0021) [2025-01-15 20:55:53,652][18905] Updated weights for policy 0, policy_version 880 (0.0013) [2025-01-15 20:56:05,098][18905] Updated weights for policy 0, policy_version 890 (0.0018) [2025-01-15 20:56:11,664][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000897_3674112.pth... [2025-01-15 20:56:11,799][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000666_2727936.pth [2025-01-15 20:56:13,697][18905] Updated weights for policy 0, policy_version 900 (0.0024) [2025-01-15 20:56:24,930][18905] Updated weights for policy 0, policy_version 910 (0.0020) [2025-01-15 20:56:34,779][18905] Updated weights for policy 0, policy_version 920 (0.0013) [2025-01-15 20:56:45,592][18905] Updated weights for policy 0, policy_version 930 (0.0014) [2025-01-15 20:56:57,177][18905] Updated weights for policy 0, policy_version 940 (0.0024) [2025-01-15 20:57:05,655][18905] Updated weights for policy 0, policy_version 950 (0.0024) [2025-01-15 20:57:17,074][18905] Updated weights for policy 0, policy_version 960 (0.0024) [2025-01-15 20:57:26,621][18905] Updated weights for policy 0, policy_version 970 (0.0037) [2025-01-15 20:57:37,201][18905] Updated weights for policy 0, policy_version 980 (0.0021) [2025-01-15 20:57:48,395][18905] Updated weights for policy 0, policy_version 990 (0.0025) [2025-01-15 20:57:57,347][18905] Updated weights for policy 0, policy_version 1000 (0.0035) [2025-01-15 20:58:08,940][18905] Updated weights for policy 0, policy_version 1010 (0.0019) [2025-01-15 20:58:11,658][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001013_4149248.pth... [2025-01-15 20:58:11,781][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000782_3203072.pth [2025-01-15 20:58:17,501][18905] Updated weights for policy 0, policy_version 1020 (0.0021) [2025-01-15 20:58:28,742][18905] Updated weights for policy 0, policy_version 1030 (0.0028) [2025-01-15 20:58:39,007][18905] Updated weights for policy 0, policy_version 1040 (0.0021) [2025-01-15 20:58:48,820][18905] Updated weights for policy 0, policy_version 1050 (0.0022) [2025-01-15 20:59:00,172][18905] Updated weights for policy 0, policy_version 1060 (0.0032) [2025-01-15 20:59:06,655][18890] Saving new best policy, reward=29.510! [2025-01-15 20:59:09,059][18905] Updated weights for policy 0, policy_version 1070 (0.0036) [2025-01-15 20:59:20,728][18905] Updated weights for policy 0, policy_version 1080 (0.0023) [2025-01-15 20:59:21,674][18890] Saving new best policy, reward=30.241! [2025-01-15 20:59:31,347][18905] Updated weights for policy 0, policy_version 1090 (0.0032) [2025-01-15 20:59:41,468][18905] Updated weights for policy 0, policy_version 1100 (0.0015) [2025-01-15 20:59:53,103][18905] Updated weights for policy 0, policy_version 1110 (0.0017) [2025-01-15 21:00:02,007][18905] Updated weights for policy 0, policy_version 1120 (0.0023) [2025-01-15 21:00:11,660][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001127_4616192.pth... [2025-01-15 21:00:11,784][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000897_3674112.pth [2025-01-15 21:00:13,992][18905] Updated weights for policy 0, policy_version 1130 (0.0017) [2025-01-15 21:00:24,610][18905] Updated weights for policy 0, policy_version 1140 (0.0023) [2025-01-15 21:00:34,395][18905] Updated weights for policy 0, policy_version 1150 (0.0020) [2025-01-15 21:00:45,859][18905] Updated weights for policy 0, policy_version 1160 (0.0032) [2025-01-15 21:00:54,606][18905] Updated weights for policy 0, policy_version 1170 (0.0028) [2025-01-15 21:01:06,189][18905] Updated weights for policy 0, policy_version 1180 (0.0018) [2025-01-15 21:01:17,374][18905] Updated weights for policy 0, policy_version 1190 (0.0019) [2025-01-15 21:01:27,577][18905] Updated weights for policy 0, policy_version 1200 (0.0026) [2025-01-15 21:01:39,529][18905] Updated weights for policy 0, policy_version 1210 (0.0036) [2025-01-15 21:01:49,185][18905] Updated weights for policy 0, policy_version 1220 (0.0018) [2025-01-15 21:02:00,663][18905] Updated weights for policy 0, policy_version 1230 (0.0036) [2025-01-15 21:02:11,666][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001239_5074944.pth... [2025-01-15 21:02:11,855][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001013_4149248.pth [2025-01-15 21:02:12,642][18905] Updated weights for policy 0, policy_version 1240 (0.0033) [2025-01-15 21:02:21,514][18905] Updated weights for policy 0, policy_version 1250 (0.0025) [2025-01-15 21:02:33,048][18905] Updated weights for policy 0, policy_version 1260 (0.0019) [2025-01-15 21:02:43,319][18905] Updated weights for policy 0, policy_version 1270 (0.0018) [2025-01-15 21:02:53,434][18905] Updated weights for policy 0, policy_version 1280 (0.0051) [2025-01-15 21:03:04,850][18905] Updated weights for policy 0, policy_version 1290 (0.0014) [2025-01-15 21:03:14,014][18905] Updated weights for policy 0, policy_version 1300 (0.0017) [2025-01-15 21:03:25,945][18905] Updated weights for policy 0, policy_version 1310 (0.0021) [2025-01-15 21:03:36,884][18905] Updated weights for policy 0, policy_version 1320 (0.0019) [2025-01-15 21:03:46,246][18905] Updated weights for policy 0, policy_version 1330 (0.0013) [2025-01-15 21:03:57,758][18905] Updated weights for policy 0, policy_version 1340 (0.0018) [2025-01-15 21:04:06,345][18905] Updated weights for policy 0, policy_version 1350 (0.0017) [2025-01-15 21:04:11,657][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001353_5541888.pth... [2025-01-15 21:04:11,847][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001127_4616192.pth [2025-01-15 21:04:17,554][18905] Updated weights for policy 0, policy_version 1360 (0.0019) [2025-01-15 21:04:28,374][18905] Updated weights for policy 0, policy_version 1370 (0.0024) [2025-01-15 21:04:37,717][18905] Updated weights for policy 0, policy_version 1380 (0.0027) [2025-01-15 21:04:49,511][18905] Updated weights for policy 0, policy_version 1390 (0.0025) [2025-01-15 21:04:58,799][18905] Updated weights for policy 0, policy_version 1400 (0.0024) [2025-01-15 21:05:10,020][18905] Updated weights for policy 0, policy_version 1410 (0.0023) [2025-01-15 21:05:21,268][18905] Updated weights for policy 0, policy_version 1420 (0.0024) [2025-01-15 21:05:30,178][18905] Updated weights for policy 0, policy_version 1430 (0.0028) [2025-01-15 21:05:42,217][18905] Updated weights for policy 0, policy_version 1440 (0.0023) [2025-01-15 21:05:51,587][18905] Updated weights for policy 0, policy_version 1450 (0.0026) [2025-01-15 21:06:02,491][18905] Updated weights for policy 0, policy_version 1460 (0.0020) [2025-01-15 21:06:11,665][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001468_6012928.pth... [2025-01-15 21:06:11,843][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001239_5074944.pth [2025-01-15 21:06:14,352][18905] Updated weights for policy 0, policy_version 1470 (0.0025) [2025-01-15 21:06:22,985][18905] Updated weights for policy 0, policy_version 1480 (0.0023) [2025-01-15 21:06:34,583][18905] Updated weights for policy 0, policy_version 1490 (0.0031) [2025-01-15 21:06:45,001][18905] Updated weights for policy 0, policy_version 1500 (0.0016) [2025-01-15 21:06:55,011][18905] Updated weights for policy 0, policy_version 1510 (0.0024) [2025-01-15 21:07:06,610][18905] Updated weights for policy 0, policy_version 1520 (0.0038) [2025-01-15 21:07:15,316][18905] Updated weights for policy 0, policy_version 1530 (0.0018) [2025-01-15 21:07:26,970][18905] Updated weights for policy 0, policy_version 1540 (0.0021) [2025-01-15 21:07:36,805][18905] Updated weights for policy 0, policy_version 1550 (0.0028) [2025-01-15 21:07:47,252][18905] Updated weights for policy 0, policy_version 1560 (0.0023) [2025-01-15 21:07:59,083][18905] Updated weights for policy 0, policy_version 1570 (0.0022) [2025-01-15 21:08:08,032][18905] Updated weights for policy 0, policy_version 1580 (0.0025) [2025-01-15 21:08:11,659][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001582_6479872.pth... [2025-01-15 21:08:11,810][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001353_5541888.pth [2025-01-15 21:08:19,825][18905] Updated weights for policy 0, policy_version 1590 (0.0016) [2025-01-15 21:08:30,054][18905] Updated weights for policy 0, policy_version 1600 (0.0037) [2025-01-15 21:08:40,211][18905] Updated weights for policy 0, policy_version 1610 (0.0042) [2025-01-15 21:08:51,935][18905] Updated weights for policy 0, policy_version 1620 (0.0024) [2025-01-15 21:09:00,171][18905] Updated weights for policy 0, policy_version 1630 (0.0024) [2025-01-15 21:09:11,789][18905] Updated weights for policy 0, policy_version 1640 (0.0032) [2025-01-15 21:09:21,556][18905] Updated weights for policy 0, policy_version 1650 (0.0025) [2025-01-15 21:09:31,744][18905] Updated weights for policy 0, policy_version 1660 (0.0022) [2025-01-15 21:09:43,156][18905] Updated weights for policy 0, policy_version 1670 (0.0026) [2025-01-15 21:09:51,515][18905] Updated weights for policy 0, policy_version 1680 (0.0022) [2025-01-15 21:10:02,980][18905] Updated weights for policy 0, policy_version 1690 (0.0033) [2025-01-15 21:10:11,669][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001699_6959104.pth... [2025-01-15 21:10:11,859][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001468_6012928.pth [2025-01-15 21:10:12,308][18905] Updated weights for policy 0, policy_version 1700 (0.0013) [2025-01-15 21:10:23,074][18905] Updated weights for policy 0, policy_version 1710 (0.0021) [2025-01-15 21:10:33,761][18905] Updated weights for policy 0, policy_version 1720 (0.0030) [2025-01-15 21:10:42,945][18905] Updated weights for policy 0, policy_version 1730 (0.0017) [2025-01-15 21:10:54,194][18905] Updated weights for policy 0, policy_version 1740 (0.0028) [2025-01-15 21:11:03,021][18905] Updated weights for policy 0, policy_version 1750 (0.0017) [2025-01-15 21:11:14,051][18905] Updated weights for policy 0, policy_version 1760 (0.0027) [2025-01-15 21:11:24,540][18905] Updated weights for policy 0, policy_version 1770 (0.0015) [2025-01-15 21:11:33,893][18905] Updated weights for policy 0, policy_version 1780 (0.0024) [2025-01-15 21:11:45,299][18905] Updated weights for policy 0, policy_version 1790 (0.0018) [2025-01-15 21:11:53,747][18905] Updated weights for policy 0, policy_version 1800 (0.0024) [2025-01-15 21:12:04,987][18905] Updated weights for policy 0, policy_version 1810 (0.0044) [2025-01-15 21:12:11,667][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001817_7442432.pth... [2025-01-15 21:12:11,834][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001582_6479872.pth [2025-01-15 21:12:14,806][18905] Updated weights for policy 0, policy_version 1820 (0.0015) [2025-01-15 21:12:24,843][18905] Updated weights for policy 0, policy_version 1830 (0.0019) [2025-01-15 21:12:35,856][18905] Updated weights for policy 0, policy_version 1840 (0.0013) [2025-01-15 21:12:44,627][18905] Updated weights for policy 0, policy_version 1850 (0.0018) [2025-01-15 21:12:55,985][18905] Updated weights for policy 0, policy_version 1860 (0.0021) [2025-01-15 21:13:04,731][18905] Updated weights for policy 0, policy_version 1870 (0.0021) [2025-01-15 21:13:11,668][18890] Saving new best policy, reward=30.845! [2025-01-15 21:13:15,672][18905] Updated weights for policy 0, policy_version 1880 (0.0025) [2025-01-15 21:13:26,680][18905] Updated weights for policy 0, policy_version 1890 (0.0023) [2025-01-15 21:13:35,543][18905] Updated weights for policy 0, policy_version 1900 (0.0025) [2025-01-15 21:13:46,924][18905] Updated weights for policy 0, policy_version 1910 (0.0029) [2025-01-15 21:13:55,349][18905] Updated weights for policy 0, policy_version 1920 (0.0022) [2025-01-15 21:14:06,595][18905] Updated weights for policy 0, policy_version 1930 (0.0018) [2025-01-15 21:14:11,668][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001936_7929856.pth... [2025-01-15 21:14:11,794][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001699_6959104.pth [2025-01-15 21:14:16,872][18905] Updated weights for policy 0, policy_version 1940 (0.0020) [2025-01-15 21:14:26,483][18905] Updated weights for policy 0, policy_version 1950 (0.0031) [2025-01-15 21:14:31,130][18890] Stopping Batcher_0... [2025-01-15 21:14:31,131][18890] Loop batcher_evt_loop terminating... [2025-01-15 21:14:31,139][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2025-01-15 21:14:31,254][18905] Weights refcount: 2 0 [2025-01-15 21:14:31,267][18905] Stopping InferenceWorker_p0-w0... [2025-01-15 21:14:31,267][18905] Loop inference_proc0-0_evt_loop terminating... [2025-01-15 21:14:31,281][18890] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001817_7442432.pth [2025-01-15 21:14:31,299][18890] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000001955_8007680.pth... [2025-01-15 21:14:31,530][18890] Stopping LearnerWorker_p0... [2025-01-15 21:14:31,531][18890] Loop learner_proc0_evt_loop terminating... [2025-01-15 21:14:31,930][18908] Stopping RolloutWorker_w4... [2025-01-15 21:14:31,932][18908] Loop rollout_proc4_evt_loop terminating... [2025-01-15 21:14:31,947][18906] Stopping RolloutWorker_w2... [2025-01-15 21:14:31,954][18906] Loop rollout_proc2_evt_loop terminating... [2025-01-15 21:14:31,985][18904] Stopping RolloutWorker_w0... [2025-01-15 21:14:31,987][18904] Loop rollout_proc0_evt_loop terminating... [2025-01-15 21:14:31,995][18910] Stopping RolloutWorker_w7... [2025-01-15 21:14:31,996][18910] Loop rollout_proc7_evt_loop terminating... [2025-01-15 21:14:32,004][18911] Stopping RolloutWorker_w6... [2025-01-15 21:14:32,008][18911] Loop rollout_proc6_evt_loop terminating... [2025-01-15 21:14:32,038][18907] Stopping RolloutWorker_w3... [2025-01-15 21:14:32,042][18907] Loop rollout_proc3_evt_loop terminating... [2025-01-15 21:14:32,051][18909] Stopping RolloutWorker_w5... [2025-01-15 21:14:32,058][18909] Loop rollout_proc5_evt_loop terminating... [2025-01-15 21:14:32,061][18903] Stopping RolloutWorker_w1... [2025-01-15 21:14:32,076][18903] Loop rollout_proc1_evt_loop terminating...