kushal-tri commited on
Commit
e40ae71
·
verified ·
1 Parent(s): 72cc66a

Training in progress, epoch 1

Browse files
model-00001-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5248e85868cfadf673f21aff3553bda95fc725be514fcb518f8605431cda1370
3
  size 4874843752
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:44f5977251fb7a78e7eba90933c06e4d9fdafb7c482f49341ad839dcb135d7fe
3
  size 4874843752
model-00002-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:adaaf154d498e47b508bd058351d88fae666bd6923ef97752791cecf35f1a5ad
3
  size 4932751008
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f69ded7665946b680ec2e95710390f0e81ccf697dfdce3fab0d3510896110f46
3
  size 4932751008
model-00003-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:deaded19ac2798595b85aa83a5b121de1ccb598159e52f793d41caeb5ee6bee0
3
  size 4330865200
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:db3bec760b17695fd96cb40d1488940e3ad73f9a01e5510ed76aeb4ef6b1078a
3
  size 4330865200
model-00004-of-00004.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:813e96187ef9e2dec23fdf90b62ac1ae750a434b9a0e73cd0847ce4b79035f8a
3
  size 1087177856
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99f67e9a3557ea5d22a0edb4f723768e5002f064a8a279081df9c3a2faa80834
3
  size 1087177856
wandb/debug-internal.log CHANGED
@@ -371,3 +371,263 @@
371
  2024-11-13 17:19:10,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
372
  2024-11-13 17:19:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
373
  2024-11-13 17:19:11,531 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
371
  2024-11-13 17:19:10,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
372
  2024-11-13 17:19:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
373
  2024-11-13 17:19:11,531 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
374
+ 2024-11-13 17:19:16,532 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
375
+ 2024-11-13 17:19:21,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
376
+ 2024-11-13 17:19:23,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
377
+ 2024-11-13 17:19:24,981 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
378
+ 2024-11-13 17:19:25,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
379
+ 2024-11-13 17:19:25,426 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
380
+ 2024-11-13 17:19:27,525 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
381
+ 2024-11-13 17:19:32,526 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
382
+ 2024-11-13 17:19:37,526 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
383
+ 2024-11-13 17:19:37,986 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
384
+ 2024-11-13 17:19:38,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
385
+ 2024-11-13 17:19:40,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
386
+ 2024-11-13 17:19:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
387
+ 2024-11-13 17:19:43,503 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
388
+ 2024-11-13 17:19:46,989 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
389
+ 2024-11-13 17:19:49,064 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
390
+ 2024-11-13 17:19:53,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
391
+ 2024-11-13 17:19:54,065 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
392
+ 2024-11-13 17:19:55,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
393
+ 2024-11-13 17:19:55,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
394
+ 2024-11-13 17:19:59,572 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
395
+ 2024-11-13 17:20:04,573 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
396
+ 2024-11-13 17:20:07,011 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
397
+ 2024-11-13 17:20:07,012 DEBUG SenderThread:1939 [sender.py:send():382] send: history
398
+ 2024-11-13 17:20:07,013 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
399
+ 2024-11-13 17:20:07,013 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
400
+ 2024-11-13 17:20:07,988 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
401
+ 2024-11-13 17:20:07,998 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
402
+ 2024-11-13 17:20:07,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
403
+ 2024-11-13 17:20:08,654 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
404
+ 2024-11-13 17:20:08,655 DEBUG SenderThread:1939 [sender.py:send():382] send: history
405
+ 2024-11-13 17:20:08,656 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
406
+ 2024-11-13 17:20:08,657 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
407
+ 2024-11-13 17:20:08,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
408
+ 2024-11-13 17:20:08,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
409
+ 2024-11-13 17:20:08,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
410
+ 2024-11-13 17:20:09,658 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
411
+ 2024-11-13 17:20:09,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
412
+ 2024-11-13 17:20:10,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
413
+ 2024-11-13 17:20:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
414
+ 2024-11-13 17:20:10,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
415
+ 2024-11-13 17:20:15,532 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
416
+ 2024-11-13 17:20:20,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
417
+ 2024-11-13 17:20:23,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
418
+ 2024-11-13 17:20:25,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
419
+ 2024-11-13 17:20:25,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
420
+ 2024-11-13 17:20:25,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
421
+ 2024-11-13 17:20:30,534 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
422
+ 2024-11-13 17:20:33,008 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
423
+ 2024-11-13 17:20:35,587 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
424
+ 2024-11-13 17:20:37,991 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
425
+ 2024-11-13 17:20:38,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
426
+ 2024-11-13 17:20:40,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
427
+ 2024-11-13 17:20:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
428
+ 2024-11-13 17:20:41,574 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
429
+ 2024-11-13 17:20:46,575 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-11-13 17:20:51,575 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
431
+ 2024-11-13 17:20:53,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
432
+ 2024-11-13 17:20:55,021 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
433
+ 2024-11-13 17:20:55,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
434
+ 2024-11-13 17:20:55,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
435
+ 2024-11-13 17:20:56,595 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
436
+ 2024-11-13 17:21:01,596 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
437
+ 2024-11-13 17:21:06,597 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
438
+ 2024-11-13 17:21:07,994 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
439
+ 2024-11-13 17:21:08,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
440
+ 2024-11-13 17:21:10,428 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
441
+ 2024-11-13 17:21:10,428 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
442
+ 2024-11-13 17:21:12,570 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
443
+ 2024-11-13 17:21:17,036 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
444
+ 2024-11-13 17:21:17,796 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
445
+ 2024-11-13 17:21:22,796 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
446
+ 2024-11-13 17:21:23,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
447
+ 2024-11-13 17:21:25,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
448
+ 2024-11-13 17:21:25,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
449
+ 2024-11-13 17:21:28,520 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
450
+ 2024-11-13 17:21:33,520 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
451
+ 2024-11-13 17:21:37,996 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
452
+ 2024-11-13 17:21:38,553 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
453
+ 2024-11-13 17:21:38,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
454
+ 2024-11-13 17:21:40,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
455
+ 2024-11-13 17:21:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
456
+ 2024-11-13 17:21:41,048 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
457
+ 2024-11-13 17:21:44,512 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
458
+ 2024-11-13 17:21:49,513 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
459
+ 2024-11-13 17:21:54,514 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
460
+ 2024-11-13 17:21:58,921 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
461
+ 2024-11-13 17:21:58,921 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
462
+ 2024-11-13 17:21:59,233 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
463
+ 2024-11-13 17:22:00,115 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
464
+ 2024-11-13 17:22:05,116 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
465
+ 2024-11-13 17:22:07,999 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
466
+ 2024-11-13 17:22:11,001 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
467
+ 2024-11-13 17:22:13,345 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
468
+ 2024-11-13 17:22:13,345 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
469
+ 2024-11-13 17:22:13,345 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
470
+ 2024-11-13 17:22:16,483 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
471
+ 2024-11-13 17:22:21,484 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
472
+ 2024-11-13 17:22:26,485 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
473
+ 2024-11-13 17:22:29,071 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
474
+ 2024-11-13 17:22:29,261 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
475
+ 2024-11-13 17:22:29,261 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
476
+ 2024-11-13 17:22:29,593 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
477
+ 2024-11-13 17:22:32,181 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
478
+ 2024-11-13 17:22:37,181 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
479
+ 2024-11-13 17:22:38,002 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
480
+ 2024-11-13 17:22:43,004 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
481
+ 2024-11-13 17:22:43,839 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
482
+ 2024-11-13 17:22:43,839 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
483
+ 2024-11-13 17:22:43,840 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
484
+ 2024-11-13 17:22:45,079 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
485
+ 2024-11-13 17:22:47,080 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
486
+ 2024-11-13 17:22:48,290 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
487
+ 2024-11-13 17:22:49,081 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
488
+ 2024-11-13 17:22:51,082 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
489
+ 2024-11-13 17:22:53,083 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
490
+ 2024-11-13 17:22:53,501 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
491
+ 2024-11-13 17:22:55,084 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
492
+ 2024-11-13 17:22:57,085 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
493
+ 2024-11-13 17:22:59,086 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
494
+ 2024-11-13 17:22:59,339 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
495
+ 2024-11-13 17:22:59,339 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
496
+ 2024-11-13 17:22:59,486 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
497
+ 2024-11-13 17:22:59,961 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
498
+ 2024-11-13 17:23:04,808 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
499
+ 2024-11-13 17:23:08,005 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
500
+ 2024-11-13 17:23:10,007 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
501
+ 2024-11-13 17:23:15,008 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
502
+ 2024-11-13 17:23:16,870 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
503
+ 2024-11-13 17:23:17,878 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
504
+ 2024-11-13 17:23:17,878 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
505
+ 2024-11-13 17:23:20,977 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
506
+ 2024-11-13 17:23:25,978 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
507
+ 2024-11-13 17:23:30,117 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
508
+ 2024-11-13 17:23:31,167 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
509
+ 2024-11-13 17:23:31,167 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
510
+ 2024-11-13 17:23:31,334 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
511
+ 2024-11-13 17:23:33,101 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
512
+ 2024-11-13 17:23:36,842 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
513
+ 2024-11-13 17:23:38,008 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
514
+ 2024-11-13 17:23:42,009 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
515
+ 2024-11-13 17:23:43,918 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
516
+ 2024-11-13 17:23:44,229 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
517
+ 2024-11-13 17:23:44,229 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
518
+ 2024-11-13 17:23:47,330 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
519
+ 2024-11-13 17:23:52,331 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
520
+ 2024-11-13 17:23:57,331 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
521
+ 2024-11-13 17:23:58,927 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
522
+ 2024-11-13 17:23:59,229 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
523
+ 2024-11-13 17:23:59,229 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
524
+ 2024-11-13 17:24:02,381 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
525
+ 2024-11-13 17:24:07,382 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
526
+ 2024-11-13 17:24:08,010 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
527
+ 2024-11-13 17:24:13,012 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
528
+ 2024-11-13 17:24:18,013 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
529
+ 2024-11-13 17:24:20,080 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
530
+ 2024-11-13 17:24:20,081 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
531
+ 2024-11-13 17:24:20,425 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
532
+ 2024-11-13 17:24:23,197 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
533
+ 2024-11-13 17:24:28,198 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
534
+ 2024-11-13 17:24:33,199 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
535
+ 2024-11-13 17:24:34,975 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
536
+ 2024-11-13 17:24:34,975 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
537
+ 2024-11-13 17:24:34,976 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
538
+ 2024-11-13 17:24:38,013 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
539
+ 2024-11-13 17:24:39,015 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
540
+ 2024-11-13 17:24:43,133 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
541
+ 2024-11-13 17:24:44,952 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
542
+ 2024-11-13 17:24:49,952 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
543
+ 2024-11-13 17:24:50,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
544
+ 2024-11-13 17:24:50,909 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
545
+ 2024-11-13 17:24:51,584 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
546
+ 2024-11-13 17:24:55,782 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
547
+ 2024-11-13 17:25:00,783 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
548
+ 2024-11-13 17:25:05,028 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
549
+ 2024-11-13 17:25:05,056 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
550
+ 2024-11-13 17:25:05,057 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
551
+ 2024-11-13 17:25:05,144 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
552
+ 2024-11-13 17:25:06,319 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
553
+ 2024-11-13 17:25:07,145 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
554
+ 2024-11-13 17:25:08,016 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
555
+ 2024-11-13 17:25:09,146 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
556
+ 2024-11-13 17:25:11,173 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
557
+ 2024-11-13 17:25:11,566 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
558
+ 2024-11-13 17:25:13,229 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
559
+ 2024-11-13 17:25:16,231 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
560
+ 2024-11-13 17:25:16,767 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
561
+ 2024-11-13 17:25:18,232 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
562
+ 2024-11-13 17:25:20,233 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
563
+ 2024-11-13 17:25:22,557 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
564
+ 2024-11-13 17:25:24,219 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
565
+ 2024-11-13 17:25:24,220 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
566
+ 2024-11-13 17:25:24,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
567
+ 2024-11-13 17:25:28,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
568
+ 2024-11-13 17:25:33,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
569
+ 2024-11-13 17:25:38,019 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
570
+ 2024-11-13 17:25:38,279 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
571
+ 2024-11-13 17:25:38,279 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
572
+ 2024-11-13 17:25:38,373 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
573
+ 2024-11-13 17:25:39,360 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
574
+ 2024-11-13 17:25:44,045 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
575
+ 2024-11-13 17:25:49,045 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
576
+ 2024-11-13 17:25:54,046 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
577
+ 2024-11-13 17:25:55,068 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
578
+ 2024-11-13 17:25:55,069 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
579
+ 2024-11-13 17:25:55,314 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
580
+ 2024-11-13 17:25:59,148 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
581
+ 2024-11-13 17:26:01,619 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
582
+ 2024-11-13 17:26:01,620 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
583
+ 2024-11-13 17:26:01,622 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
584
+ 2024-11-13 17:26:01,622 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
585
+ 2024-11-13 17:26:01,622 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
586
+ 2024-11-13 17:26:01,622 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
587
+ 2024-11-13 17:26:01,623 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
588
+ 2024-11-13 17:26:01,623 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
589
+ 2024-11-13 17:26:01,623 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
590
+ 2024-11-13 17:26:01,623 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
591
+ 2024-11-13 17:26:01,624 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
592
+ 2024-11-13 17:26:01,624 DEBUG SenderThread:1939 [sender.py:send():382] send: history
593
+ 2024-11-13 17:26:01,624 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
594
+ 2024-11-13 17:26:01,624 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
595
+ 2024-11-13 17:26:02,253 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
596
+ 2024-11-13 17:26:02,254 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
597
+ 2024-11-13 17:26:04,255 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
598
+ 2024-11-13 17:26:04,625 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
599
+ 2024-11-13 17:26:08,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
600
+ 2024-11-13 17:26:08,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
601
+ 2024-11-13 17:26:08,130 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
602
+ 2024-11-13 17:26:08,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
603
+ 2024-11-13 17:26:10,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
604
+ 2024-11-13 17:26:15,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
605
+ 2024-11-13 17:26:20,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
606
+ 2024-11-13 17:26:23,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
607
+ 2024-11-13 17:26:23,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
608
+ 2024-11-13 17:26:23,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
609
+ 2024-11-13 17:26:26,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
610
+ 2024-11-13 17:26:31,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
611
+ 2024-11-13 17:26:36,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
612
+ 2024-11-13 17:26:38,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
613
+ 2024-11-13 17:26:38,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
614
+ 2024-11-13 17:26:38,164 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
615
+ 2024-11-13 17:26:38,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
616
+ 2024-11-13 17:26:42,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
617
+ 2024-11-13 17:26:47,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
618
+ 2024-11-13 17:26:52,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
619
+ 2024-11-13 17:26:53,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
620
+ 2024-11-13 17:26:53,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
621
+ 2024-11-13 17:26:53,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
622
+ 2024-11-13 17:26:58,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
623
+ 2024-11-13 17:27:03,303 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
624
+ 2024-11-13 17:27:08,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
625
+ 2024-11-13 17:27:08,015 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
626
+ 2024-11-13 17:27:08,168 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
627
+ 2024-11-13 17:27:08,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
628
+ 2024-11-13 17:27:09,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
629
+ 2024-11-13 17:27:14,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
630
+ 2024-11-13 17:27:19,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
631
+ 2024-11-13 17:27:23,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
632
+ 2024-11-13 17:27:23,015 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
633
+ 2024-11-13 17:27:23,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log CHANGED
The diff for this file is too large to render. See raw diff
 
wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.7172, "train/grad_norm": 0.341796875, "train/learning_rate": 3.0865828381745515e-06, "train/epoch": 1.22, "train/global_step": 15, "_timestamp": 1731518293.2375202, "_runtime": 485.8045001029968, "_step": 6, "eval/loss": 0.6777312159538269, "eval/runtime": 1.6459, "eval/samples_per_second": 61.97, "eval/steps_per_second": 1.215}
 
1
+ {"train/loss": 0.6062, "train/grad_norm": 0.318359375, "train/learning_rate": 6.698729810778065e-07, "train/epoch": 1.94, "train/global_step": 24, "_timestamp": 1731518761.6185403, "_runtime": 954.1855201721191, "_step": 9, "eval/loss": 0.677400529384613, "eval/runtime": 1.6389, "eval/samples_per_second": 62.238, "eval/steps_per_second": 1.22, "train_runtime": 955.0891, "train_samples_per_second": 13.306, "train_steps_per_second": 0.025, "total_flos": 9.400248301309133e+16, "train_loss": 0.6483509813745817}
wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/logs/debug-internal.log CHANGED
@@ -371,3 +371,263 @@
371
  2024-11-13 17:19:10,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
372
  2024-11-13 17:19:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
373
  2024-11-13 17:19:11,531 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
371
  2024-11-13 17:19:10,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
372
  2024-11-13 17:19:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
373
  2024-11-13 17:19:11,531 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
374
+ 2024-11-13 17:19:16,532 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
375
+ 2024-11-13 17:19:21,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
376
+ 2024-11-13 17:19:23,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
377
+ 2024-11-13 17:19:24,981 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
378
+ 2024-11-13 17:19:25,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
379
+ 2024-11-13 17:19:25,426 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
380
+ 2024-11-13 17:19:27,525 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
381
+ 2024-11-13 17:19:32,526 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
382
+ 2024-11-13 17:19:37,526 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
383
+ 2024-11-13 17:19:37,986 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
384
+ 2024-11-13 17:19:38,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
385
+ 2024-11-13 17:19:40,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
386
+ 2024-11-13 17:19:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
387
+ 2024-11-13 17:19:43,503 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
388
+ 2024-11-13 17:19:46,989 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
389
+ 2024-11-13 17:19:49,064 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
390
+ 2024-11-13 17:19:53,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
391
+ 2024-11-13 17:19:54,065 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
392
+ 2024-11-13 17:19:55,426 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
393
+ 2024-11-13 17:19:55,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
394
+ 2024-11-13 17:19:59,572 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
395
+ 2024-11-13 17:20:04,573 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
396
+ 2024-11-13 17:20:07,011 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
397
+ 2024-11-13 17:20:07,012 DEBUG SenderThread:1939 [sender.py:send():382] send: history
398
+ 2024-11-13 17:20:07,013 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
399
+ 2024-11-13 17:20:07,013 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
400
+ 2024-11-13 17:20:07,988 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
401
+ 2024-11-13 17:20:07,998 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
402
+ 2024-11-13 17:20:07,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
403
+ 2024-11-13 17:20:08,654 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
404
+ 2024-11-13 17:20:08,655 DEBUG SenderThread:1939 [sender.py:send():382] send: history
405
+ 2024-11-13 17:20:08,656 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
406
+ 2024-11-13 17:20:08,657 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
407
+ 2024-11-13 17:20:08,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
408
+ 2024-11-13 17:20:08,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
409
+ 2024-11-13 17:20:08,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
410
+ 2024-11-13 17:20:09,658 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
411
+ 2024-11-13 17:20:09,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
412
+ 2024-11-13 17:20:10,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
413
+ 2024-11-13 17:20:10,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
414
+ 2024-11-13 17:20:10,999 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
415
+ 2024-11-13 17:20:15,532 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
416
+ 2024-11-13 17:20:20,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
417
+ 2024-11-13 17:20:23,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
418
+ 2024-11-13 17:20:25,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
419
+ 2024-11-13 17:20:25,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
420
+ 2024-11-13 17:20:25,533 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
421
+ 2024-11-13 17:20:30,534 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
422
+ 2024-11-13 17:20:33,008 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
423
+ 2024-11-13 17:20:35,587 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
424
+ 2024-11-13 17:20:37,991 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
425
+ 2024-11-13 17:20:38,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
426
+ 2024-11-13 17:20:40,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
427
+ 2024-11-13 17:20:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
428
+ 2024-11-13 17:20:41,574 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
429
+ 2024-11-13 17:20:46,575 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
430
+ 2024-11-13 17:20:51,575 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
431
+ 2024-11-13 17:20:53,908 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
432
+ 2024-11-13 17:20:55,021 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
433
+ 2024-11-13 17:20:55,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
434
+ 2024-11-13 17:20:55,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
435
+ 2024-11-13 17:20:56,595 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
436
+ 2024-11-13 17:21:01,596 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
437
+ 2024-11-13 17:21:06,597 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
438
+ 2024-11-13 17:21:07,994 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
439
+ 2024-11-13 17:21:08,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
440
+ 2024-11-13 17:21:10,428 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
441
+ 2024-11-13 17:21:10,428 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
442
+ 2024-11-13 17:21:12,570 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
443
+ 2024-11-13 17:21:17,036 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
444
+ 2024-11-13 17:21:17,796 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
445
+ 2024-11-13 17:21:22,796 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
446
+ 2024-11-13 17:21:23,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
447
+ 2024-11-13 17:21:25,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
448
+ 2024-11-13 17:21:25,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
449
+ 2024-11-13 17:21:28,520 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
450
+ 2024-11-13 17:21:33,520 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
451
+ 2024-11-13 17:21:37,996 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
452
+ 2024-11-13 17:21:38,553 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
453
+ 2024-11-13 17:21:38,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
454
+ 2024-11-13 17:21:40,427 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
455
+ 2024-11-13 17:21:40,427 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
456
+ 2024-11-13 17:21:41,048 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
457
+ 2024-11-13 17:21:44,512 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
458
+ 2024-11-13 17:21:49,513 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
459
+ 2024-11-13 17:21:54,514 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
460
+ 2024-11-13 17:21:58,921 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
461
+ 2024-11-13 17:21:58,921 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
462
+ 2024-11-13 17:21:59,233 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
463
+ 2024-11-13 17:22:00,115 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
464
+ 2024-11-13 17:22:05,116 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
465
+ 2024-11-13 17:22:07,999 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
466
+ 2024-11-13 17:22:11,001 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
467
+ 2024-11-13 17:22:13,345 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
468
+ 2024-11-13 17:22:13,345 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
469
+ 2024-11-13 17:22:13,345 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
470
+ 2024-11-13 17:22:16,483 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
471
+ 2024-11-13 17:22:21,484 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
472
+ 2024-11-13 17:22:26,485 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
473
+ 2024-11-13 17:22:29,071 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
474
+ 2024-11-13 17:22:29,261 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
475
+ 2024-11-13 17:22:29,261 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
476
+ 2024-11-13 17:22:29,593 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
477
+ 2024-11-13 17:22:32,181 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
478
+ 2024-11-13 17:22:37,181 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
479
+ 2024-11-13 17:22:38,002 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
480
+ 2024-11-13 17:22:43,004 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
481
+ 2024-11-13 17:22:43,839 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
482
+ 2024-11-13 17:22:43,839 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
483
+ 2024-11-13 17:22:43,840 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
484
+ 2024-11-13 17:22:45,079 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
485
+ 2024-11-13 17:22:47,080 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
486
+ 2024-11-13 17:22:48,290 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
487
+ 2024-11-13 17:22:49,081 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
488
+ 2024-11-13 17:22:51,082 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
489
+ 2024-11-13 17:22:53,083 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
490
+ 2024-11-13 17:22:53,501 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
491
+ 2024-11-13 17:22:55,084 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
492
+ 2024-11-13 17:22:57,085 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
493
+ 2024-11-13 17:22:59,086 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
494
+ 2024-11-13 17:22:59,339 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
495
+ 2024-11-13 17:22:59,339 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
496
+ 2024-11-13 17:22:59,486 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
497
+ 2024-11-13 17:22:59,961 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
498
+ 2024-11-13 17:23:04,808 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
499
+ 2024-11-13 17:23:08,005 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
500
+ 2024-11-13 17:23:10,007 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
501
+ 2024-11-13 17:23:15,008 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
502
+ 2024-11-13 17:23:16,870 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
503
+ 2024-11-13 17:23:17,878 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
504
+ 2024-11-13 17:23:17,878 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
505
+ 2024-11-13 17:23:20,977 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
506
+ 2024-11-13 17:23:25,978 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
507
+ 2024-11-13 17:23:30,117 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
508
+ 2024-11-13 17:23:31,167 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
509
+ 2024-11-13 17:23:31,167 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
510
+ 2024-11-13 17:23:31,334 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
511
+ 2024-11-13 17:23:33,101 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
512
+ 2024-11-13 17:23:36,842 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
513
+ 2024-11-13 17:23:38,008 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
514
+ 2024-11-13 17:23:42,009 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
515
+ 2024-11-13 17:23:43,918 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
516
+ 2024-11-13 17:23:44,229 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
517
+ 2024-11-13 17:23:44,229 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
518
+ 2024-11-13 17:23:47,330 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
519
+ 2024-11-13 17:23:52,331 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
520
+ 2024-11-13 17:23:57,331 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
521
+ 2024-11-13 17:23:58,927 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
522
+ 2024-11-13 17:23:59,229 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
523
+ 2024-11-13 17:23:59,229 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
524
+ 2024-11-13 17:24:02,381 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
525
+ 2024-11-13 17:24:07,382 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
526
+ 2024-11-13 17:24:08,010 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
527
+ 2024-11-13 17:24:13,012 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
528
+ 2024-11-13 17:24:18,013 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
529
+ 2024-11-13 17:24:20,080 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
530
+ 2024-11-13 17:24:20,081 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
531
+ 2024-11-13 17:24:20,425 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
532
+ 2024-11-13 17:24:23,197 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
533
+ 2024-11-13 17:24:28,198 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
534
+ 2024-11-13 17:24:33,199 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
535
+ 2024-11-13 17:24:34,975 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
536
+ 2024-11-13 17:24:34,975 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
537
+ 2024-11-13 17:24:34,976 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
538
+ 2024-11-13 17:24:38,013 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
539
+ 2024-11-13 17:24:39,015 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
540
+ 2024-11-13 17:24:43,133 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
541
+ 2024-11-13 17:24:44,952 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
542
+ 2024-11-13 17:24:49,952 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
543
+ 2024-11-13 17:24:50,909 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
544
+ 2024-11-13 17:24:50,909 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
545
+ 2024-11-13 17:24:51,584 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
546
+ 2024-11-13 17:24:55,782 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
547
+ 2024-11-13 17:25:00,783 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
548
+ 2024-11-13 17:25:05,028 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
549
+ 2024-11-13 17:25:05,056 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
550
+ 2024-11-13 17:25:05,057 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
551
+ 2024-11-13 17:25:05,144 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
552
+ 2024-11-13 17:25:06,319 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
553
+ 2024-11-13 17:25:07,145 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
554
+ 2024-11-13 17:25:08,016 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
555
+ 2024-11-13 17:25:09,146 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
556
+ 2024-11-13 17:25:11,173 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
557
+ 2024-11-13 17:25:11,566 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
558
+ 2024-11-13 17:25:13,229 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
559
+ 2024-11-13 17:25:16,231 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
560
+ 2024-11-13 17:25:16,767 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
561
+ 2024-11-13 17:25:18,232 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
562
+ 2024-11-13 17:25:20,233 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
563
+ 2024-11-13 17:25:22,557 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
564
+ 2024-11-13 17:25:24,219 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
565
+ 2024-11-13 17:25:24,220 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
566
+ 2024-11-13 17:25:24,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
567
+ 2024-11-13 17:25:28,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
568
+ 2024-11-13 17:25:33,289 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
569
+ 2024-11-13 17:25:38,019 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
570
+ 2024-11-13 17:25:38,279 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
571
+ 2024-11-13 17:25:38,279 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
572
+ 2024-11-13 17:25:38,373 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
573
+ 2024-11-13 17:25:39,360 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
574
+ 2024-11-13 17:25:44,045 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
575
+ 2024-11-13 17:25:49,045 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
576
+ 2024-11-13 17:25:54,046 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
577
+ 2024-11-13 17:25:55,068 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
578
+ 2024-11-13 17:25:55,069 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
579
+ 2024-11-13 17:25:55,314 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
580
+ 2024-11-13 17:25:59,148 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
581
+ 2024-11-13 17:26:01,619 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
582
+ 2024-11-13 17:26:01,620 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: partial_history
583
+ 2024-11-13 17:26:01,622 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
584
+ 2024-11-13 17:26:01,622 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
585
+ 2024-11-13 17:26:01,622 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
586
+ 2024-11-13 17:26:01,622 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
587
+ 2024-11-13 17:26:01,623 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
588
+ 2024-11-13 17:26:01,623 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
589
+ 2024-11-13 17:26:01,623 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
590
+ 2024-11-13 17:26:01,623 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
591
+ 2024-11-13 17:26:01,624 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
592
+ 2024-11-13 17:26:01,624 DEBUG SenderThread:1939 [sender.py:send():382] send: history
593
+ 2024-11-13 17:26:01,624 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: summary_record
594
+ 2024-11-13 17:26:01,624 INFO SenderThread:1939 [sender.py:_save_file():1403] saving file wandb-summary.json with policy end
595
+ 2024-11-13 17:26:02,253 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/wandb-summary.json
596
+ 2024-11-13 17:26:02,254 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
597
+ 2024-11-13 17:26:04,255 INFO Thread-12 :1939 [dir_watcher.py:_on_file_modified():288] file/dir modified: /opt/ml/model//sft-codecontests-qwen_ds-code-contests_model-Qwen2.5-Coder-7B-Instruct_sch-cosine_lr-1e-5_bs-64_acc-8_len-2048/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/files/output.log
598
+ 2024-11-13 17:26:04,625 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
599
+ 2024-11-13 17:26:08,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
600
+ 2024-11-13 17:26:08,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
601
+ 2024-11-13 17:26:08,130 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
602
+ 2024-11-13 17:26:08,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
603
+ 2024-11-13 17:26:10,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
604
+ 2024-11-13 17:26:15,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
605
+ 2024-11-13 17:26:20,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
606
+ 2024-11-13 17:26:23,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
607
+ 2024-11-13 17:26:23,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
608
+ 2024-11-13 17:26:23,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
609
+ 2024-11-13 17:26:26,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
610
+ 2024-11-13 17:26:31,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
611
+ 2024-11-13 17:26:36,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
612
+ 2024-11-13 17:26:38,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
613
+ 2024-11-13 17:26:38,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
614
+ 2024-11-13 17:26:38,164 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
615
+ 2024-11-13 17:26:38,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
616
+ 2024-11-13 17:26:42,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
617
+ 2024-11-13 17:26:47,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
618
+ 2024-11-13 17:26:52,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
619
+ 2024-11-13 17:26:53,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
620
+ 2024-11-13 17:26:53,014 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
621
+ 2024-11-13 17:26:53,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
622
+ 2024-11-13 17:26:58,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
623
+ 2024-11-13 17:27:03,303 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
624
+ 2024-11-13 17:27:08,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
625
+ 2024-11-13 17:27:08,015 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
626
+ 2024-11-13 17:27:08,168 DEBUG SenderThread:1939 [sender.py:send():382] send: stats
627
+ 2024-11-13 17:27:08,299 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
628
+ 2024-11-13 17:27:09,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
629
+ 2024-11-13 17:27:14,301 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
630
+ 2024-11-13 17:27:19,302 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: status_report
631
+ 2024-11-13 17:27:23,014 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: stop_status
632
+ 2024-11-13 17:27:23,015 DEBUG SenderThread:1939 [sender.py:send_request():409] send_request: stop_status
633
+ 2024-11-13 17:27:23,300 DEBUG HandlerThread:1939 [handler.py:handle_request():146] handle_request: internal_messages
wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/run-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1.wandb CHANGED
Binary files a/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/run-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1.wandb and b/wandb/run-20241113_171007-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1/run-kushalarora-rvv-main-2024-11-13-16-43-09-915-0exw0n-algo-1.wandb differ