| 2021-09-20 14:28:10,787 INFO MainThread:8660 [wandb_setup.py:_flush():69] setting env: {} | |
| 2021-09-20 14:28:10,787 INFO MainThread:8660 [wandb_setup.py:_flush():69] setting login settings: {} | |
| 2021-09-20 14:28:10,787 INFO MainThread:8660 [wandb_init.py:_log_setup():348] Logging user logs to /home/leandro/codeparrot/wandb/run-20210920_142810-36cw69uv/logs/debug.log | |
| 2021-09-20 14:28:10,787 INFO MainThread:8660 [wandb_init.py:_log_setup():349] Logging internal logs to /home/leandro/codeparrot/wandb/run-20210920_142810-36cw69uv/logs/debug-internal.log | |
| 2021-09-20 14:28:10,788 INFO MainThread:8660 [wandb_init.py:init():381] calling init triggers | |
| 2021-09-20 14:28:10,788 INFO MainThread:8660 [wandb_init.py:init():386] wandb.init called with sweep_config: {} | |
| config: {'train_batch_size': 2, 'valid_batch_size': 2, 'weight_decay': 0.1, 'shuffle_buffer': 1000, 'learning_rate': 0.0002, 'lr_scheduler_type': 'cosine', 'num_warmup_steps': 750, 'gradient_accumulation_steps': 16, 'max_train_steps': 50000, 'max_eval_steps': -1, 'seq_length': 1024, 'seed': 1, 'save_checkpoint_steps': 50000, 'backend': 'nccl', 'deepspeed_plugin': 'None', 'distributed_type': 'DistributedType.MULTI_GPU', 'num_processes': '16', 'process_index': '0', 'local_process_index': '0', 'device': 'cuda:0', 'use_fp16': 'True', 'initialized': 'True'} | |
| 2021-09-20 14:28:10,788 INFO MainThread:8660 [wandb_init.py:init():430] starting backend | |
| 2021-09-20 14:28:10,788 INFO MainThread:8660 [backend.py:_multiprocessing_setup():70] multiprocessing start_methods=fork,spawn,forkserver, using: spawn | |
| 2021-09-20 14:28:10,805 INFO MainThread:8660 [backend.py:ensure_launched():135] starting backend process... | |
| 2021-09-20 14:28:10,816 INFO MainThread:8660 [backend.py:ensure_launched():139] started backend process with pid: 9038 | |
| 2021-09-20 14:28:10,818 INFO MainThread:8660 [wandb_init.py:init():435] backend started and connected | |
| 2021-09-20 14:28:10,825 INFO MainThread:8660 [wandb_init.py:init():494] updated telemetry | |
| 2021-09-20 14:28:10,826 INFO MainThread:8660 [wandb_init.py:init():517] communicating current version | |
| 2021-09-20 14:28:11,406 INFO MainThread:8660 [wandb_init.py:init():522] got version response | |
| 2021-09-20 14:28:11,406 INFO MainThread:8660 [wandb_init.py:init():530] communicating run to backend with 30 second timeout | |
| 2021-09-20 14:28:11,486 INFO MainThread:8660 [wandb_init.py:init():557] starting run threads in backend | |
| 2021-09-20 14:28:12,872 INFO MainThread:8660 [wandb_run.py:_console_start():1605] atexit reg | |
| 2021-09-20 14:28:12,873 INFO MainThread:8660 [wandb_run.py:_redirect():1479] redirect: SettingsConsole.REDIRECT | |
| 2021-09-20 14:28:12,873 INFO MainThread:8660 [wandb_run.py:_redirect():1484] Redirecting console. | |
| 2021-09-20 14:28:12,876 INFO MainThread:8660 [wandb_run.py:_redirect():1540] Redirects installed. | |
| 2021-09-20 14:28:12,876 INFO MainThread:8660 [wandb_init.py:init():582] run started, returning control to user process | |
| 2021-09-28 04:43:42,819 INFO MainThread:8660 [wandb_run.py:_atexit_cleanup():1575] got exitcode: 0 | |
| 2021-09-28 04:43:42,820 INFO MainThread:8660 [wandb_run.py:_restore():1547] restore | |
| 2021-09-28 04:43:45,246 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 1 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 792 | |
| total_bytes: 792 | |
| } | |
| 2021-09-28 04:43:45,595 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 1 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 792 | |
| total_bytes: 792 | |
| } | |
| 2021-09-28 04:43:45,944 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 3 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 792 | |
| total_bytes: 4501 | |
| } | |
| 2021-09-28 04:43:46,045 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 2743423 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,147 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 16759935 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,248 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 27868287 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,349 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 39042175 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,450 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 50535551 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,551 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 61062271 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,652 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 69983359 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,753 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 80895103 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,855 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 90905727 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:46,957 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 101956735 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,058 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 111901823 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,159 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 122363007 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,260 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 133414015 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,361 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 141810815 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,462 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,563 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,664 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,765 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,866 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:47,973 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| 2021-09-28 04:43:48,123 INFO MainThread:8660 [wandb_run.py:_wait_for_finish():1697] got exit ret: done: true | |
| exit_result { | |
| } | |
| file_counts { | |
| wandb_count: 6 | |
| } | |
| pusher_stats { | |
| uploaded_bytes: 148562394 | |
| total_bytes: 148562394 | |
| } | |
| local_info { | |
| } | |
| 2021-09-28 04:43:49,213 INFO MainThread:8660 [wandb_run.py:_show_summary():1853] rendering summary | |
| 2021-09-28 04:43:49,213 INFO MainThread:8660 [wandb_run.py:_show_history():1898] rendering history | |
| 2021-09-28 04:43:49,214 INFO MainThread:8660 [wandb_run.py:_show_files():1947] logging synced files | |