|
2025-01-20 22:53:02,316 INFO monitor.py:688 -- Starting monitor using ray installation: /usr/local/lib/python3.10/dist-packages/ray/__init__.py |
|
2025-01-20 22:53:02,316 INFO monitor.py:689 -- Ray version: 2.40.0 |
|
2025-01-20 22:53:02,317 INFO monitor.py:690 -- Ray commit: 22541c38dbef25286cd6d19f1c151bf4fd62f2ed |
|
2025-01-20 22:53:02,317 INFO monitor.py:691 -- Monitor started with command: ['/usr/local/lib/python3.10/dist-packages/ray/autoscaler/_private/monitor.py', '--logs-dir=/tmp/ray/session_2025-01-20_22-53-01_396103_8700/logs', '--logging-rotate-bytes=536870912', '--logging-rotate-backup-count=5', '--gcs-address=192.168.0.2:59305', '--monitor-ip=192.168.0.2'] |
|
2025-01-20 22:53:02,336 INFO monitor.py:159 -- session_name: session_2025-01-20_22-53-01_396103_8700 |
|
2025-01-20 22:53:02,339 INFO monitor.py:191 -- Starting autoscaler metrics server on port 44217 |
|
2025-01-20 22:53:02,340 INFO monitor.py:216 -- Monitor: Started |
|
2025-01-20 22:53:02,358 INFO autoscaler.py:280 -- disable_node_updaters:False |
|
2025-01-20 22:53:02,358 INFO autoscaler.py:288 -- disable_launch_config_check:True |
|
2025-01-20 22:53:02,358 INFO autoscaler.py:300 -- foreground_node_launch:False |
|
2025-01-20 22:53:02,358 INFO autoscaler.py:310 -- worker_liveness_check:True |
|
2025-01-20 22:53:02,358 INFO autoscaler.py:318 -- worker_rpc_drain:True |
|
2025-01-20 22:53:02,359 INFO autoscaler.py:368 -- StandardAutoscaler: {'cluster_name': 'default', 'max_workers': 0, 'upscaling_speed': 1.0, 'docker': {}, 'idle_timeout_minutes': 0, 'provider': {'type': 'readonly', 'use_node_id_as_ip': True, 'disable_launch_config_check': True}, 'auth': {}, 'available_node_types': {'ray.head.default': {'resources': {}, 'node_config': {}, 'max_workers': 0}}, 'head_node_type': 'ray.head.default', 'file_mounts': {}, 'cluster_synced_files': [], 'file_mounts_sync_continuously': False, 'rsync_exclude': [], 'rsync_filter': [], 'initialization_commands': [], 'setup_commands': [], 'head_setup_commands': [], 'worker_setup_commands': [], 'head_start_ray_commands': [], 'worker_start_ray_commands': []} |
|
2025-01-20 22:53:02,363 INFO monitor.py:383 -- Autoscaler has not yet received load metrics. Waiting. |
|
2025-01-20 22:53:07,375 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:07,376 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:07.375763 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:07,378 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:53:12,391 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:12,392 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:12.392555 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:12,395 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:53:17,406 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:17,407 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:17.407433 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:17,409 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:22,419 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:22,420 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:22.420219 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:22,422 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:27,435 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:27,435 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:27.435728 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:27,438 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:53:32,447 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:32,448 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:32.448230 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:32,450 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:37,462 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:37,463 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:37.463509 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:37,466 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:42,476 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:42,476 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:42.476608 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:42,479 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:47,489 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:47,490 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:47.490641 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:47,493 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:52,504 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:52,505 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:52.504892 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:52,507 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:53:57,517 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:53:57,518 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:53:57.518638 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:53:57,521 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:54:02,533 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:02,534 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:02.534050 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:02,535 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:54:07,547 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:07,547 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:07.547358 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:07,549 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:54:12,559 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:12,560 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:12.559854 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:12,562 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:54:17,574 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:17,575 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:17.574937 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:17,577 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:54:22,590 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:22,591 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:22.591444 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:22,594 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:54:27,604 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:27,605 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:27.605185 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:27,607 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:54:32,616 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:32,617 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:32.616810 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:32,619 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:54:37,631 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:37,631 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:37.631866 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
10.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:37,633 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:54:42,643 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:42,644 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:42.644290 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
8.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:42,646 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:54:47,655 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:47,655 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:47.655855 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
6.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:47,657 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:54:52,667 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:52,668 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:52.668538 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
6.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:52,671 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:54:57,683 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:54:57,684 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:54:57.683954 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
6.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:54:57,685 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:55:02,696 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:02,697 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:02.696844 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
5.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:02,699 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:55:07,711 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:07,712 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:07.712012 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
1.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:07,714 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:55:12,727 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:12,728 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:12.728006 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:12,730 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:17,741 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:17,742 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:17.741889 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:17,744 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:22,757 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:22,758 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:22.758128 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:22,761 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:27,772 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:27,773 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:27.772814 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:27,775 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:32,787 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:32,788 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:32.788058 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:32,790 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:55:37,802 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:37,803 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:37.803668 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:37,806 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:42,818 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:42,819 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:42.819418 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:42,821 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:55:47,832 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:47,833 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:47.833173 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:47,835 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:55:52,847 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:52,848 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:52.848534 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:52,850 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:55:57,862 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:55:57,863 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:55:57.862914 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:55:57,865 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:02,877 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:02,878 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:02.878594 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:02,881 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:07,891 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:07,892 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:07.892554 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:07,895 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:12,903 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:12,904 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:12.904600 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:12,907 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:17,918 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:17,919 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:17.919592 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:17,922 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:22,934 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:22,935 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:22.935343 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:22,937 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:27,950 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:27,951 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:27.950939 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:27,953 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:56:32,963 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:32,964 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:32.964658 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:32,967 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:56:37,979 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:37,980 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:37.980716 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:37,983 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:56:42,995 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:42,996 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:42.996065 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:42,998 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:48,010 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:48,011 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:48.011590 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:48,014 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:56:53,026 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:53,027 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:53.027256 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:53,029 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:56:58,042 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:56:58,043 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:56:58.043731 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:56:58,046 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:03,056 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:03,057 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:03.057319 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:03,060 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:08,072 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:08,073 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:08.073211 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:08,075 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:13,089 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:13,090 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:13.089823 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:13,092 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:18,102 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:18,103 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:18.103596 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:18,106 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:23,118 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:23,119 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:23.119470 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:23,121 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:28,133 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:28,134 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:28.134205 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:28,136 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:33,149 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:33,150 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:33.149985 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:33,152 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:38,164 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:38,165 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:38.165646 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:38,168 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:43,180 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:43,181 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:43.180798 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:43,183 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:48,195 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:48,196 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:48.196638 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:48,199 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:57:53,211 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:53,212 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:53.211919 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:53,213 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:57:58,223 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:57:58,224 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:57:58.224520 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:57:58,226 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:03,239 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:03,239 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:03.239756 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:03,242 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:08,254 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:08,255 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:08.255631 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:08,258 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:13,268 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:13,268 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:13.268761 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:13,271 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:18,284 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:18,285 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:18.285545 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:18,288 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:23,301 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:23,302 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:23.302686 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:23,305 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:28,317 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:28,318 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:28.318620 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:28,321 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:33,331 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:33,332 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:33.332751 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:33,335 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:58:38,343 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:38,344 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:38.343899 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:38,346 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:43,357 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:43,358 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:43.358026 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:43,360 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:48,372 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:48,373 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:48.373241 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:48,375 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:53,387 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:53,388 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:53.388201 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:53,390 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:58:58,401 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:58:58,402 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:58:58.401999 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:58:58,404 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:03,417 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:03,417 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:03.417510 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:03,419 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:59:08,430 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:08,431 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:08.431009 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:08,432 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:59:13,444 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:13,444 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:13.444595 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:13,446 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:59:18,455 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:18,456 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:18.456067 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:18,458 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:59:23,472 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:23,473 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:23.472905 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:23,475 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:28,488 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:28,489 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:28.488902 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:28,491 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:33,500 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:33,501 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:33.501479 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:33,504 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:38,517 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:38,518 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:38.518456 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:38,521 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:43,532 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:43,533 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:43.533021 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:43,536 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 22:59:48,548 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:48,548 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:48.548687 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:48,551 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 22:59:53,562 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:53,563 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:53.562997 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:53,565 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 22:59:58,577 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 22:59:58,577 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 22:59:58.577734 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 22:59:58,580 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:03,590 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:03,590 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:03.590797 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:03,592 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 23:00:08,599 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:08,599 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:08.599583 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:08,601 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:00:13,614 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:13,615 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:13.615126 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:13,618 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:18,631 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:18,632 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:18.631932 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:18,634 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:00:23,644 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:23,644 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:23.644662 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:23,647 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:00:28,660 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:28,661 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:28.661049 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:28,663 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:33,677 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:33,678 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:33.678356 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:33,681 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:38,694 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:38,695 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:38.695520 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:38,698 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:00:43,711 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:43,711 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:43.711769 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:43,714 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:48,727 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:48,728 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:48.728085 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:48,730 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:00:53,740 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:53,740 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:53.740737 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:53,743 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:00:58,755 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:00:58,756 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:00:58.756252 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:00:58,759 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
2025-01-20 23:01:03,771 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:03,772 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:03.772630 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:03,775 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:08,788 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:08,789 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:08.789123 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:08,791 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:13,803 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:13,803 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:13.803888 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:13,806 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:18,817 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:18,817 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:18.817750 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:18,820 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:23,831 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:23,831 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:23.831469 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:23,833 INFO autoscaler.py:470 -- The autoscaler took 0.002 seconds to complete the update iteration. |
|
2025-01-20 23:01:28,845 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:28,846 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:28.845788 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:28,848 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:33,861 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:33,861 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:33.861760 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:33,864 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:38,874 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:38,875 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:38.875272 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:38,877 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:43,890 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:43,891 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:43.891060 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:43,893 INFO autoscaler.py:470 -- The autoscaler took 0.003 seconds to complete the update iteration. |
|
2025-01-20 23:01:48,906 INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes. |
|
2025-01-20 23:01:48,907 INFO autoscaler.py:427 -- |
|
======== Autoscaler status: 2025-01-20 23:01:48.907064 ======== |
|
Node status |
|
--------------------------------------------------------------- |
|
Active: |
|
1 node_13be7277f830f5a8b967d2a0092091c94c7576cfebf8a5fa66025fcf |
|
Pending: |
|
(no pending nodes) |
|
Recent failures: |
|
(no failures) |
|
|
|
Resources |
|
--------------------------------------------------------------- |
|
Usage: |
|
0.0/20.0 CPU |
|
0.0/2.0 GPU |
|
0B/78.69GiB memory |
|
0B/2.00GiB object_store_memory |
|
|
|
Demands: |
|
(no resource demands) |
|
2025-01-20 23:01:48,909 INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration. |
|
|
