2025-01-15 18:17:33,374	INFO monitor.py:688 -- Starting monitor using ray installation: /usr/local/lib/python3.10/dist-packages/ray/__init__.py
2025-01-15 18:17:33,374	INFO monitor.py:689 -- Ray version: 2.40.0
2025-01-15 18:17:33,374	INFO monitor.py:690 -- Ray commit: 22541c38dbef25286cd6d19f1c151bf4fd62f2ed
2025-01-15 18:17:33,374	INFO monitor.py:691 -- Monitor started with command: ['/usr/local/lib/python3.10/dist-packages/ray/autoscaler/_private/monitor.py', '--logs-dir=/tmp/ray/session_2025-01-15_18-17-32_502817_531332/logs', '--logging-rotate-bytes=536870912', '--logging-rotate-backup-count=5', '--gcs-address=192.168.0.2:44704', '--monitor-ip=192.168.0.2']
2025-01-15 18:17:33,392	INFO monitor.py:159 -- session_name: session_2025-01-15_18-17-32_502817_531332
2025-01-15 18:17:33,394	INFO monitor.py:191 -- Starting autoscaler metrics server on port 44217
2025-01-15 18:17:33,395	ERROR monitor.py:208 -- An exception occurred while starting the metrics server.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ray/autoscaler/_private/monitor.py", line 197, in __init__
    prometheus_client.start_http_server(
  File "/usr/local/lib/python3.10/dist-packages/prometheus_client/exposition.py", line 170, in start_wsgi_server
    httpd = make_server(addr, port, app, TmpServer, handler_class=_SilentHandler)
  File "/usr/lib/python3.10/wsgiref/simple_server.py", line 154, in make_server
    server = server_class((host, port), handler_class)
  File "/usr/lib/python3.10/socketserver.py", line 452, in __init__
    self.server_bind()
  File "/usr/lib/python3.10/wsgiref/simple_server.py", line 50, in server_bind
    HTTPServer.server_bind(self)
  File "/usr/lib/python3.10/http/server.py", line 137, in server_bind
    socketserver.TCPServer.server_bind(self)
  File "/usr/lib/python3.10/socketserver.py", line 466, in server_bind
    self.socket.bind(self.server_address)
OSError: [Errno 98] Address already in use
2025-01-15 18:17:33,397	INFO monitor.py:216 -- Monitor: Started
2025-01-15 18:17:33,414	INFO autoscaler.py:280 -- disable_node_updaters:False
2025-01-15 18:17:33,414	INFO autoscaler.py:288 -- disable_launch_config_check:True
2025-01-15 18:17:33,414	INFO autoscaler.py:300 -- foreground_node_launch:False
2025-01-15 18:17:33,414	INFO autoscaler.py:310 -- worker_liveness_check:True
2025-01-15 18:17:33,414	INFO autoscaler.py:318 -- worker_rpc_drain:True
2025-01-15 18:17:33,415	INFO autoscaler.py:368 -- StandardAutoscaler: {'cluster_name': 'default', 'max_workers': 0, 'upscaling_speed': 1.0, 'docker': {}, 'idle_timeout_minutes': 0, 'provider': {'type': 'readonly', 'use_node_id_as_ip': True, 'disable_launch_config_check': True}, 'auth': {}, 'available_node_types': {'ray.head.default': {'resources': {}, 'node_config': {}, 'max_workers': 0}}, 'head_node_type': 'ray.head.default', 'file_mounts': {}, 'cluster_synced_files': [], 'file_mounts_sync_continuously': False, 'rsync_exclude': [], 'rsync_filter': [], 'initialization_commands': [], 'setup_commands': [], 'head_setup_commands': [], 'worker_setup_commands': [], 'head_start_ray_commands': [], 'worker_start_ray_commands': []}
2025-01-15 18:17:33,417	INFO monitor.py:383 -- Autoscaler has not yet received load metrics. Waiting.
2025-01-15 18:17:38,428	INFO autoscaler.py:147 -- The autoscaler took 0.0 seconds to fetch the list of non-terminated nodes.
2025-01-15 18:17:38,429	INFO autoscaler.py:427 -- 
======== Autoscaler status: 2025-01-15 18:17:38.429693 ========
Node status
---------------------------------------------------------------
Active:
 1 node_8d847287a9164ff537a79b3daadceaf6ee68d7e24360ab7ec6517c1c
Pending:
 (no pending nodes)
Recent failures:
 (no failures)

Resources
---------------------------------------------------------------
Usage:
 20.0/20.0 CPU
 2.0/2.0 GPU
 0B/79.74GiB memory
 0B/2.00GiB object_store_memory

Demands:
 {'CPU': 20.0, 'GPU': 2.0}: 3+ pending tasks/actors
2025-01-15 18:17:38,432	INFO autoscaler.py:470 -- The autoscaler took 0.004 seconds to complete the update iteration.