| Gcs Debug state: | |
| GcsNodeManager: | |
| - RegisterNode request count: 1 | |
| - DrainNode request count: 0 | |
| - GetAllNodeInfo request count: 25 | |
| GcsActorManager: | |
| - RegisterActor request count: 0 | |
| - CreateActor request count: 0 | |
| - GetActorInfo request count: 0 | |
| - GetNamedActorInfo request count: 0 | |
| - GetAllActorInfo request count: 0 | |
| - KillActor request count: 0 | |
| - ListNamedActors request count: 0 | |
| - Registered actors count: 0 | |
| - Destroyed actors count: 0 | |
| - Named actors count: 0 | |
| - Unresolved actors count: 0 | |
| - Pending actors count: 0 | |
| - Created actors count: 0 | |
| - owners_: 0 | |
| - actor_to_register_callbacks_: 0 | |
| - actor_to_restart_callbacks_: 0 | |
| - actor_to_create_callbacks_: 0 | |
| - sorted_destroyed_actor_list_: 0 | |
| GcsResourceManager: | |
| - GetAllAvailableResources request count: 0 | |
| - GetAllTotalResources request count: 0 | |
| - GetAllResourceUsage request count: 1876 | |
| GcsPlacementGroupManager: | |
| - CreatePlacementGroup request count: 0 | |
| - RemovePlacementGroup request count: 0 | |
| - GetPlacementGroup request count: 0 | |
| - GetAllPlacementGroup request count: 0 | |
| - WaitPlacementGroupUntilReady request count: 0 | |
| - GetNamedPlacementGroup request count: 0 | |
| - Scheduling pending placement group count: 0 | |
| - Registered placement groups count: 0 | |
| - Named placement group count: 0 | |
| - Pending placement groups count: 0 | |
| - Infeasible placement groups count: 0 | |
| Publisher: | |
| GCS_NODE_INFO_CHANNEL | |
| - cumulative published messages: 1 | |
| - cumulative published bytes: 536 | |
| - current buffered bytes: 0 | |
| RAY_LOG_CHANNEL | |
| - cumulative published messages: 1945 | |
| - cumulative published bytes: 1548626 | |
| - current buffered bytes: 819 | |
| GCS_JOB_CHANNEL | |
| - cumulative published messages: 1 | |
| - cumulative published bytes: 257 | |
| - current buffered bytes: 257 | |
| [runtime env manager] ID to URIs table: | |
| [runtime env manager] URIs reference table: | |
| GcsTaskManager: | |
| -Total num task events reported: 53 | |
| -Total num status task events dropped: 0 | |
| -Total num profile events dropped: 0 | |
| -Current num of task events stored: 15 | |
| -Total num of actor creation tasks: 0 | |
| -Total num of actor tasks: 0 | |
| -Total num of normal tasks: 14 | |
| -Total num of driver tasks: 1 | |
| GcsAutoscalerStateManager: | |
| - last_seen_autoscaler_state_version_: 0 | |
| - last_cluster_resource_state_version_: 0 | |
| - pending demands: | |
| Global stats: 112984 total (4 active) | |
| Queueing time: mean = 57.833 us, max = 225.583 ms, min = -0.023 s, total = 6.534 s | |
| Execution time: mean = 226.779 us, total = 25.622 s | |
| Event stats: | |
| event_loop_lag_probe - 37591 total (0 active), Execution time: mean = 25.515 us, total = 959.127 ms, Queueing time: mean = 9.811 us, max = 6.895 ms, min = 1.268 us, total = 368.819 ms | |
| GcsInMemoryStore.Put - 15079 total (0 active), Execution time: mean = 20.423 us, total = 307.959 ms, Queueing time: mean = 148.163 us, max = 225.278 ms, min = 1.495 us, total = 2.234 s | |
| RayletLoadPulled - 9399 total (1 active), Execution time: mean = 245.031 us, total = 2.303 s, Queueing time: mean = 74.310 us, max = 3.337 ms, min = 10.621 us, total = 698.439 ms | |
| NodeManagerService.grpc_client.GetResourceLoad.OnReplyReceived - 9397 total (0 active), Execution time: mean = 75.135 us, total = 706.043 ms, Queueing time: mean = 100.337 us, max = 15.090 ms, min = 11.188 us, total = 942.870 ms | |
| NodeManagerService.grpc_client.GetResourceLoad - 9397 total (0 active), Execution time: mean = 1.381 ms, total = 12.974 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut - 3767 total (0 active), Execution time: mean = 495.101 us, total = 1.865 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut.HandleRequestImpl - 3767 total (0 active), Execution time: mean = 39.550 us, total = 148.985 ms, Queueing time: mean = 91.016 us, max = 601.063 us, min = 9.911 us, total = 342.857 ms | |
| NodeInfoGcsService.grpc_server.CheckAlive.HandleRequestImpl - 3758 total (0 active), Execution time: mean = 35.093 us, total = 131.881 ms, Queueing time: mean = 105.191 us, max = 1.496 ms, min = 13.520 us, total = 395.307 ms | |
| NodeInfoGcsService.grpc_server.CheckAlive - 3758 total (0 active), Execution time: mean = 515.449 us, total = 1.937 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| ClusterResourceManager.ResetRemoteNodeView - 3134 total (1 active), Execution time: mean = 10.071 us, total = 31.564 ms, Queueing time: mean = 74.590 us, max = 1.648 ms, min = 7.259 us, total = 233.764 ms | |
| HealthCheck - 3131 total (0 active), Execution time: mean = 7.650 us, total = 23.952 ms, Queueing time: mean = 91.251 us, max = 1.139 ms, min = 11.335 us, total = 285.708 ms | |
| GcsInMemoryStore.Get - 1936 total (0 active), Execution time: mean = 22.545 us, total = 43.647 ms, Queueing time: mean = 12.438 us, max = 137.899 us, min = 2.227 us, total = 24.081 ms | |
| InternalKVGcsService.grpc_server.InternalKVGet - 1935 total (0 active), Execution time: mean = 564.908 us, total = 1.093 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVGet.HandleRequestImpl - 1935 total (0 active), Execution time: mean = 54.816 us, total = 106.068 ms, Queueing time: mean = 128.343 us, max = 22.982 ms, min = 9.184 us, total = 248.344 ms | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage - 1876 total (0 active), Execution time: mean = 615.451 us, total = 1.155 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage.HandleRequestImpl - 1876 total (0 active), Execution time: mean = 90.748 us, total = 170.244 ms, Queueing time: mean = 104.800 us, max = 273.843 us, min = 15.511 us, total = 196.604 ms | |
| GCSServer.deadline_timer.debug_state_dump - 940 total (1 active, 1 running), Execution time: mean = 1.367 ms, total = 1.285 s, Queueing time: mean = 43.693 us, max = 214.701 us, min = -0.023 s, total = 41.072 ms | |
| GCSServer.deadline_timer.debug_state_event_stats_print - 157 total (1 active), Execution time: mean = 1.530 ms, total = 240.263 ms, Queueing time: mean = 59.748 us, max = 238.837 us, min = 13.943 us, total = 9.380 ms | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo - 25 total (0 active), Execution time: mean = 3.222 ms, total = 80.540 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo.HandleRequestImpl - 25 total (0 active), Execution time: mean = 976.134 us, total = 24.403 ms, Queueing time: mean = 1.887 ms, max = 22.652 ms, min = 13.842 us, total = 47.163 ms | |
| GcsResourceManager::Update - 22 total (0 active), Execution time: mean = 123.575 us, total = 2.719 ms, Queueing time: mean = 99.677 us, max = 175.233 us, min = 20.234 us, total = 2.193 ms | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo - 21 total (0 active), Execution time: mean = 488.802 us, total = 10.265 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo.HandleRequestImpl - 21 total (0 active), Execution time: mean = 27.527 us, total = 578.066 us, Queueing time: mean = 119.221 us, max = 990.002 us, min = 13.763 us, total = 2.504 ms | |
| GcsInMemoryStore.GetAll - 6 total (0 active), Execution time: mean = 8.811 us, total = 52.865 us, Queueing time: mean = 33.528 us, max = 40.516 us, min = 10.559 us, total = 201.166 us | |
| NodeInfoGcsService.grpc_server.GetClusterId.HandleRequestImpl - 5 total (0 active), Execution time: mean = 801.096 us, total = 4.005 ms, Queueing time: mean = 170.426 us, max = 455.845 us, min = 90.434 us, total = 852.129 us | |
| NodeInfoGcsService.grpc_server.GetClusterId - 5 total (0 active), Execution time: mean = 2.499 ms, total = 12.495 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| PeriodicalRunner.RunFnPeriodically - 4 total (0 active), Execution time: mean = 138.179 us, total = 552.714 us, Queueing time: mean = 114.822 ms, max = 225.583 ms, min = 3.931 ms, total = 459.287 ms | |
| InternalKVGcsService.grpc_server.InternalKVDel - 1 total (0 active), Execution time: mean = 669.950 us, total = 669.950 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsHealthCheckManager::AddNode - 1 total (0 active), Execution time: mean = 18.185 us, total = 18.185 us, Queueing time: mean = 727.000 ns, max = 727.000 ns, min = 727.000 ns, total = 727.000 ns | |
| JobInfoGcsService.grpc_server.GetAllJobInfo - 1 total (0 active), Execution time: mean = 493.580 us, total = 493.580 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.Delete - 1 total (0 active), Execution time: mean = 16.907 us, total = 16.907 us, Queueing time: mean = 10.800 us, max = 10.800 us, min = 10.800 us, total = 10.800 us | |
| NodeInfoGcsService.grpc_server.RegisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 176.392 us, total = 176.392 us, Queueing time: mean = 87.087 us, max = 87.087 us, min = 87.087 us, total = 87.087 us | |
| InternalKVGcsService.grpc_server.InternalKVExists - 1 total (0 active), Execution time: mean = 476.372 us, total = 476.372 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVExists.HandleRequestImpl - 1 total (0 active), Execution time: mean = 35.867 us, total = 35.867 us, Queueing time: mean = 42.367 us, max = 42.367 us, min = 42.367 us, total = 42.367 us | |
| GcsInMemoryStore.Exists - 1 total (0 active), Execution time: mean = 9.652 us, total = 9.652 us, Queueing time: mean = 7.678 us, max = 7.678 us, min = 7.678 us, total = 7.678 us | |
| JobInfoGcsService.grpc_server.AddJob.HandleRequestImpl - 1 total (0 active), Execution time: mean = 137.810 us, total = 137.810 us, Queueing time: mean = 45.765 us, max = 45.765 us, min = 45.765 us, total = 45.765 us | |
| InternalKVGcsService.grpc_server.InternalKVDel.HandleRequestImpl - 1 total (0 active), Execution time: mean = 77.259 us, total = 77.259 us, Queueing time: mean = 90.912 us, max = 90.912 us, min = 90.912 us, total = 90.912 us | |
| JobInfoGcsService.grpc_server.GetAllJobInfo.HandleRequestImpl - 1 total (0 active), Execution time: mean = 22.304 us, total = 22.304 us, Queueing time: mean = 90.923 us, max = 90.923 us, min = 90.923 us, total = 90.923 us | |
| InternalKVGcsService.grpc_server.GetInternalConfig - 1 total (0 active), Execution time: mean = 621.211 us, total = 621.211 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.AddJob - 1 total (0 active), Execution time: mean = 660.939 us, total = 660.939 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID - 1 total (0 active), Execution time: mean = 483.254 us, total = 483.254 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID.HandleRequestImpl - 1 total (0 active), Execution time: mean = 22.277 us, total = 22.277 us, Queueing time: mean = 96.823 us, max = 96.823 us, min = 96.823 us, total = 96.823 us | |
| NodeInfoGcsService.grpc_server.RegisterNode - 1 total (0 active), Execution time: mean = 1.036 ms, total = 1.036 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.GetInternalConfig.HandleRequestImpl - 1 total (0 active), Execution time: mean = 43.799 us, total = 43.799 us, Queueing time: mean = 92.967 us, max = 92.967 us, min = 92.967 us, total = 92.967 us |