| Gcs Debug state: | |
| GcsNodeManager: | |
| - RegisterNode request count: 1 | |
| - DrainNode request count: 0 | |
| - GetAllNodeInfo request count: 34 | |
| GcsActorManager: | |
| - RegisterActor request count: 0 | |
| - CreateActor request count: 0 | |
| - GetActorInfo request count: 0 | |
| - GetNamedActorInfo request count: 0 | |
| - GetAllActorInfo request count: 0 | |
| - KillActor request count: 0 | |
| - ListNamedActors request count: 0 | |
| - Registered actors count: 0 | |
| - Destroyed actors count: 0 | |
| - Named actors count: 0 | |
| - Unresolved actors count: 0 | |
| - Pending actors count: 0 | |
| - Created actors count: 0 | |
| - owners_: 0 | |
| - actor_to_register_callbacks_: 0 | |
| - actor_to_restart_callbacks_: 0 | |
| - actor_to_create_callbacks_: 0 | |
| - sorted_destroyed_actor_list_: 0 | |
| GcsResourceManager: | |
| - GetAllAvailableResources request count: 0 | |
| - GetAllTotalResources request count: 0 | |
| - GetAllResourceUsage request count: 1734 | |
| GcsPlacementGroupManager: | |
| - CreatePlacementGroup request count: 0 | |
| - RemovePlacementGroup request count: 0 | |
| - GetPlacementGroup request count: 0 | |
| - GetAllPlacementGroup request count: 0 | |
| - WaitPlacementGroupUntilReady request count: 0 | |
| - GetNamedPlacementGroup request count: 0 | |
| - Scheduling pending placement group count: 0 | |
| - Registered placement groups count: 0 | |
| - Named placement group count: 0 | |
| - Pending placement groups count: 0 | |
| - Infeasible placement groups count: 0 | |
| Publisher: | |
| GCS_JOB_CHANNEL | |
| - cumulative published messages: 1 | |
| - cumulative published bytes: 256 | |
| - current buffered bytes: 256 | |
| RAY_LOG_CHANNEL | |
| - cumulative published messages: 1788 | |
| - cumulative published bytes: 1425363 | |
| - current buffered bytes: 819 | |
| GCS_NODE_INFO_CHANNEL | |
| - cumulative published messages: 1 | |
| - cumulative published bytes: 534 | |
| - current buffered bytes: 0 | |
| [runtime env manager] ID to URIs table: | |
| [runtime env manager] URIs reference table: | |
| GcsTaskManager: | |
| -Total num task events reported: 41 | |
| -Total num status task events dropped: 0 | |
| -Total num profile events dropped: 0 | |
| -Current num of task events stored: 11 | |
| -Total num of actor creation tasks: 0 | |
| -Total num of actor tasks: 0 | |
| -Total num of normal tasks: 10 | |
| -Total num of driver tasks: 1 | |
| GcsAutoscalerStateManager: | |
| - last_seen_autoscaler_state_version_: 0 | |
| - last_cluster_resource_state_version_: 0 | |
| - pending demands: | |
| Global stats: 104509 total (4 active) | |
| Queueing time: mean = 65.580 us, max = 279.303 ms, min = -0.000 s, total = 6.854 s | |
| Execution time: mean = 244.751 us, total = 25.579 s | |
| Event stats: | |
| event_loop_lag_probe - 34752 total (0 active), Execution time: mean = 25.460 us, total = 884.770 ms, Queueing time: mean = 9.992 us, max = 13.097 ms, min = 1.785 us, total = 347.236 ms | |
| GcsInMemoryStore.Put - 13942 total (0 active), Execution time: mean = 25.490 us, total = 355.380 ms, Queueing time: mean = 186.436 us, max = 278.545 ms, min = 1.649 us, total = 2.599 s | |
| RayletLoadPulled - 8689 total (1 active), Execution time: mean = 248.728 us, total = 2.161 s, Queueing time: mean = 75.691 us, max = 472.090 us, min = -0.000 s, total = 657.677 ms | |
| NodeManagerService.grpc_client.GetResourceLoad.OnReplyReceived - 8686 total (0 active), Execution time: mean = 77.320 us, total = 671.605 ms, Queueing time: mean = 102.781 us, max = 2.744 ms, min = 11.645 us, total = 892.754 ms | |
| NodeManagerService.grpc_client.GetResourceLoad - 8686 total (0 active), Execution time: mean = 1.498 ms, total = 13.012 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVPut.HandleRequestImpl - 3482 total (0 active), Execution time: mean = 43.314 us, total = 150.818 ms, Queueing time: mean = 103.863 us, max = 479.211 us, min = 11.443 us, total = 361.650 ms | |
| InternalKVGcsService.grpc_server.InternalKVPut - 3482 total (0 active), Execution time: mean = 569.112 us, total = 1.982 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.CheckAlive.HandleRequestImpl - 3475 total (0 active), Execution time: mean = 35.900 us, total = 124.754 ms, Queueing time: mean = 117.215 us, max = 1.498 ms, min = 20.329 us, total = 407.321 ms | |
| NodeInfoGcsService.grpc_server.CheckAlive - 3475 total (0 active), Execution time: mean = 592.931 us, total = 2.060 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| ClusterResourceManager.ResetRemoteNodeView - 2898 total (1 active), Execution time: mean = 9.587 us, total = 27.782 ms, Queueing time: mean = 75.922 us, max = 391.368 us, min = 10.687 us, total = 220.022 ms | |
| HealthCheck - 2894 total (0 active), Execution time: mean = 7.742 us, total = 22.407 ms, Queueing time: mean = 103.532 us, max = 1.431 ms, min = 16.806 us, total = 299.623 ms | |
| GcsInMemoryStore.Get - 1803 total (0 active), Execution time: mean = 20.822 us, total = 37.542 ms, Queueing time: mean = 13.378 us, max = 128.508 us, min = 3.701 us, total = 24.120 ms | |
| InternalKVGcsService.grpc_server.InternalKVGet.HandleRequestImpl - 1802 total (0 active), Execution time: mean = 58.316 us, total = 105.085 ms, Queueing time: mean = 104.719 us, max = 269.544 us, min = 13.461 us, total = 188.703 ms | |
| InternalKVGcsService.grpc_server.InternalKVGet - 1802 total (0 active), Execution time: mean = 612.791 us, total = 1.104 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage.HandleRequestImpl - 1734 total (0 active), Execution time: mean = 93.589 us, total = 162.283 ms, Queueing time: mean = 117.293 us, max = 296.469 us, min = 14.122 us, total = 203.387 ms | |
| NodeResourceInfoGcsService.grpc_server.GetAllResourceUsage - 1734 total (0 active), Execution time: mean = 684.957 us, total = 1.188 s, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GCSServer.deadline_timer.debug_state_dump - 869 total (1 active, 1 running), Execution time: mean = 1.422 ms, total = 1.235 s, Queueing time: mean = 72.631 us, max = 323.875 us, min = -0.000 s, total = 63.117 ms | |
| GCSServer.deadline_timer.debug_state_event_stats_print - 145 total (1 active), Execution time: mean = 1.607 ms, total = 233.042 ms, Queueing time: mean = 64.913 us, max = 182.019 us, min = 12.907 us, total = 9.412 ms | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo - 34 total (0 active), Execution time: mean = 626.288 us, total = 21.294 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetAllNodeInfo.HandleRequestImpl - 34 total (0 active), Execution time: mean = 38.829 us, total = 1.320 ms, Queueing time: mean = 128.920 us, max = 189.981 us, min = 20.090 us, total = 4.383 ms | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo - 21 total (0 active), Execution time: mean = 639.802 us, total = 13.436 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| WorkerInfoGcsService.grpc_server.AddWorkerInfo.HandleRequestImpl - 21 total (0 active), Execution time: mean = 30.627 us, total = 643.161 us, Queueing time: mean = 132.369 us, max = 239.162 us, min = 26.759 us, total = 2.780 ms | |
| GcsResourceManager::Update - 12 total (0 active), Execution time: mean = 111.572 us, total = 1.339 ms, Queueing time: mean = 139.823 us, max = 191.304 us, min = 21.005 us, total = 1.678 ms | |
| GcsInMemoryStore.GetAll - 6 total (0 active), Execution time: mean = 16.446 us, total = 98.675 us, Queueing time: mean = 76.435 us, max = 101.411 us, min = 14.900 us, total = 458.608 us | |
| NodeInfoGcsService.grpc_server.GetClusterId - 5 total (0 active), Execution time: mean = 1.973 ms, total = 9.867 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.GetClusterId.HandleRequestImpl - 5 total (0 active), Execution time: mean = 857.228 us, total = 4.286 ms, Queueing time: mean = 337.070 us, max = 1.136 ms, min = 109.531 us, total = 1.685 ms | |
| PeriodicalRunner.RunFnPeriodically - 4 total (0 active), Execution time: mean = 329.116 us, total = 1.316 ms, Queueing time: mean = 141.901 ms, max = 279.303 ms, min = 4.231 ms, total = 567.605 ms | |
| GcsInMemoryStore.Delete - 1 total (0 active), Execution time: mean = 23.668 us, total = 23.668 us, Queueing time: mean = 13.279 us, max = 13.279 us, min = 13.279 us, total = 13.279 us | |
| NodeInfoGcsService.grpc_server.RegisterNode - 1 total (0 active), Execution time: mean = 1.281 ms, total = 1.281 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel - 1 total (0 active), Execution time: mean = 906.560 us, total = 906.560 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| GcsInMemoryStore.Exists - 1 total (0 active), Execution time: mean = 16.051 us, total = 16.051 us, Queueing time: mean = 10.229 us, max = 10.229 us, min = 10.229 us, total = 10.229 us | |
| GcsHealthCheckManager::AddNode - 1 total (0 active), Execution time: mean = 17.206 us, total = 17.206 us, Queueing time: mean = 807.000 ns, max = 807.000 ns, min = 807.000 ns, total = 807.000 ns | |
| JobInfoGcsService.grpc_server.AddJob - 1 total (0 active), Execution time: mean = 801.197 us, total = 801.197 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.GetInternalConfig - 1 total (0 active), Execution time: mean = 677.583 us, total = 677.583 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| InternalKVGcsService.grpc_server.InternalKVDel.HandleRequestImpl - 1 total (0 active), Execution time: mean = 79.707 us, total = 79.707 us, Queueing time: mean = 128.483 us, max = 128.483 us, min = 128.483 us, total = 128.483 us | |
| JobInfoGcsService.grpc_server.GetAllJobInfo - 1 total (0 active), Execution time: mean = 1.012 ms, total = 1.012 ms, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID - 1 total (0 active), Execution time: mean = 682.058 us, total = 682.058 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| JobInfoGcsService.grpc_server.GetNextJobID.HandleRequestImpl - 1 total (0 active), Execution time: mean = 24.558 us, total = 24.558 us, Queueing time: mean = 134.773 us, max = 134.773 us, min = 134.773 us, total = 134.773 us | |
| InternalKVGcsService.grpc_server.InternalKVExists.HandleRequestImpl - 1 total (0 active), Execution time: mean = 35.149 us, total = 35.149 us, Queueing time: mean = 48.922 us, max = 48.922 us, min = 48.922 us, total = 48.922 us | |
| JobInfoGcsService.grpc_server.GetAllJobInfo.HandleRequestImpl - 1 total (0 active), Execution time: mean = 58.080 us, total = 58.080 us, Queueing time: mean = 154.830 us, max = 154.830 us, min = 154.830 us, total = 154.830 us | |
| InternalKVGcsService.grpc_server.InternalKVExists - 1 total (0 active), Execution time: mean = 493.698 us, total = 493.698 us, Queueing time: mean = 0.000 s, max = -0.000 s, min = 9223372036.855 s, total = 0.000 s | |
| NodeInfoGcsService.grpc_server.RegisterNode.HandleRequestImpl - 1 total (0 active), Execution time: mean = 212.378 us, total = 212.378 us, Queueing time: mean = 124.333 us, max = 124.333 us, min = 124.333 us, total = 124.333 us | |
| JobInfoGcsService.grpc_server.AddJob.HandleRequestImpl - 1 total (0 active), Execution time: mean = 153.988 us, total = 153.988 us, Queueing time: mean = 131.439 us, max = 131.439 us, min = 131.439 us, total = 131.439 us | |
| InternalKVGcsService.grpc_server.GetInternalConfig.HandleRequestImpl - 1 total (0 active), Execution time: mean = 32.554 us, total = 32.554 us, Queueing time: mean = 44.202 us, max = 44.202 us, min = 44.202 us, total = 44.202 us |