prometheus监控k8s的metric详解（第二版）第三章 kubernetes-apiservers （第二部分）flowcontrol 流量控制（上）请求和请求的资源消耗

文章目录

• 1 requests
• • 1.1 requests
  • • `apiserver_flowcontrol_current_executing_requests`
    • `apiserver_flowcontrol_current_inqueue_requests`
    • `apiserver_flowcontrol_dispatched_requests_total`
  • 1.2 request execution（执行的请求）
  • • 1.2.1 `apiserver_flowcontrol_request_execution_seconds_count`
    • 1.2.2 `apiserver_flowcontrol_request_execution_seconds_bucket`
    • 1.2.3 `apiserver_flowcontrol_request_execution_seconds_sum`
  • 1.3 queue length（队列长度）
  • • `apiserver_flowcontrol_request_queue_length_after_enqueue_count`
    • `apiserver_flowcontrol_request_queue_length_after_enqueue_bucket`
    • `apiserver_flowcontrol_request_queue_length_after_enqueue_sum`
  • 1.4 request wait
  • • `apiserver_flowcontrol_request_wait_duration_seconds_count`
    • `apiserver_flowcontrol_request_wait_duration_seconds_bucket`
    • `apiserver_flowcontrol_request_wait_duration_seconds_sum`
  • 1.5 concurrency （并发）
  • • `apiserver_flowcontrol_request_concurrency_in_use`
    • `apiserver_flowcontrol_request_concurrency_limit`
• 2 S值和R值
• • 2.1 S值边界
  • • `apiserver_flowcontrol_next_s_bounds`
    • `apiserver_flowcontrol_next_discounted_s_bounds`
  • 资源消耗
  • • `apiserver_flowcontrol_current_r`
    • `apiserver_flowcontrol_dispatch_r`
    • `apiserver_flowcontrol_latest_s`

APIserver 流量控制器部分

1 requests

1.1 requests

apiserver_flowcontrol_current_executing_requests

• 作用：当前正在并发执行的请求数量，不包括等待中的请求
• 示例：找其中一个master节点

apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 1
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 0
apiserver_flowcontrol_current_executing_requests{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0

• 说明：
  • priority_level：请求所属的优先级级别（由高到低如下：）
    • system：系统关键请求（如节点心跳）
    • leader-election：领导者选举请求
    • workload-high：高优先级工作负载
    • workload-low：低优先级工作负载
    • global-default：默认优先级
    • catch-all：未匹配其他分类的请求
  • flow_schema：流模式分类，匹配请求的分类规则。
    • system-nodes→ system（节点相关请求）
    • system-leader-election→ leader-election（领导者选举）
    • kube-controller-manager→ workload-high（控制器管理器）
    • service-accounts→ workload-low（服务账户请求）
    • catch-all→ catch-all（默认捕获所有未分类请求）

apiserver_flowcontrol_current_inqueue_requests

• 作用：当前在队列中等待执行的请求数量

apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 0
apiserver_flowcontrol_current_inqueue_requests{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0

apiserver_flowcontrol_dispatched_requests_total

• 作用：从队列中调出并开始执行的请求总数
  • 注意，这是个积累值。
• 示例

apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 189
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="exempt", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 7826662
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1685
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 1102842
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 572296
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 21308252
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="probes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 9488552
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 11560126
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 22398691
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 16786870
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 16965770
apiserver_flowcontrol_dispatched_requests_total{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25083

• 说明
  • flow_schema 的值比之前多出了两个​
    • exempt: 豁免流模式，匹配完全免除流控限制的请求。
    • probes: 探针流模式，匹配就绪性和存活性探针请求。
    • system-node-high：节点高级操作请求
  • priority_level 的值比前边多了1个
    • exempt：不受并发限制和排队策略约束，其优先级最高
    • node-high：节点高级操作，优先级在system之后

1.2 request execution（执行的请求）

1.2.1 apiserver_flowcontrol_request_execution_seconds_count

• 作用：直方图计数器，表示已完成执行的请求总数

如果没有问题和 apiserver_flowcontrol_dispatched_requests_total（已调度的请求数） 的值应该是相等的

• 示例：

apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 189
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="exempt", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 7827416
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1685
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 1102967
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 572357
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 21310711
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="probes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 9489578
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 11561334
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 22401181
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 16787214
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 16966205
apiserver_flowcontrol_request_execution_seconds_count{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25083

1.2.2 apiserver_flowcontrol_request_execution_seconds_bucket

• 作用：请求执行时间的分布统计
  • le标签：定义"小于等于"的时间桶边界（单位：秒）。
• 示例：过滤flow_schema="system-nodes"的数据

apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="+Inf", priority_level="system"} 16966382
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0", priority_level="system"} 0
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.005", priority_level="system"} 16122069
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.02", priority_level="system"} 16963071
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.05", priority_level="system"} 16966254
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.1", priority_level="system"} 16966285
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.2", priority_level="system"} 16966312
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.5", priority_level="system"} 16966346
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="1", priority_level="system"} 16966369
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="10", priority_level="system"} 16966382
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="2", priority_level="system"} 16966375
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="30", priority_level="system"} 16966382
apiserver_flowcontrol_request_execution_seconds_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="5", priority_level="system"} 16966381

1.2.3 apiserver_flowcontrol_request_execution_seconds_sum

• 作用：直方图的总和，表示该类请求执行时间的累计总和（单位：秒）
• 示例

apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 1.493296991
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="exempt", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 15620.0811469824
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1.550149784999999
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 753.9678882400026
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 373.61963880199636
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 12114.61388403801
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="probes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="exempt"} 21284.580194949616
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 105321.59872943368
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 70248.44957117751
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 44821.80115281418
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 35522.4987695499
apiserver_flowcontrol_request_execution_seconds_sum{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 107.40221201900025

1.3 queue length（队列长度）

apiserver_flowcontrol_request_queue_length_after_enqueue_count

• 作用：成功入队的请求次数

  • 实际直译是：统计入队后（after_enqueue）队列长度（request_queue）这件事的计数（count）

• 示例：过滤其中一个节点，查询结果如下

apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1685
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 1103148
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 572435
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 21314028
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 11562994
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 22404610
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 16787683
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 16966802
apiserver_flowcontrol_request_queue_length_after_enqueue_count{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25083

apiserver_flowcontrol_request_queue_length_after_enqueue_bucket

• 作用：所有成功入队请求，在“入队完成之后”队列总长度这个数值的分布统计。
  • 值可理解为“次”
  • le 标签的值的单位是个
• 示例：

apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="+Inf", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0", priority_level="system"} 0
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="10", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="100", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="1000", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="25", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="250", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="50", priority_level="system"} 16967667
apiserver_flowcontrol_request_queue_length_after_enqueue_bucket{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="500", priority_level="system"} 16967667

• 说明：
  • 成功加入队列的请求，加入队列后，队列长度小于10（le=“10”）的情况，发生了16967667 次。
  • 成功加入队列的请求，加入队列后，队列长度小于25（le=“25”）的情况，发生了16967667 次。
  • 可推知：队列从未发生过超过 10 的排队。

apiserver_flowcontrol_request_queue_length_after_enqueue_sum

• 作用：所有成功入队请求，在“入队完成之后”队列总长度这个数值的和。
• 示例

apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1685
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 1103511
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 572613
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 21320657
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 11566346
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 22411538
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 16788634
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 16968035
apiserver_flowcontrol_request_queue_length_after_enqueue_sum{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25083

1.4 request wait

apiserver_flowcontrol_request_wait_duration_seconds_count

• 作用：最终被执行的等待的请求数量
  • execute=“true”：说明是最终被执行的
• 示例

apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 189
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 1685
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 1103552
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 572632
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 21321425
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 11566735
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 22412320
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 16788744
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 16968175
apiserver_flowcontrol_request_wait_duration_seconds_count{cluster="cto-k8s-pro", execute="true", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25083

apiserver_flowcontrol_request_wait_duration_seconds_bucket

• 作用：最终被执行的请求的等待时间的分布
  • execute=“true”：表示最终被执行
  • 值单位是次数（或者个）
  • 标签 le的值单位是 秒
• 示例

apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="+Inf", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0", priority_level="system"} 0
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.005", priority_level="system"} 14148402
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.02", priority_level="system"} 16908777
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.05", priority_level="system"} 16968277
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.1", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.2", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="0.5", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="1", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="10", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="2", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="30", priority_level="system"} 16968292
apiserver_flowcontrol_request_wait_duration_seconds_bucket{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", le="5", priority_level="system"} 16968292

• 说明：
• 等待时间在0.005秒以内的请求是 4148402个
• 等待时间在0.005~0.02 秒的请求是 16908777- 4148402 个
• 等待时间在0.02~0.05 秒的请求是 16968277-16908777 个……

apiserver_flowcontrol_request_wait_duration_seconds_sum

• 作用：最终被执行的请求的等待时间的和，单位为秒。
• 示例

apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 1.0789226650000006
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 3.608209941999999
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 9964.505176708028
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 5121.448756089104
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 62874.24528134557
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 42837.242703885866
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 68624.74396176587
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 45584.89044591805
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 39703.64033399948
apiserver_flowcontrol_request_wait_duration_seconds_sum{cluster="cto-k8s-pro", execute="true", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 107.73069220199991

1.5 concurrency （并发）

apiserver_flowcontrol_request_concurrency_in_use

• 作用：正在执行的请求数
• 示例：

apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="catch-all", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="catch-all"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="global-default", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="kube-controller-manager", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="kube-scheduler", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="kube-system-service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="service-accounts", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 1
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="system-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="system-node-high", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="system-nodes", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 0
apiserver_flowcontrol_request_concurrency_in_use{cluster="cto-k8s-pro", flow_schema="workload-leader-election", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 0

apiserver_flowcontrol_request_concurrency_limit

• 作用：不同 优先级级别（Priority Level） 设置的 并发请求限制
• 示例：

apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="catch-all"} 13
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="global-default"} 49
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="leader-election"} 25
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="node-high"} 98
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="system"} 74
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-high"} 98
apiserver_flowcontrol_request_concurrency_limit{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-low"} 245

2 S值和R值

• S 值
  • APF 公平排队的虚拟时间/调度进度值。
  • 用来给请求打“服务顺序时间戳”
  • S 越小的队列/请求越先被调度
  • 本质表示“这个队列已经获得了多少服务”
• R 值
  • R 是请求的服务量/成本
  • 调度时会用 R 推进 S值

2.1 S值边界

apiserver_flowcontrol_next_s_bounds

• 作用：下一次调度计算将要使用的 S 的边界值
  • bound 标签有 max、min两个值
• 示例

apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 6.168791
apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 100380.2901747
apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 90320.48018628
apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 74743.81635768
apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 41325.08533074
apiserver_flowcontrol_next_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 149616.61333626
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 6.168791
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 100380.2901747
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 90320.48018628
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 74743.81635768
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 41325.08533074
apiserver_flowcontrol_next_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 149616.61333626

apiserver_flowcontrol_next_discounted_s_bounds

• 作用：经过衰减的 S值
• 示例：

apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 6.168791
apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 100370.24949839
apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 90317.33573556
apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 74740.77586256
apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 41321.48720482
apiserver_flowcontrol_next_discounted_s_bounds{bound="max", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 149601.66076184
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="global-default"} 6.168791
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="leader-election"} 100370.24949839
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="node-high"} 90317.33573556
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="system"} 74740.77586256
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-high"} 41321.48720482
apiserver_flowcontrol_next_discounted_s_bounds{bound="min", cluster="cto-k8s-pro", instance="10.xxx.182.60:6443", job="kubernetes-apiservers", priority_level="workload-low"} 149601.66076184

资源消耗

apiserver_flowcontrol_current_r

• 作用：该 PriorityLevel 当前的全局 R 基线
  • 它的R值不是绝对的“累积”
    • 会发生“基准对齐”
    • “对齐”时它会增加。（因此它会大于其他两个值）
• 示例：

apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="catch-all"} 0
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="global-default"} 0.22017623
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="leader-election"} 2145.12232076
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="node-high"} 11029.56525809
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="system"} 8103.10870958
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-high"} 940.50135381
apiserver_flowcontrol_current_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-low"} 46025.6234504

apiserver_flowcontrol_dispatch_r

• 作用：这次请求被调度时算出的 R
• 示例：

apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="global-default"} 0.18509813
apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="leader-election"} 2145.11651634
apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="node-high"} 11046.65599282
apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="system"} 8115.12778822
apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-high"} 941.67064405
apiserver_flowcontrol_dispatch_r{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-low"} 46038.78653828

apiserver_flowcontrol_latest_s

• 作用：本次（即最新一次 latest）调度用的虚拟时间 S
  • 调度器用来判断“本次谁先跑”的虚拟时间
• 示例：

apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="global-default"} 0.18509813
apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="leader-election"} 2145.11651634
apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="node-high"} 11049.20608306
apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="system"} 8116.75154908
apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-high"} 941.8998394
apiserver_flowcontrol_latest_s{cluster="cto-k8s-pro", instance="10.xxx.182.61:6443", job="kubernetes-apiservers", priority_level="workload-low"} 46040.55993005

示例：说明三者的变化和关系

• t0 初始状态
  • 系统刚启动，PriorityLevel 的全局 R 基线 current_r = 0。
  • 队列为空，没有请求被调度，所以 latest_s 和 dispatch_r 都不存在。
  • 没有调度事件发生，一切等待请求进入。
• t1 请求1 dispatch
  • 请求1进入队列，调度器开始计算它的虚拟时间 S（latest_s），得到 latest_s = 10。
  • 调度器判断“谁先跑”，发现请求1是下一个要调度的请求，于是dispatch它。
  • 在 dispatch 的瞬间，调度器计算 dispatch_r = 10（这个请求当前的 R 值）。
  • PriorityLevel 的全局 R 基线 current_r 被更新为 dispatch_r，即 current_r = 10，记录这次 dispatch 的位置。
• t2 baseline alignment（队列里还有请求触发对齐）
  • 请求队列中还有其他请求未调度，调度器为了保证公平，会检查并对齐当前 PriorityLevel 的 R 基线。
  • 调度器把 current_r 从 10 提高到 12，保证未调度请求的 R 不会落后太多，实现公平调度。
  • 这一步没有新的请求 dispatch，所以 latest_s 和 dispatch_r 不变/未产生。
• t3 请求2 dispatch
  • 调度器开始调度请求2，计算它的虚拟时间 latest_s = 12，用于判断谁先跑。
  • 请求2被 dispatch，计算出它的 dispatch_r = 12。
  • PriorityLevel 的全局 R 基线 current_r 更新为 dispatch_r 或对齐后的值，这里两者相等，因此 current_r = 12。