Scaling and Autoscaling

Nantian Gateway supports Horizontal Pod Autoscaling (HPA) for the data plane, control plane, and dashboard. When enabled, the Helm chart creates autoscaling/v2 HorizontalPodAutoscaler resources targeting each component.

How it works

When HPA is enabled, the chart creates a separate HPA resource for each enabled component. Each HPA watches CPU and memory utilization and adjusts replicas to match the configured targets.

Component	Scale behavior
Data plane	Stateless — adding replicas increases request capacity without coordination. Each replica receives its full routing snapshot from the control plane over gRPC/xDS.
Control plane	The active leader performs reconciliation. Standby replicas provide high availability. HPA ensures enough standbys are available during leader transitions.
Dashboard	Stateless Next.js application. HPA scales dashboard instances based on admin UI traffic.

Prerequisites

Resource requests on each component’s container are required for HPA to function. The Kubernetes metrics pipeline uses container resource requests to calculate utilization percentages.

The chart ships default resource requests for each component. Adjust these values to match your workload before enabling HPA.

Configuration values

Value	Default	Description
`hpa.enabled`	`false`	Enable HPA for all components
`hpa.targetCPUUtilization`	`70`	Target CPU utilization percentage
`hpa.targetMemoryUtilization`	`75`	Target memory utilization percentage
`hpa.dataplane.minReplicas`	`2`	Minimum data plane replicas
`hpa.dataplane.maxReplicas`	`10`	Maximum data plane replicas
`hpa.controlplane.minReplicas`	`2`	Minimum control plane replicas
`hpa.controlplane.maxReplicas`	`5`	Maximum control plane replicas
`hpa.dashboard.minReplicas`	`2`	Minimum dashboard replicas
`hpa.dashboard.maxReplicas`	`5`	Maximum dashboard replicas

Enabling HPA

Set hpa.enabled to true in your values override. A typical configuration looks like this:

hpa:
  enabled: true
  targetCPUUtilization: 70
  targetMemoryUtilization: 75
  dataplane:
    minReplicas: 2
    maxReplicas: 10
  controlplane:
    minReplicas: 2
    maxReplicas: 5
  dashboard:
    minReplicas: 2
    maxReplicas: 5

Apply the chart with your overrides:

helm upgrade nantian-gw nantian/nantian-gw \
  --namespace nantian-gw \
  --values hpa-values.yaml

Control plane scaling

The control plane runs with Kubernetes leader election. Only the elected leader performs reconciliation and publishes xDS snapshots. Standby instances wait for the leader lease and take over if the leader fails.

Enabling HPA for the control plane ensures sufficient standby replicas are available for high availability. When the leader fails, a standby can immediately take over without waiting for a new pod to start.

Checking HPA status

After enabling HPA, confirm the HPA resources are created and tracking metrics:

# List all HPA resources
kubectl get hpa -n nantian-gw

# Inspect individual HPA details
kubectl describe hpa nantian-gw-dataplane -n nantian-gw
kubectl describe hpa nantian-gw-controlplane -n nantian-gw
kubectl describe hpa nantian-gw-dashboard -n nantian-gw

Look for these signals in the describe output:

AbleToScale: The HPA can fetch metrics and make scaling decisions.
ScalingActive: The HPA is active and has selected a target.
ScalingLimited: The current replica count is at the minimum or maximum boundary.

If metrics show <unknown> for CPU or memory, check that the metrics-server is running in your cluster and that data plane pods have resource requests configured.

# Verify metrics-server is available
kubectl get deployment metrics-server -n kube-system

# Confirm data plane pods report resource usage
kubectl top pods -n nantian-gw -l app.kubernetes.io/component=dataplane

Scaling beyond CPU and memory

Vertical Pod Autoscaling (VPA)

For workloads where right-sizing is more important than horizontal scaling, consider Vertical Pod Autoscaler. VPA adjusts CPU and memory requests based on historical usage patterns:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: nantian-gw-dataplane
  namespace: nantian-gw
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nantian-gw-dataplane
  updatePolicy:
    updateMode: "Auto"

KEDA (Event-Driven Autoscaling)

For event-driven scaling — scaling the data plane based on request queue depth, Prometheus metrics, or external event sources — use KEDA:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: nantian-gw-dataplane-prometheus
  namespace: nantian-gw
spec:
  scaleTargetRef:
    name: nantian-gw-dataplane
  minReplicaCount: 2
  maxReplicaCount: 20
  triggers:
  - type: prometheus
    metadata:
      serverAddress: http://prometheus-operated.monitoring:9090
      metricName: http_requests_per_second
      query: rate(nantian_gw_dataplane_http_requests_total[1m])
      threshold: "5000"

This scales the data plane based on actual request rate rather than CPU utilization, which is more responsive for traffic spikes.

Custom Metrics Scaling

HPA supports custom and external metrics when a Prometheus adapter is installed:

helm install prometheus-adapter prometheus-community/prometheus-adapter \
  --namespace monitoring

Then configure HPA with a custom metric:

hpa:
  enabled: true
  dataplane:
    minReplicas: 2
    maxReplicas: 20
  customMetrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "5000"

Scaling benchmarks

The following table provides approximate throughput per data plane replica under typical conditions (TLS-terminated HTTP/1.1, 1KB response body, keepalive connections):

Replicas	Approximate RPS	P99 Latency	CPU per replica
1	8,000	15ms	~1.1 cores
2	16,000	18ms	~1.1 cores
4	32,000	22ms	~1.0 cores
8	60,000	28ms	~1.0 cores

These numbers are from the project’s nightly performance suite (platform-release/results/nightly/). Actual throughput depends on TLS configuration, request/response sizes, backend latency, and CPU architecture.

Troubleshooting HPA

If replicas are not scaling as expected:

Symptom	Likely Cause	Fix
HPA shows `<unknown>` metrics	metrics-server not running	`kubectl get deployment metrics-server -n kube-system`
HPA target is 0%	Pods not receiving traffic	Verify service selectors and network policies
HPA never scales up	`maxReplicas` too low	Increase maxReplicas in Helm values
HPA oscillates (flapping)	Threshold too aggressive	Raise `targetCPUUtilization` or increase `minReplicas`