What is Nantian Gateway

Nantian Gateway is a Kubernetes Gateway API implementation with a built-in AI gateway.

Unlike other gateway projects that require you to deploy a separate AI proxy (LiteLLM, Portkey, etc.) alongside your ingress, Nantian Gateway handles AI provider routing, token counting, rate limiting, prompt guarding, and cost tracking within the same proxy that handles your HTTP traffic — no extra hop, no additional operational complexity.

# One install. One proxy. Gateway + AI gateway.
helm repo add nantian-gw https://chart.nantian.dev
helm install nantian-gw nantian-gw/nantian-gw --namespace nantian-gw --create-namespace

Why

Teams that need AI gateway capabilities today have two options:

1. Use a standalone AI proxy (LiteLLM, Portkey, AI Gateway) — adds a hop, doubles the infrastructure to manage, and introduces a separate auth/observability surface.
2. Use a Kubernetes Gateway (Istio, Envoy Gateway, Contour) — no AI features at all. You still need the standalone AI proxy.

Nantian Gateway is the only option that combines both in one process:

• Gateway API routing for your regular HTTP/gRPC/TCP/UDP traffic
• AI Gateway features for your LLM traffic — model routing, semantic cache, PII masking, content safety, A/B testing, fallback, cost tracking, Langfuse observability
• Wasm extensibility for custom filters — write in Rust, deploy via CRD

All in one Helm install, one control plane, one data plane binary.

Quick Example

# 1. Install the gateway stack
helm repo add nantian-gw https://chart.nantian.dev
helm install nantian-gw nantian-gw/nantian-gw \
  --namespace nantian-gw --create-namespace

# 2. Create a Gateway (entry point)
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
  namespace: default
spec:
  gatewayClassName: nantian-gw
  listeners:
  - name: http
    port: 80
    protocol: HTTP
EOF

# 3. Create an HTTPRoute (routing rule)
kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: my-route
  namespace: default
spec:
  parentRefs:
  - name: my-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api
    backendRefs:
    - name: my-backend-service
      port: 8080
EOF

Traffic to /api is now routed to my-backend-service:8080. No custom CRDs, no proprietary config language.

Architecture

Nantian Gateway is a split-plane gateway stack:

Gateway API resources -> Go control plane -> internal snapshot -> gRPC/xDS -> Rust data plane -> backends

The Go control plane watches Kubernetes resources, translates Gateway API and Nantian extension resources into internal routing state, reports status, and publishes runtime snapshots over gRPC/xDS. The Rust data plane receives those snapshots and handles live traffic — HTTP, gRPC, TCP, UDP, TLS, and AI provider requests.

The control plane and data plane are independent binaries in separate repositories, each with its own build, test, and release cycle.

What You Get

Capability	Details
Gateway API	v1.5.1, 60+ features, all conformance tests pass
AI Gateway	Model routing, semantic cache, PII masking, content safety, A/B testing, fallback, cost tracking, Langfuse, token policies
Wasm plugins	Custom filters in Rust, wasmtime sandbox, deploy via CRD
Service Mesh	Gateway API Mesh model, no sidecars required
Dashboard	Next.js admin UI with 30+ pages
Observability	Prometheus metrics, Grafana dashboards, OpenTelemetry tracing, structured logging
Performance	9,000-11,000 RPS, P50 3-4ms, P99 11-15ms, ~105 MiB memory
Operations	Helm chart, HA config, PDBs, topology spread, alerting rules, backup procedures

Next Steps

• Quick Start
• Helm Installation
• AI Gateway Features
• Operations
• API Reference