Repository navigation
compose-model-service puts URLRewrite filter under backendRef, breaking the data path #85
Description
What happened?
Every request through a ModelService returns HTTP 500 from the control plane gateway. The model serving pod on the workload cluster is healthy and never receives the request.
compose-model-service builds an HTTPRoute that attaches a URLRewrite filter to each backendRef:
The Gateway API spec only allows RequestHeaderModifier and ResponseHeaderModifier at the backendRef level. URLRewrite is a rule-level-only filter. Envoy Gateway rejects the route accordingly:
status:
parents:
- conditions:
- type: ResolvedRefs
status: "False"
reason: UnsupportedRefValue
message: Specific filter is not supported within BackendRef, only
RequestHeaderModifier and ResponseHeaderModifier are supported
With the route unresolved, Envoy emits a direct_response for every matching request:
{
"method": "POST",
"x-envoy-origin-path": "/ml-team/qwen/v1/chat/completions",
"response_code": "500",
"response_code_details": "direct_response",
"route_name": "httproute/ml-team/qwen-3e76dee66d25/rule/0/match/0/*"
}
The function comment says per-backend filters are intentional, to support endpoints with different rewritePath values:
Each backendRef carries its own URLRewrite filter so endpoints with different rewritePaths (e.g. composed replicas vs. external SaaS providers) are rewritten correctly per-backend.
But Gateway API doesn't allow this shape.
How can we reproduce it?
Walk through docs/getting-started.md to deploy qwen-demo on GKE. Once ModelDeployment reports READY=True, hit the endpoint:
address=$(kubectl get ms qwen -n ml-team -o jsonpath='{.status.address}')
kubectl run curl-test --image=curlimages/curl --restart=Never \
--command -- curl -sS -i "$address/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{"model":"Qwen/Qwen2.5-0.5B-Instruct","messages":[{"role":"user","content":"hi"}]}'
kubectl wait --for=jsonpath='{.status.phase}'=Succeeded pod/curl-test
kubectl logs curl-testResponse: HTTP/1.1 500 Internal Server Error.
Then inspect the composed HTTPRoute:
kubectl get httproute -n ml-team -o yamlThe ResolvedRefs condition reports UnsupportedRefValue with the message above.
What environment did it happen in?
Modelplane version: v0.1.0-dev.1779486230.g401e962-dirty (built from #83)
Crossplane version: v2.3.1
Inference backend: KServe v0.16.0
Cloud provider: GKE (g2-standard-8, NVIDIA L4)
Kubernetes version: control plane v1.34.0 (kind), workload v1.35.3-gke.1389000
provider-helm: v1.2.4
provider-kubernetes: v1.2.5
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
Move URLRewrite filter from backendRef to rule level
negz commentedon May 27, 2026
The root cause here is that a
ModelServicebacked by endpoints with different path conventions (e.g. a composed endpoint serving at/v1/...alongside an external provider like Groq at/openai/v1/...) needs per-backend URL rewriting within a single weighted traffic split.The Gateway API spec supports this —
URLRewriteis listed as an Extended feature at theHTTPBackendReflevel. The fix would be to move theURLRewritefilter from the rule level into eachbackendRef:Envoy Gateway can't do this. It only supports
RequestHeaderModifierandResponseHeaderModifierat thebackendReflevel. Per-backend path rewriting is blocked on Envoy itself —WeightedCluster.ClusterWeightin Envoy's route proto doesn't supportprefix_rewrite. See envoyproxy/gateway#7099 and envoyproxy/gateway#7020 (both open, no timeline). The original tracking issue for backendRef filter support (envoyproxy/gateway#2572) was closed after header modification was shipped, but path rewrite support was explicitly deferred pending upstream Envoy changes.