AI Proxy Response Body Unexpected #14715
Description
Is there an existing issue for this?
- I have searched the existing issues
Kong version ($ kong version)
3.9.1
Current Behavior
We use the AI proxy with the official doc
https://developer.konghq.com/plugins/ai-proxy/examples/sdk-two-routes/
But meet the unexpected response payload likes:
Expected Behavior
The response body model is model name, not $(uri_captures.model)
curl -X POST http://localhost:8000/gemini/gemini-2.0-flash-lite/chat/completions -s -H 'Content-Type: application/json' --data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }' | jq .
{
"usage": {
"prompt_tokens": 11,
"total_tokens": 19,
"completion_tokens": 8
},
"choices": [
{
"message": {
"role": "assistant",
"content": "1 + 1 = 2\n"
},
"index": 0,
"finish_reason": "stop"
}
],
"model": "$(uri_captures.model)",
"object": "chat.completion"
}
We want the response with openAI
https://platform.openai.com/docs/api-reference/chat/create
{
"usage": {
"prompt_tokens": 11,
"total_tokens": 19,
"completion_tokens": 8
},
"choices": [
{
"message": {
"role": "assistant",
"content": "1 + 1 = 2\n"
},
"index": 0,
"finish_reason": "stop"
}
],
"model": "gemini-2.0-flash-lite",
"object": "chat.completion"
}
Steps To Reproduce
with the route configure
curl -X GET http://localhost:8001/routes/gemini-chat -s | jq .
{
"snis": null,
"tags": [
"gemini-chat"
],
"request_buffering": true,
"response_buffering": true,
"name": "gemini-chat",
"path_handling": "v0",
"https_redirect_status_code": 426,
"strip_path": true,
"headers": null,
"created_at": 1753337080,
"updated_at": 1753337080,
"protocols": [
"http",
"https"
],
"methods": null,
"sources": null,
"destinations": null,
"preserve_host": false,
"paths": [
"~/gemini/(?<model>[^#?/]+)/chat/completions$"
],
"service": {
"id": "32055a79-446a-402d-a543-e6434702bb0f"
},
"id": "4e5555a1-6dff-45ba-b233-447a773273f1",
"hosts": null,
"regex_priority": 0
}
curl -i -X POST http://localhost:8001/routes/gemini-chat/plugins/ \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--data '
{
"name": "ai-proxy",
"config": {
"route_type": "llm/v1/chat",
"auth": {
"param_name": "key",
"param_value": "${token}",
"param_location": "query"
},
"model": {
"provider": "gemini",
"name": "$(uri_captures.model)"
}
}
}
'
Anything else?
No response