Skip to content

Commit 7bba93f

Browse files
tomek-labukGuaris
tomek-labuk
and
Guaris
authored
feat(ai-gateway): AI Semantic Response Guard plugin (#2757)
* Add scaffold plugin structure * Add WIP contend and examples * change config example * fix * Update ai gw landing page * Add pgvecotr example * fixes * icons --------- Co-authored-by: Angel <[email protected]>
1 parent 44ade39 commit 7bba93f

File tree

9 files changed

+366
-0
lines changed

9 files changed

+366
-0
lines changed

‎app/_kong_plugins/ai-semantic-response-guard/changelog.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"3.12.0.0": [
3+
{
4+
"message": "Added new plugin to permit or block prompts based on semantic similarity to known LLM responses, preventing misuse of llm/v1/chat or llm/v1/completions requests",
5+
"scope": "Plugin",
6+
"type": "feature"
7+
}
8+
]
9+
}

‎app/_kong_plugins/ai-semantic-response-guard/examples/allow-and-deny-responses-pgvector.yaml

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
description: Block or allow LLM responses based on semantic similarity to defined rules.
2+
3+
extended_description: |
4+
The AI Semantic Response Guard plugin analyzes the full response from an LLM service and filters it
5+
based on semantic similarity to configured allow or deny patterns.
6+
7+
Deny rules take precedence over allow rules. Responses matching a deny pattern are blocked,
8+
even if they also match an allow pattern. Responses not matching any allow pattern are blocked
9+
when allow rules are set.
10+
11+
title: 'Allow and deny using pgvector as a vector database'
12+
13+
weight: 900
14+
15+
requirements:
16+
- "[AI Proxy plugin](/plugins/ai-proxy/) or [AI Proxy Advanced plugin](/plugins/ai-proxy-advanced/) configured with an LLM service."
17+
- "A [PostgreSQL database with pgvector extension](https://github.com/pgvector/pgvector) installed and reachable from {{site.base_gateway}}."
18+
- "Port `5432`, or your custom PostgreSQL port, is open and reachable from {{site.base_gateway}}."
19+
20+
variables:
21+
header_value:
22+
value: $OPENAI_API_KEY
23+
description: Your OpenAI API key
24+
pgvector_host:
25+
value: $PGVECTOR_HOST
26+
description: The host where your pgvector-enabled PostgreSQL instance runs
27+
pgvector_user:
28+
value: $PGVECTOR_USER
29+
description: Database user for pgvector
30+
pgvector_password:
31+
value: $PGVECTOR_PASSWORD
32+
description: Database password for pgvector
33+
34+
config:
35+
embeddings:
36+
auth:
37+
header_name: Authorization
38+
header_value: Bearer ${header_value}
39+
model:
40+
name: text-embedding-3-small
41+
provider: openai
42+
search:
43+
threshold: 0.7
44+
vectordb:
45+
strategy: pgvector
46+
distance_metric: cosine
47+
threshold: 0.7
48+
dimensions: 1024
49+
pgvector:
50+
host: ${pgvector_host}
51+
port: 5432
52+
database: kong-pgvector
53+
user: ${pgvector_user}
54+
password: ${pgvector_password}
55+
ssl: false
56+
ssl_required: false
57+
ssl_verify: false
58+
ssl_version: tlsv1_2
59+
timeout: 5000
60+
rules:
61+
allow_responses:
62+
- Troubleshooting networks and connectivity issues
63+
- Managing cloud platforms (AWS, Azure, GCP)
64+
- Security hardening and incident response strategies
65+
- DevOps pipelines, automation, and observability
66+
- Software engineering concepts and language syntax
67+
- IT governance, compliance, and regulatory guidance
68+
- Continuous integration and deployment practices
69+
- Writing documentation and explaining technical concepts
70+
- Operating system administration and configuration
71+
- Best practices for collaboration and productivity tools
72+
deny_responses:
73+
- Unauthorized penetration testing or hacking tutorials
74+
- Methods for bypassing software licensing or DRM
75+
- Step-by-step instructions for exploiting vulnerabilities
76+
- Techniques to evade or disable security controls
77+
- Collecting or exposing personal or employee data
78+
- Using AI for impersonation, phishing, or fraud
79+
- Manipulative social engineering techniques
80+
- Advice on breaking internal IT or security policies
81+
- Entertainment, dating, or other non-work topics
82+
- Political, religious, or otherwise sensitive discussions unrelated to work
83+
84+
tools:
85+
- deck
86+
- admin-api
87+
- konnect-api
88+
- kic
89+
- terraform

‎app/_kong_plugins/ai-semantic-response-guard/examples/allow-and-deny-responses-redis.yaml

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
description: Block or allow LLM responses based on semantic similarity to defined rules.
2+
3+
extended_description: |
4+
The AI Semantic Response Guard plugin analyzes the full response from an LLM service and filters it
5+
based on semantic similarity to configured allow or deny patterns.
6+
7+
Deny rules take precedence over allow rules. Responses matching a deny pattern are blocked,
8+
even if they also match an allow pattern. Responses not matching any allow pattern are blocked
9+
when allow rules are set.
10+
11+
title: 'Allow and deny responses using Redis as a vector database'
12+
13+
weight: 900
14+
15+
requirements:
16+
- "[AI Proxy plugin](/plugins/ai-proxy/) or [AI Proxy Advanced plugin](/plugins/ai-proxy-advanced/) configured with an LLM service."
17+
- "A [Redis](https://redis.io/docs/latest/) instance or another supported vector database."
18+
- "Port `6379`, or your custom Redis port, is open and reachable from {{site.base_gateway}}."
19+
20+
variables:
21+
header_value:
22+
value: $OPENAI_API_KEY
23+
description: Your OpenAI API key
24+
redis_host:
25+
value: $REDIS_HOST
26+
description: The host where your Redis instance runs
27+
28+
config:
29+
embeddings:
30+
auth:
31+
header_name: Authorization
32+
header_value: Bearer ${header_value}
33+
model:
34+
name: text-embedding-3-small
35+
provider: openai
36+
search:
37+
threshold: 0.7
38+
vectordb:
39+
strategy: redis
40+
distance_metric: cosine
41+
threshold: 0.7
42+
dimensions: 1024
43+
redis:
44+
host: ${redis_host}
45+
port: 6379
46+
rules:
47+
allow_responses:
48+
- Network troubleshooting and diagnostics
49+
- Cloud infrastructure management (AWS, Azure, GCP)
50+
- Cybersecurity best practices and incident response
51+
- DevOps workflows and automation
52+
- Programming concepts and language usage
53+
- IT policy and compliance guidance
54+
- Software development lifecycle and CI/CD
55+
- Documentation writing and technical explanation
56+
- System administration and configuration
57+
- Productivity and collaboration tools usage
58+
deny_responses:
59+
- Hacking techniques or penetration testing without authorization
60+
- Bypassing software licensing or digital rights management
61+
- Instructions on exploiting vulnerabilities or writing malware
62+
- Circumventing security controls or access restrictions
63+
- Gathering personal or confidential employee information
64+
- Using AI to impersonate or phish others
65+
- Social engineering tactics or manipulation techniques
66+
- Guidance on violating company IT policies
67+
- Content unrelated to work, such as entertainment or dating
68+
- Political, religious, or sensitive non-work-related discussions
69+
70+
tools:
71+
- deck
72+
- admin-api
73+
- konnect-api
74+
- kic
75+
- terraform

‎app/_kong_plugins/ai-semantic-response-guard/examples/allow-responses.yaml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
description: Allow only specific LLM responses based on semantic similarity to defined rules.
2+
3+
extended_description: |
4+
The AI Semantic Response Guard plugin analyzes the full response from an LLM service and permits it
5+
only if it semantically matches one of the configured allow patterns.
6+
7+
If a response does not match any of the allow patterns, it is blocked with a 400 Bad Request.
8+
9+
title: 'Allow only responses'
10+
11+
weight: 900
12+
13+
requirements:
14+
- "[AI Proxy plugin](/plugins/ai-proxy/) or [AI Proxy Advanced plugin](/plugins/ai-proxy-advanced/) configured with an LLM service."
15+
- "A [Redis](https://redis.io/docs/latest/) instance or another supported vector database."
16+
- "Port `6379`, or your custom Redis port, is open and reachable from {{site.base_gateway}}."
17+
18+
variables:
19+
header_value:
20+
value: $OPENAI_API_KEY
21+
description: Your OpenAI API key
22+
redis_host:
23+
value: $REDIS_HOST
24+
description: The host where your Redis instance runs
25+
26+
config:
27+
embeddings:
28+
auth:
29+
header_name: Authorization
30+
header_value: Bearer ${header_value}
31+
model:
32+
name: text-embedding-3-small
33+
provider: openai
34+
search:
35+
threshold: 0.7
36+
vectordb:
37+
strategy: redis
38+
distance_metric: cosine
39+
threshold: 0.7
40+
dimensions: 1024
41+
redis:
42+
host: ${redis_host}
43+
port: 6379
44+
rules:
45+
allow_responses:
46+
- Network troubleshooting and diagnostics
47+
- Cloud infrastructure management (AWS, Azure, GCP)
48+
- Cybersecurity best practices and incident response
49+
- DevOps workflows and automation
50+
- Programming concepts and language usage
51+
- IT policy and compliance guidance
52+
- Software development lifecycle and CI/CD
53+
- Documentation writing and technical explanation
54+
- System administration and configuration
55+
- Productivity and collaboration tools usage
56+
57+
tools:
58+
- deck
59+
- admin-api
60+
- konnect-api
61+
- kic
62+
- terraform

‎app/_kong_plugins/ai-semantic-response-guard/examples/deny-responses.yaml

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
description: Block specific LLM responses based on semantic similarity to defined rules.
2+
3+
extended_description: |
4+
The AI Semantic Response Guard plugin analyzes the full response from an LLM service and blocks it
5+
if it semantically matches one of the configured deny patterns.
6+
7+
Responses that do not match any deny pattern are permitted.
8+
9+
title: 'Deny only responses'
10+
11+
weight: 900
12+
13+
requirements:
14+
- "[AI Proxy plugin](/plugins/ai-proxy/) or [AI Proxy Advanced plugin](/plugins/ai-proxy-advanced/) configured with an LLM service."
15+
- "A [Redis](https://redis.io/docs/latest/) instance or another supported vector database."
16+
- "Port `6379`, or your custom Redis port, is open and reachable from {{site.base_gateway}}."
17+
18+
variables:
19+
header_value:
20+
value: $OPENAI_API_KEY
21+
description: Your OpenAI API key
22+
redis_host:
23+
value: $REDIS_HOST
24+
description: The host where your Redis instance runs
25+
26+
config:
27+
embeddings:
28+
auth:
29+
header_name: Authorization
30+
header_value: Bearer ${header_value}
31+
model:
32+
name: text-embedding-3-small
33+
provider: openai
34+
search:
35+
threshold: 0.7
36+
vectordb:
37+
strategy: redis
38+
distance_metric: cosine
39+
threshold: 0.7
40+
dimensions: 1024
41+
redis:
42+
host: ${redis_host}
43+
port: 6379
44+
rules:
45+
deny_responses:
46+
- Hacking techniques or penetration testing without authorization
47+
- Bypassing software licensing or digital rights management
48+
- Instructions on exploiting vulnerabilities or writing malware
49+
- Circumventing security controls or access restrictions
50+
- Gathering personal or confidential employee information
51+
- Using AI to impersonate or phish others
52+
- Social engineering tactics or manipulation techniques
53+
- Guidance on violating company IT policies
54+
- Content unrelated to work, such as entertainment or dating
55+
- Political, religious, or sensitive non-work-related discussions
56+
57+
tools:
58+
- deck
59+
- admin-api
60+
- konnect-api
61+
- kic
62+
- terraform

‎app/_kong_plugins/ai-semantic-response-guard/index.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: 'AI Semantic Response Guard'
3+
name: 'AI Semantic Response Guard'
4+
5+
content_type: plugin
6+
tier: ai_gateway_enterprise
7+
8+
publisher: kong-inc
9+
description: 'Permit or block prompts based on semantic similarity to known LLM responses, preventing misuse of llm/v1/chat or llm/v1/completions requests'
10+
11+
products:
12+
- gateway
13+
- ai-gateway
14+
15+
works_on:
16+
- on-prem
17+
- konnect
18+
19+
min_version:
20+
gateway: '3.12'
21+
22+
topologies:
23+
on_prem:
24+
- hybrid
25+
- db-less
26+
- traditional
27+
konnect_deployments:
28+
- hybrid
29+
- cloud-gateways
30+
- serverless
31+
32+
icon: ai-semantic-response-guard.png # e.g. acme.svg or acme.png
33+
34+
tags:
35+
- ai
36+
---
37+
38+
The AI Semantic Response Guard plugin extends the AI Prompt Guard plugin by filtering LLM responses based on semantic similarity to predefined rules. It helps prevent unwanted or unsafe responses when serving `llm/v1/chat`, `llm/v1/completions`, or `llm/v1/embeddings` requests through Kong AI Gateway.
39+
40+
You can use a combination of `allow` and `deny` response rules to maintain integrity and compliance when returning responses from an LLM service.
41+
42+
## How it works
43+
44+
The plugin analyzes the semantic content of the full LLM response before it is returned to the client. The matching behavior is as follows:
45+
46+
* If any `deny_responses` are set and the response matches a pattern in the deny list, the response is blocked with a `400 Bad Request`.
47+
* If any `allow_responses` are set, but the response matches none of the allowed patterns, the response is also blocked with a `400 Bad Request`.
48+
* If any `allow_responses` are set and the response matches one of the allowed patterns, the response is permitted.
49+
* If both `deny_responses` and `allow_responses` are set, the `deny` condition takes precedence. A response that matches a deny pattern will be blocked, even if it also matches an allow pattern. If the response does not match any deny pattern, it must still match an allow pattern to be permitted.
50+
51+
## Response processing
52+
53+
To enforce these rules, the plugin:
54+
55+
1. **Disables streaming** (`stream=false`) to ensure the full response body is buffered before analysis.
56+
2. **Intercepts the response body** using the `guard-response` filter.
57+
3. **Extracts response text**, supporting JSON parsing of multiple LLM formats and gzipped content.
58+
4. **Generates embeddings** for the extracted text.
59+
5. **Searches the vector database** (Redis, Pgvector, or other) against configured `allow_responses` or `deny_responses`.
60+
6. **Applies the decision rules** described above.
61+
62+
If a response is blocked or if a system error occurs during evaluation, the plugin returns a `400 Bad Request` to the client without exposing that the Semantic Response Guard blocked it.

‎app/_kong_plugins/ai-semantic-response-guard/reference.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
content_type: reference
3+
---

‎app/_landing_pages/ai-gateway.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -431,6 +431,10 @@ rows:
431431
- type: plugin
432432
config:
433433
slug: ai-aws-guardrails
434+
- blocks:
435+
- type: plugin
436+
config:
437+
slug: ai-semantic-response-guard
434438
- blocks:
435439
- type: card
436440
config:

‎app/assets/icons/plugins/ai-semantic-response-guard.png

8.38 KB
Loading

0 commit comments

Comments
 (0)