Filters and Middleware - Tyk Documentation

Availability

Edition	Deployment Type
Community & Enterprise	Self-Managed, Hybrid

Filter execution is an Enterprise Edition feature. In the Community Edition, you can create filters and attach them, but AI Studio does not run them. Each filter passes the content through without changes, and AI Studio logs a warning.

Filter Types

A filter is one of two types. You select the type in Filter type on the filter form.

Type	What it does
Script	Runs Tengo code that you write against the request or the response. Most of this page describes script filters.
Guardrail (Enterprise Edition)	Runs a detection provider and blocks, redacts, or logs the content when the provider finds a match. You do not write code. The providers are a built-in pattern library, Lakera, Microsoft Presidio, Azure AI Content Safety, Azure AI Language PII, Amazon Bedrock Guardrails, and a generic HTTP classifier.

Both types attach, run, and record compliance events in the same way. In the API, the type is the kind field of a filter (script or guardrail). Filters that existed before v2.2.0 are script filters. New Enterprise installations include four default guardrails. They are not attached, so they have no effect until you attach them. To skip them, set SKIP_FILTER_DEFAULTS=true. For guardrail providers, detectors, and actions, refer to Guardrails. The Filters List View allows administrators to manage filters and middleware applied to prompts or data sent to Large Language Models (LLMs) via the AI Gateway or Chat Rooms. Filters and middleware ensure data governance, compliance, and security by processing or controlling the flow of information. Below is an enhanced description with the distinction between Filters and Middleware:

Filters: Unified Blocking and Modification

Filters in Tyk AI Studio provide comprehensive request/response processing with both blocking and modification capabilities:

Blocking Filters:
- Purpose: Governance controls that deny requests based on content analysis.
- Behavior:
  - Analyze message content, metadata, and context.
  - Block requests that violate policies or contain restricted content.
  - Example: Block prompts containing PII, sensitive keywords, or unauthorized patterns.
Modification Filters:
- Purpose: Transform message content before it reaches the LLM or after tool responses.
- Behavior:
  - Redact sensitive information (emails, phone numbers, SSNs).
  - Enhance system prompts with safety instructions.
  - Normalize or transform content across vendors.
  - Example: Automatically redact PII while allowing the request to proceed.
Combined Approach:
- Filters can both inspect AND modify in a single script.
- Example: Redact emails from user messages, but block if SSN is detected.

Key Capabilities:

✅ Request Filters: Modify/block requests before reaching LLM
- LLM Proxy Requests (before reaching LLM)
- Chat Session Messages (before RAG search and LLM)
- File Content (before RAG indexing)
- Tool Responses (after tool execution)
✅ Response Filters: Block LLM responses based on content
- LLM Proxy Responses (REST and streaming)
- Chat Session Responses (regular and streaming)
✅ Tool Filters: Run on the arguments sent to a tool and on the response from the tool. Refer to Tool Filters.

When you attach several filters to an LLM, they run from top to bottom in the order on the LLM form. Use the arrows on the form to change the order. The first filter that blocks stops the request. AI Studio stores the order and sends it to Edge Gateways.

Table Overview

Name:
- The name of the filter or middleware (e.g., Anonymize PII (LLM), Fixed PII Filter).
Description:
- A brief summary of the filter or middleware’s functionality (e.g., “Uses Regex to remove obvious PII”).
Actions:
- A menu (three-dot icon) that allows administrators to:
  - Edit the filter or middleware.
  - Delete the filter or middleware.

Features

Add Filter Button:
- A green button labeled + ADD FILTER, located in the top-right corner. Clicking this button opens a form to create a new filter or middleware.
Pagination Dropdown:
- Located at the bottom-left corner, this control allows administrators to adjust the number of entries displayed per page.

Examples of Filters and Middleware

Filters:
- PII Detector: A regex-based filter that blocks prompts containing sensitive PII.
- JIRA Field Analysis: Ensures no PII is included in data retrieved from JIRA fields before passing to the LLM.
Middleware:
- Anonymize PII (LLM): Uses an LLM to anonymize sensitive data before sending it downstream.
- NER Service Filter: A Named Entity Recognition (NER) microservice that modifies outputs to remove identified entities.

Use Cases

Prompt Validation with Filters:
- Ensures that only compliant and secure prompts are sent to LLMs.
- Example: Blocking a prompt with sensitive data that should not be processed by an unapproved vendor.
Data Preprocessing with Middleware:
- Prepares data from tools or external sources for safe interaction with LLMs by modifying or anonymizing content.
- Example: Removing sensitive ticket details from a JIRA query before sending to an LLM.
Organizational Security:
- Both filters and middleware ensure sensitive information is protected and handled in line with organizational governance policies.
Enhanced Tool Interactions:
- Middleware supports tools by transforming their outputs, enabling richer and safer LLM interactions.

Key Benefits

Improved Data Governance:
- Filters and middleware work together to enforce strict controls over data flow, protecting sensitive information.
Flexibility:
- Middleware allows for data transformation, enhancing interoperability between tools and LLMs.
- Filters ensure compliance without altering user-provided prompts.
Compliance and Security:
- Prevent unauthorized or sensitive data from reaching unapproved vendors, ensuring regulatory compliance.

This detailed structure for Filters and Middleware provides organizations with robust governance tools to secure and optimize data workflows in the Tyk AI Studio.

Filter Edit View (and example Filter)

The Filter Edit View enables administrators to create or modify filters using the Tengo scripting language. Filters serve as governance tools that analyze input data (e.g., prompts or files) and decide whether the content is permitted to pass to the upstream LLM. In this example, the filter uses regular expressions (regex) to detect Personally Identifiable Information (PII) and blocks the prompt if any matches are found.

Form Sections and Fields

Name (Required):
- Specifies the name of the filter (e.g., PII Detector).
Description (Optional):
- A brief summary of the filter’s purpose and functionality (e.g., “Simple Regex-based PII detector to prevent the wrong data being sent to LLMs”).
Filter type (Required):
- Script or Guardrail. Refer to Filter Types. The rest of this section describes script filters.
Script (Required):
- A Tengo script that defines the logic of the filter. The script evaluates input data and determines whether the filter approves or blocks it.
- The example script detects PII using a collection of regex patterns and blocks the data if a match is found.

Script Templates

A Load Template dropdown sits above the script editor. Use it to start a new filter from a ready-made script instead of an empty one. If the editor already contains a script, AI Studio asks for confirmation before it replaces the content. The available templates depend on the Is this a Response Filter? checkbox:

Filter Type	Template	Description
Request	PII Redaction	Redact emails, phone numbers, and SSNs using regex patterns
Response	Response Guardrails (Pattern Matching)	Block harmful content and policy violations using patterns
Response	Response Guardrails (LLM Check)	Use an LLM to validate response safety and compliance

PII Redaction (Request Filter)

Redacts email addresses, phone numbers, and US Social Security Numbers from the request payload, then allows the request to continue. Each redaction step re-parses the payload before it applies the next pattern. The filter also logs a pii_redacted compliance event that summarizes what it redacted.

// Comprehensive PII Redaction Filter
// This script redacts multiple types of personally identifiable information

tyk := import("tyk")

// Step 1: Redact email addresses
step1 := tyk.redact_pattern(
    input,
    "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
    "[EMAIL]"
)

// Step 2: Parse the result and redact phone numbers
json := import("json")
parsed := json.decode(step1)

// Extract messages from the modified payload
temp_messages := []
if !is_error(parsed) && parsed.messages {
    for msg in parsed.messages {
        temp_messages = append(temp_messages, msg)
    }
}

// Build temp input for second redaction
temp_input := {
    raw_input: step1,
    messages: temp_messages,
    vendor_name: input.vendor_name,
    model_name: input.model_name,
    is_chat: input.is_chat,
    context: input.context
}

// Step 2: Redact phone numbers
step2 := tyk.redact_pattern(
    temp_input,
    "\\(?\\d{3}\\)?[\\s.-]?\\d{3}[\\s.-]?\\d{4}",
    "[PHONE]"
)

// Step 3: Redact SSN (Social Security Numbers)
parsed2 := json.decode(step2)
temp_messages2 := []
if !is_error(parsed2) && parsed2.messages {
    for msg in parsed2.messages {
        temp_messages2 = append(temp_messages2, msg)
    }
}

temp_input2 := {
    raw_input: step2,
    messages: temp_messages2,
    vendor_name: input.vendor_name,
    model_name: input.model_name,
    is_chat: input.is_chat,
    context: input.context
}

final_payload := tyk.redact_pattern(
    temp_input2,
    "\\d{3}-\\d{2}-\\d{4}",
    "[SSN]"
)

output := {
    block: false,
    payload: final_payload,
    message: "PII redacted",
    compliance_events: [
        {
            event_type: "pii_redacted",
            severity: "warning",
            description: "Email, phone, and SSN patterns redacted",
            metadata: { "redacted_types": ["email", "phone", "ssn"] }
        }
    ]
}

Response Guardrails (Pattern Matching)

A response filter that blocks LLM responses with forbidden phrases, such as refund promises, harmful instructions, or unauthorized commitments. It works for both streaming and non-streaming responses. For a streaming buffer, it waits until it has 150 characters of context before it evaluates.

// Response Filter: Pattern-Based Guardrails
// This filter blocks responses containing harmful or policy-violating content

text := import("text")

// Get response text (works for both streaming and non-streaming)
response_text := ""
if input.is_chunk {
    response_text = input.current_buffer
} else {
    response_text = input.raw_input
}

// For streaming: wait for sufficient context before evaluation
min_evaluation_length := 150

// Default output
output := {
    block: false,
    message: ""
}

// Define patterns to block
blocked_patterns := [
    // Refund promises
    "will refund",
    "issue a refund",
    "provide a refund",
    "get a refund",
    "refund you",

    // Harmful content
    "instructions for making",
    "how to build a weapon",
    "steps to create explosives",
    "how to hack",
    "bypass security",

    // Commitments
    "I promise to",
    "I guarantee",
    "definitely will happen"
]

// Only evaluate if we have enough context
if !input.is_chunk || len(response_text) >= min_evaluation_length {
    // Check for blocked patterns
    is_blocked := false
    detected_pattern := ""

    for pattern in blocked_patterns {
        if text.contains(text.to_lower(response_text), pattern) {
            is_blocked = true
            detected_pattern = pattern
            break
        }
    }

    compliance_events_list := []
    if detected_pattern != "" {
        compliance_events_list = [
            {
                event_type: "harmful_content_detected",
                severity: is_blocked ? "critical" : "warning",
                description: "Detected pattern: '" + detected_pattern + "'",
                metadata: { "matched_pattern": detected_pattern, "blocked": is_blocked ? "true" : "false" }
            }
        ]
    }

    output = {
        block: is_blocked,
        message: is_blocked ? "Response blocked: Contains forbidden phrase '" + detected_pattern + "'" : "",
        compliance_events: compliance_events_list
    }
}

Response Guardrails (LLM Check)

A response filter that sends the response to a second, fast LLM and asks it to judge policy compliance, instead of matching fixed phrases. It waits for 200 characters of streaming context before it evaluates.

This template calls tyk.llm(1, settings, policy_check_prompt). The 1 is a placeholder LLM ID. Change it to your policy-checker LLM’s ID before you save the filter. Otherwise the script calls whichever LLM has ID 1 in your AI Studio instance.

// Response Filter: LLM-Based Policy Check
// This filter uses another LLM to analyze responses for policy violations

tyk := import("tyk")
text := import("text")

// Get response text
response_text := ""
if input.is_chunk {
    response_text = input.current_buffer
} else {
    response_text = input.raw_input
}

// For streaming: only evaluate once buffer reaches minimum size
min_buffer_size := 200

// Default output
output := {
    block: false,
    message: ""
}

// Only evaluate if we have enough context
if !input.is_chunk || len(response_text) >= min_buffer_size {
    // Use a fast, cheap LLM to check for policy violations
    // Configure the LLM call with minimal tokens for fast response
    settings := {
        model_name: "gpt-3.5-turbo",
        temperature: 0.0,
        max_tokens: 10,
        system_prompt: "You are a content policy checker. Respond ONLY with 'yes' or 'no'."
    }

    // Craft a policy check prompt
    policy_check_prompt := `Does this text promise a refund, commit to a specific action, or violate customer service policies?

Text: ` + response_text

    // Call the policy checker LLM (LLM ID 1 - adjust as needed)
    policy_result := tyk.llm(1, settings, policy_check_prompt)

    // Check if LLM detected a violation
    violates_policy := text.contains(text.to_lower(policy_result), "yes")

    compliance_events_list := []
    if violates_policy {
        compliance_events_list = [
            {
                event_type: "policy_violation_detected",
                severity: "critical",
                description: "LLM policy checker flagged response as violating policy"
            }
        ]
    }

    output = {
        block: violates_policy,
        message: violates_policy ? "Response blocked: LLM detected policy violation" : "",
        compliance_events: compliance_events_list
    }
}

Templates are a starting point. Edit the script after you load a template to match your own regex patterns, blocked phrases, or LLM ID.

New Unified Script API

Modern filters use a unified API that provides rich context and supports both blocking and modification: Input Object:

input := {
    raw_input: "...",      // Full JSON request payload
    messages: [            // Normalized message array with roles
        {role: "system", content: "You are helpful"},
        {role: "user", content: "Hello"}
    ],
    vendor_name: "openai", // LLM vendor (openai, anthropic, google_ai, etc.)
    model_name: "gpt-4",   // Model being called
    is_chat: false,        // Context: chat session (true) or proxy (false)
    context: {             // Additional metadata
        llm_id: 123,
        app_id: 456,
        user_id: 789
    }
}

Output Object:

output := {
    block: false,           // Set true to block the request
    payload: "",            // Modified JSON payload (or empty for no change)
    messages: [],           // Alternative: modified message array
    message: ""             // Optional reason/log message
}

Example Script 1: Blocking Filter (PII Detection)

This script blocks requests containing PII patterns:

text := import("text")

// Check all user messages for email addresses
should_block := false
block_reason := ""

for msg in input.messages {
    if msg.role == "user" {
        if text.contains(msg.content, "@") {
            should_block = true
            block_reason = "Email addresses not allowed"
            break
        }
    }
}

output := {
    block: should_block,
    payload: input.raw_input,
    message: block_reason
}

Example Script 2: Modification Filter (Email Redaction)

This script redacts emails while allowing the request to proceed:

tyk := import("tyk")

// Use helper to redact email addresses across all messages
modified_payload := tyk.redact_pattern(
    input,
    "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
    "[EMAIL_REDACTED]"
)

output := {
    block: false,
    payload: modified_payload,
    message: "Emails redacted"
}

Example Script 3: Advanced Modification (Messages Array)

This script shows complex message modification using the messages array approach:

text := import("text")

// Modify messages based on role
modified := []

for msg in input.messages {
    new_msg := {
        role: msg.role,
        content: msg.content
    }

    // Add safety prefix to system prompts
    if msg.role == "system" {
        new_msg.content = "[SAFETY MODE] " + msg.content
    }

    // Redact emails from user messages
    if msg.role == "user" {
        new_msg.content = text.replace(msg.content, "@", "[AT]", -1)
    }

    modified = append(modified, new_msg)
}

output := {
    block: false,
    messages: modified,  // System handles vendor-specific JSON reconstruction
    message: "Content modified"
}

Example Script 4: Combined Blocking + Modification

This script redacts emails but blocks if SSN is detected:

text := import("text")
tyk := import("tyk")

// First check for SSN (hard block)
ssn_pattern := "\\d{3}-\\d{2}-\\d{4}"
has_ssn := false

for msg in input.messages {
    if text.re_match(ssn_pattern, msg.content) {
        has_ssn = true
        break
    }
}

if has_ssn {
    output := {
        block: true,
        payload: "",
        message: "Blocked: SSN detected"
    }
} else {
    // No SSN - redact emails and continue
    modified_payload := tyk.redact_pattern(
        input,
        "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
        "[EMAIL]"
    )

    output := {
        block: false,
        payload: modified_payload,
        message: "Emails redacted"
    }
}

Available Helper Functions

The midsommar module provides helper functions for common message modification tasks:

redact_pattern(input, pattern, replacement):
- Redacts a regex pattern from all messages (system, user, assistant)
- Parameters:
  - input - The input object provided to your script
  - pattern - Regular expression pattern (string)
  - replacement - Replacement string
- Returns: Modified payload as string
- Example: tyk.redact_pattern(input, "\\d{3}-\\d{2}-\\d{4}", "[SSN]")

Vendor-Agnostic: Helper functions automatically handle differences between OpenAI, Anthropic, Google AI, and other vendor formats.

Message Modification Approaches

Approach 1: Helper Functions (Simple, recommended for pattern-based redaction)

tyk := import("tyk")
modified := tyk.redact_pattern(input, "@\\S+", "[EMAIL]")
output := {block: false, payload: modified, message: ""}

Approach 2: Messages Array (Flexible, recommended for complex logic)

modified := []
for msg in input.messages {
    new_msg := {role: msg.role, content: msg.content}
    if msg.role == "user" {
        // Apply custom modification logic
        new_msg.content = transform(msg.content)
    }
    modified = append(modified, new_msg)
}
output := {block: false, messages: modified, message: ""}

Accessing Message Context

Scripts can access rich contextual information:

// Access vendor and model information
if input.vendor_name == "anthropic" {
    // Anthropic-specific logic
}

// Count messages by role
user_count := 0
for msg in input.messages {
    if msg.role == "user" {
        user_count = user_count + 1
    }
}

// Check if this is a chat session or proxy request
if input.is_chat {
    // Chat-specific logic
}

// Access metadata
app_id := input.context.app_id
user_id := input.context.user_id

Testing Filter Scripts

A collapsible Test panel sits below the script editor on the Add/Edit Filter form. Use it to run the current script against sample input and check the output, without saving the filter first.

Testing a filter script, like creating or updating one, requires Tyk AI Studio Enterprise edition. On Community edition, the test request returns an “Enterprise Feature” error.

Expand the panel, fill in the fields, and click Run filter. The fields map directly onto the script input object described earlier on this page:

Field	Maps To	Description
Raw Input	`raw_input`	Sample request or response text, depending on the filter type.
Vendor	`vendor_name`	LLM provider: OpenAI, Anthropic, Google AI, or Azure OpenAI.
Model Name	`model_name`	Model identifier, for example `gpt-4` or `claude-3-opus`.
Is Response	`is_response`	Marks the input as response-side. Pre-selected when Is this a Response Filter? is checked on the form.
Is Chat	`is_chat`	Marks the input as a chat session context rather than a direct proxy call.
Is Chunk (Streaming)	`is_chunk`	Simulates a streaming chunk. Selecting it reveals three more fields.
Chunk Index	`chunk_index`	Chunk number, shown only when Is Chunk is selected.
Status Code	`status_code`	HTTP status code from the LLM, shown only when Is Chunk is selected.
Current Buffer	`current_buffer`	Accumulated response text so far, shown only when Is Chunk is selected.
Context (JSON)	`context`	Free-form JSON object. The default is `{"llm_id": 1, "app_id": 1, "request_id": "test-request-id"}`, which matches the context that the proxy sends.

From v2.2.0, if Raw Input is a JSON request body with a messages array, the panel sends that array as input.messages. If Raw Input is plain text, input.messages is empty. If the script raises compliance events, the panel shows them below the result. AI Studio runs the script in a sandbox with a 5 second timeout. If the script does not finish in time, AI Studio stops the script and the panel reports a timeout. On success, the panel shows the resulting output object: block, payload, message, and messages when the script sets it. On failure, it shows the Tengo runtime error. Under the hood, the panel calls:

POST /api/v1/filters/test

with a body of {"script": "<tengo script>", "input": {<script input object>}}, and an admin session or API token. You can call this endpoint directly, for example from a CI pipeline. Use it to validate a filter script before you create or update the filter through the API.

Action Buttons

Update Filter / Create Filter:
- Saves the filter configuration, making it active for future data processing.
Back to Filters:
- Returns to the Filters List View without saving changes.

Purpose and Benefits

Data Governance:
- Enforces strict control over what data can be sent to LLMs, ensuring compliance with privacy regulations.
Flexibility:
- Filters can be tailored to specific organizational needs using custom scripts.
Security:
- Prevents sensitive or unauthorized data from leaking to unapproved vendors or external systems.

This Filter Edit View provides a robust and customizable interface for creating scripts to enforce data governance and security in the Tyk AI Studio.

Example Middleware for Tools

Middleware filters in the Tyk AI Studio modify data coming from tools before passing it to the LLM. These filters are applied to sanitize, anonymize, or enhance the data to ensure it complies with organizational standards and privacy regulations. Below is an example of a middleware filter that sanitizes Personally Identifiable Information (PII), specifically email addresses, from the tool’s output.

Middleware Script: Email Redaction Example

// Import the 'text' module for regular expression operations
text := import("text")

// Define regular expression patterns for various PII
email_pattern := `[\w\.-]+@[\w\.-]+\.\w+`

// Define the function to sanitize PII in the input string
filter := func(input) {
    // Replace email addresses
    input = text.re_replace(email_pattern, input, "[REDACTED EMAIL]")

    return input
}

// Process the input payload
result := filter(payload)

Explanation of the Script

Module Import:
- The text module is imported to enable regular expression operations (text.re_replace).
Regex Pattern:
- A regex pattern is defined to detect email addresses:
  - Example pattern: [\w\.-]+@[\w\.-]+\.\w+
  - This pattern matches standard email formats.
Filter Function:
- The filter function accepts an input string (e.g., tool output) and:
  - Uses text.re_replace to identify email addresses.
  - Replaces detected email addresses with [REDACTED EMAIL].
Return Processed Output:
- The sanitized output is returned, ensuring that sensitive information like email addresses is redacted before reaching the LLM.

Use Case for Middleware

Tool Example: Imagine a tool, such as Support Ticket Viewer, which retrieves user tickets from a system. These tickets often contain email addresses. Middleware ensures that no sensitive email information is included in the output sent to the LLM.

Input Payload Example:

User email: john.doe@example.com has reported an issue with their account.

Sanitized Output:

User email: [REDACTED EMAIL] has reported an issue with their account.

Benefits of Middleware

Data Privacy:
- Protects sensitive user information by ensuring it is sanitized before being sent to external systems.
Compliance:
- Ensures organizational adherence to privacy laws like GDPR or HIPAA.
Enhanced Security:
- Prevents accidental sharing of PII with external vendors or LLMs.

Available Tengo Modules

Filters have access to powerful standard library modules:

text - String Operations

text := import("text")

// Common functions:
text.contains(str, substr)           // Check if substring exists
text.replace(str, old, new, n)       // Replace occurrences
text.to_upper(str)                   // Convert to uppercase
text.to_lower(str)                   // Convert to lowercase
text.split(str, sep)                 // Split string
text.trim_space(str)                 // Remove whitespace
text.re_match(pattern, str)          // Regex match
text.re_replace(pattern, str, repl)  // Regex replace

json - JSON Operations

json := import("json")

parsed := json.decode(json_string)   // Parse JSON
encoded := json.encode(object)       // Encode to JSON

fmt - Formatting and Printing

fmt := import("fmt")

fmt.println("Debug:", variable)      // Print for debugging
formatted := fmt.sprintf("Value: %v", val)  // Format strings

tyk - Extended Capabilities (Enterprise)

tyk := import("tyk")

// Redact regex patterns from all messages (vendor-agnostic)
modified := tyk.redact_pattern(input, pattern, replacement)
// Returns: Modified payload as string

// Make HTTP requests from within filters
result := tyk.makeHTTPRequest(method, url, headers, body)
// Returns: {status: 200, response: "..."}

// Call other LLMs for analysis/enrichment
response := tyk.llm(llm_id, settings_object, prompt)
// Parameters:
//   - llm_id: ID of the managed LLM to use
//   - settings_object: Map with model settings (model_name, temperature, max_tokens, etc.)
//   - prompt: The prompt text to send
// Returns: LLM response as string

Example - Use LLM for PII Detection:

tyk := import("tyk")

// Get user message
user_msg := ""
for msg in input.messages {
    if msg.role == "user" {
        user_msg = msg.content
        break
    }
}

// Use another LLM to detect PII
// Define settings for the LLM call
settings := {
    model_name: "gpt-3.5-turbo",
    temperature: 0.0,
    max_tokens: 10,
    system_prompt: "You are a PII detection assistant. Answer only 'yes' or 'no'."
}

pii_check_prompt := "Does this text contain PII?: " + user_msg
pii_result := tyk.llm(1, settings, pii_check_prompt)

output := {
    block: pii_result == "yes",
    payload: input.raw_input,
    message: pii_result == "yes" ? "PII detected by LLM" : ""
}

Example - Call External Service:

tyk := import("tyk")
json := import("json")

// Get user message
user_msg := ""
for msg in input.messages {
    if msg.role == "user" {
        user_msg = msg.content
    }
}

// Call external PII detection API
headers := {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_TOKEN"
}
body := json.encode({text: user_msg})

result := tyk.makeHTTPRequest("POST", "https://pii-api.example.com/detect", headers, body)

// Parse response
response := json.decode(result.response)
has_pii := response.has_pii

output := {
    block: has_pii,
    payload: input.raw_input,
    message: has_pii ? "External PII service detected sensitive data" : ""
}

Compliance Event Reporting

Available from Tyk AI Studio v2.1.0 (Enterprise edition). Filter scripts can emit structured Compliance Events to record governance-relevant activity, such as PII redactions, content rewrites, or guardrail triggers, without affecting the block/allow decision. Events flow through the analytics pipeline into the Compliance dashboard, where they appear in the Filter Events tab with severity filters, drill-down, and CSV export. To emit events, set the optional compliance_events field on the script output:

tyk := import("tyk")

modified_payload := tyk.redact_pattern(
    input,
    "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}",
    "[EMAIL_REDACTED]"
)

output := {
    block: false,
    payload: modified_payload,
    message: "",
    compliance_events: [
        {
            event_type: "pii_redacted",
            severity: "info",
            description: "Email addresses redacted from request",
            metadata: { "redacted_types": ["email"] }
        }
    ]
}

Each event accepts:

Field	Required	Description
`event_type`	Yes	Free-form type, for example `pii_redacted`, `content_rewritten`, `policy_violation`, `silent_failure`. Events without an `event_type` are skipped.
`severity`	No	`info`, `warning`, or `critical`. Defaults to `info`.
`description`	No	Human-readable description shown in the dashboard.
`metadata`	No	Arbitrary key-value data. Use `matched_pattern` (string) and `redacted_types` (array) for consistency with the bundled scripts.

Conditional Emission. Most filters should only emit events when a condition actually fires. Build the list dynamically:

events := []
if redaction_count > 0 {
    events = append(events, {
        event_type: "pii_redacted",
        severity: "info",
        description: fmt.sprintf("%d PII values redacted", redaction_count),
        metadata: { "redacted_types": redacted_types }
    })
}

output := {
    block: false,
    payload: modified_payload,
    message: "",
    compliance_events: events
}

Key behaviors:

Recording is asynchronous, so filter latency is unaffected.
Events never change the block/allow decision. Redact-and-allow with a critical event is a legitimate pattern.
Events work in every filter scope: proxy request/response, chat request/response, file reference, and tool response filters.
On Edge Gateways, events are batched and forwarded to the control plane on the analytics pulse.

The platform enriches each event with the app, user, LLM, vendor, model, filter name, and filter scope at recording time. For the event schema, querying via API, dashboard surfaces, and suggested event-type conventions, see Compliance Events.

Runtime Limits and Outbound Calls (Enterprise)

Filter scripts run on the goroutine that serves the request. For this reason, AI Studio limits every script execution. Each limit is an environment variable with a safe default. A change takes effect on the next execution, without a restart.

Variable	Default	Limit
`FILTER_SCRIPT_TIMEOUT`	`5s`	The maximum time for one script execution. When the time ends, AI Studio stops the script and the filter reports `script timed out`. A request filter then blocks the request. A response filter lets the response through. `0` removes the limit. A number without a unit is read as seconds.
`FILTER_SCRIPT_MAX_ALLOCS`	`10000000`	The maximum number of objects that one execution can allocate in the script VM. `0` removes the limit.
`FILTER_SCRIPT_ALLOW_OS`	`false`	Gives scripts access to the Tengo `os` module (environment variables, file system, and process execution). Keep it off. The process environment contains the secrets master key (`TYK_AI_SECRET_KEY`). A script with `os` access can read the key and decrypt all stored secrets. Set it to `true` only for old scripts that need the module.
`FILTER_HTTP_TIMEOUT`	`10s`	The maximum time for one `tyk.makeHTTPRequest` call.
`FILTER_HTTP_MAX_RESPONSE_BYTES`	`1048576`	The largest response body that `tyk.makeHTTPRequest` returns. A larger body is an error. AI Studio does not truncate it.
`FILTER_LLM_TIMEOUT`	`30s`	The maximum time for one `tyk.llm` call.

tyk.makeHTTPRequest uses the same outbound rules as LLM upstreams:

Only http and https URLs are allowed.
When LLM_UPSTREAM_ALLOWED_HOSTS is set, only those hosts are allowed.
When LLM_UPSTREAM_BLOCK_INTERNAL=true, internal addresses are blocked. To allow an internal classifier, add it to LLM_UPSTREAM_ALLOWED_INTERNAL_HOSTS.
AI Studio checks redirects against the same rules.

tyk.llm gets the credential of the target LLM in the same way as the gateway. An LLM with a key stored as a $SECRET/... or $ENV/... reference works from a script. Because response filters let the response through when they time out, a slow response filter does not block. If a response filter removes sensitive data, such as PII, a timeout sends the response to the user without that change. Keep response filter scripts fast, and test them with large responses. AI Studio compiles each script one time and caches it. Each execution runs on its own copy, so executions do not share state. When the caller of a request or chat session disconnects, AI Studio stops the script.

From v2.2.0, scripts cannot use the Tengo os module unless you set FILTER_SCRIPT_ALLOW_OS=true. Scripts that call os fail after the upgrade.

Tool Filters

You can attach filters to tools. From v2.2.0, tool filters run in two directions:

Filter direction	When it runs	What the script gets
Request filter (Is this a Response Filter? cleared)	Before AI Studio calls the tool	The tool call arguments, as JSON
Response filter (Is this a Response Filter? selected)	After the tool returns, before the response goes to the caller	The tool response body

Tool filters run on every way that a tool is called: in chat, on the REST endpoint, and on all MCP transports. They run on AI Studio and on Edge Gateways. The arguments that a request filter gets have the same format on every transport:

{
  "operation_id": "getExchangeRate",
  "parameters": {"currency": ["EUR"]},
  "payload": {},
  "headers": {}
}

A request filter can change parameters, payload, and headers, and return them in output.payload. It cannot change operation_id. If it tries, AI Studio ignores the change and logs a warning. input.context contains tool_id, tool_name, app_id, and user_id, and also session_id and call_id when they are available. Tool filters fail closed in both directions. If a filter blocks the call, or the script fails, the caller gets a generic refusal: HTTP 403 with blocked by policy, or the MCP equivalent. The refusal is the same for input and output blocks. It does not contain the filter name or the reason, so a caller cannot use it to learn your filter configuration. AI Studio writes the reason to the logs and records a compliance event for every block.

In versions before v2.2.0, every filter on a tool ran on the tool’s response, whatever its direction. From v2.2.0, a filter that is not a response filter runs on the tool’s input. After the upgrade, check the filters on your tools. The gateway logs a warning the first time that each such filter runs.

Tool Response Filter Examples

Tool responses are plain strings, not JSON-structured messages, so they require simpler handling.

Example 1: Block Tool Responses Containing Errors

text := import("text")

// Access tool response from messages array
tool_output := ""
if len(input.messages) > 0 {
    tool_output = input.messages[0].content
}

// Block if response indicates an error
should_block := text.contains(tool_output, "error") || text.contains(tool_output, "failed")

output := {
    block: should_block,
    payload: input.raw_input,
    message: should_block ? "Tool returned error response" : ""
}

Example 2: Redact PII from Tool Responses

text := import("text")

// Tool responses are plain strings - use direct string manipulation
modified := input.raw_input

// Get tool content
if len(input.messages) > 0 {
    content := input.messages[0].content

    // Redact email addresses
    modified = text.re_replace("[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", content, "[EMAIL]")

    // Redact phone numbers
    modified = text.re_replace("\\(?\\d{3}\\)?[\\s.-]?\\d{3}[\\s.-]?\\d{4}", modified, "[PHONE]")
}

output := {
    block: false,
    payload: modified,
    message: "PII redacted from tool response"
}

Example 3: Filter Based on Tool Name

// Access tool metadata from context
tool_name := ""
if input.context && input.context.tool_name {
    tool_name = input.context.tool_name
}

// Only allow specific tools to return data
allowed_tools := ["weather_api", "stock_prices"]
is_allowed := false

for allowed in allowed_tools {
    if tool_name == allowed {
        is_allowed = true
        break
    }
}

output := {
    block: !is_allowed,
    payload: input.raw_input,
    message: is_allowed ? "" : "Tool '" + tool_name + "' is not allowed"
}

Note: The tyk.redact_pattern() helper is designed for LLM message structures (JSON format) and will not work with tool responses. For tool responses, use direct string manipulation with the text module as shown above.

Filter Execution Order in Chat Sessions

When a user sends a message in a chat session, filters are executed at multiple points in the pipeline:

1. User Message Filters (Before RAG)

When: After preprocessing, before RAG vector search Purpose: Redact PII from user messages before they’re used for vector similarity search Context: input.messages[0].role == "user"

text := import("text")

// Redact before RAG search
modified := text.replace(input.messages[0].content, "@", "[EMAIL]", -1)

output := {
    block: false,
    payload: modified,
    message: ""
}

Impact: The filtered/modified message is used for:

✅ RAG vector similarity search
✅ Subsequent LLM processing
✅ Chat history storage

2. File Content Filters (Before RAG)

When: When files are attached to messages Purpose: Filter sensitive content from uploaded files before indexing Context: input.context.file_ref contains the file reference

3. Tool Response Filters (After Tool Execution)

When: After a tool returns data, before sending to LLM Purpose: Filter sensitive data from external API responses Context: input.messages[0].role == "tool", input.context.tool_name available

Best Practices

Always define output - Scripts must set the output variable
Use tyk.redact_pattern for LLM messages - Handles vendor differences automatically
Use text module for tool responses - Direct string manipulation
Use messages array for complex LLM modifications - Gives you full control
Provide clear block messages - Help users understand policy violations
Test across vendors - OpenAI, Anthropic, and Google AI have different formats
Check message roles - Different logic for system, user, assistant, and tool messages
Handle edge cases - Empty arrays, missing fields, etc.
Consider RAG impact - Filters run before RAG, so redactions affect vector search

Migration from Legacy API

Old API (still supported for backward compatibility):

filter := func(payload) {
    // Returns true/false for blocking only
    return true
}
result := filter(payload)

New Unified API (recommended):

// Rich input with messages, vendor info, context
output := {
    block: false,       // Blocking capability
    payload: "",        // Modification capability
    messages: [],       // Alternative modification approach
    message: ""         // Optional message
}

Response Filters

Response Filters enable administrators to block LLM responses based on content analysis, providing governance controls on what LLMs can say to end users.

Key Characteristics

Block-Only: Response filters can only block responses, not modify them
Works on LLM Responses: On LLMs and chat, response filters run on the LLM response. On tools, the same flag selects the tool output direction. Refer to Tool Filters.
Streaming Support: Execute per-chunk during streaming with access to accumulated buffer
Script-Controlled: Filter scripts decide when to evaluate based on buffer length

Configuration

Enable response filtering by checking “Is this a Response Filter?” when creating or editing a filter in the admin UI.

Script API for Response Filters

Response filters use the same ScriptInput/ScriptOutput structure as request filters, with additional response-specific fields: Input Object (Non-Streaming):

input := {
    raw_input: "The LLM's complete response text",
    is_response: true,
    is_chunk: false,
    vendor_name: "openai",
    model_name: "gpt-4",
    is_chat: true,  // or false for proxy
    context: {
        llm_id: 123,
        session_id: "abc-123",  // Only in chat context
        user_id: 789,
        chat_id: 456,
        status_code: 200
    }
}

Input Object (Streaming):

input := {
    raw_input: "current chunk text",
    is_response: true,
    is_chunk: true,
    is_final: false,        // true on the extra run after the last chunk
    chunk_index: 5,
    current_buffer: "accumulated response text so far",
    vendor_name: "openai",
    model_name: "gpt-4",
    is_chat: false,
    context: {
        llm_id: 123,
        app_id: 456
    }
}

Output Object:

output := {
    block: false,   // Set true to block/interrupt response
    message: ""     // Block reason (shown to user)
}

Note: The payload and messages fields in output are ignored for response filters.

Example 1: Block Refund Promises (Works for Streaming and Non-Streaming)

text := import("text")

// Get response text - use buffer for streaming
response_text := input.is_chunk ? input.current_buffer : input.raw_input

// Default output
output := {
    block: false,
    message: ""
}

// For streaming: only evaluate once we have enough context
if !input.is_chunk || len(response_text) >= 100 {
    // Check for forbidden patterns
    forbidden := ["will refund", "issue a refund", "provide a refund", "get a refund"]
    should_block := false

    for pattern in forbidden {
        if text.contains(text.to_lower(response_text), pattern) {
            should_block = true
            break
        }
    }

    output = {
        block: should_block,
        message: should_block ? "Response blocked: Cannot promise refunds to customers" : ""
    }
}

Example 2: Block Harmful Content (Streaming with Buffer Check)

text := import("text")

// Get response text (streaming-aware)
response_text := input.is_chunk ? input.current_buffer : input.raw_input

// Default output
output := {
    block: false,
    message: ""
}

// For streaming: script controls when to evaluate based on buffer length
// Wait until we have enough context before checking
if !input.is_chunk || len(response_text) >= 150 {
    // Check for harmful instruction patterns
    harmful := [
        "instructions for making",
        "how to build a weapon",
        "steps to create explosives"
    ]

    is_harmful := false
    for pattern in harmful {
        if text.contains(text.to_lower(response_text), pattern) {
            is_harmful = true
            break
        }
    }

    output = {
        block: is_harmful,
        message: is_harmful ? "Response blocked: Potentially harmful content detected" : ""
    }
}

Example 3: Combined with LLM Analysis

Use another LLM to analyze the response for policy violations:

tyk := import("tyk")
text := import("text")

// Get response text
response_text := input.is_chunk ? input.current_buffer : input.raw_input

// Default output
output := {
    block: false,
    message: ""
}

// For streaming: only evaluate once buffer is large enough
if !input.is_chunk || len(response_text) >= 200 {
    // Use fast LLM to check for policy violations
    settings := {
        model_name: "gpt-3.5-turbo",
        temperature: 0.0,
        max_tokens: 5,
        system_prompt: "You are a content policy checker. Answer only 'yes' or 'no'."
    }

    check_prompt := "Does this response promise a refund or commit to a specific action?: " + response_text
    result := tyk.llm(1, settings, check_prompt)

    should_block := text.contains(text.to_lower(result), "yes")

    output = {
        block: should_block,
        message: should_block ? "Response blocked: LLM policy violation detected" : ""
    }
}

Response Filter Execution

Proxy (REST):

Executes after response hooks (if any)
Full response available in input.raw_input
If blocked: Error returned to client instead of response

Proxy (Streaming):

Executes on every chunk
Executes one more time when the stream is complete, with is_final: true, an empty raw_input, and the full response in current_buffer. A short response that never reached the buffer threshold of the script is then still checked.
Access to both raw_input (current chunk) and current_buffer (accumulated text)
If blocked: Streaming stops, error sent to client
If blocked on the final run: The stream ends with an error event, and AI Studio logs a blocked response. The chunks that the client already received are not removed. To block content before the client receives it, check current_buffer on each chunk. Do not rely only on the final run.

Chat (Non-Streaming):

Executes before adding to chat history
If blocked: Error published to queue instead of response

Chat (Streaming):

Executes on every chunk before publishing to queue
If blocked: Error published, streaming callback returns error to stop further chunks

Best Practices for Response Filters

Buffer Management (Streaming): Script controls evaluation timing based on len(current_buffer)
Performance: Keep filter logic lightweight (executes per-chunk in streaming)
Clear Messages: Provide helpful block messages for users
Fail Open: Filters fail open on script errors (response allowed through)
LLM-Only: Response filters only work on LLM responses, not tool responses
Block-Only: Cannot modify responses, only block them

When to Use Response Filters vs Request Filters

Use Request Filters When:

Preventing sensitive data from reaching the LLM
Redacting PII before LLM processing
Enforcing input policies

Use Response Filters When:

Preventing LLMs from making commitments (refunds, promises)
Blocking harmful or inappropriate LLM outputs
Enforcing corporate communication policies
Detecting policy violations in LLM responses

This unified filter system demonstrates how flexible and powerful Tyk AI Studio’s scripting capabilities are, enabling administrators to enforce strict data governance policies while supporting advanced LLM and tool integration workflows with both blocking and modification capabilities.

​Availability

​Filter Types

​Filters: Unified Blocking and Modification

​Table Overview

​Features

​Examples of Filters and Middleware

​Use Cases

​Key Benefits

​Filter Edit View (and example Filter)

​Form Sections and Fields

​Script Templates

​PII Redaction (Request Filter)

​Response Guardrails (Pattern Matching)

​Response Guardrails (LLM Check)

​New Unified Script API

​Example Script 1: Blocking Filter (PII Detection)

​Example Script 2: Modification Filter (Email Redaction)

​Example Script 3: Advanced Modification (Messages Array)

​Example Script 4: Combined Blocking + Modification

​Available Helper Functions

​Message Modification Approaches

​Accessing Message Context

​Testing Filter Scripts

​Action Buttons

​Purpose and Benefits

​Example Middleware for Tools

​Middleware Script: Email Redaction Example

​Explanation of the Script

​Use Case for Middleware

​Benefits of Middleware

​Available Tengo Modules

​text - String Operations

​json - JSON Operations

​fmt - Formatting and Printing

​tyk - Extended Capabilities (Enterprise)

​Compliance Event Reporting

​Runtime Limits and Outbound Calls (Enterprise)

​Tool Filters

​Tool Response Filter Examples

​Example 1: Block Tool Responses Containing Errors

​Example 2: Redact PII from Tool Responses

​Example 3: Filter Based on Tool Name

​Filter Execution Order in Chat Sessions

​1. User Message Filters (Before RAG)

​2. File Content Filters (Before RAG)

​3. Tool Response Filters (After Tool Execution)

​Best Practices

​Migration from Legacy API

​Response Filters

​Key Characteristics

​Configuration

​Script API for Response Filters

​Example 1: Block Refund Promises (Works for Streaming and Non-Streaming)

​Example 2: Block Harmful Content (Streaming with Buffer Check)

​Example 3: Combined with LLM Analysis

​Response Filter Execution

​Best Practices for Response Filters

​When to Use Response Filters vs Request Filters

Availability

Filter Types

Filters: Unified Blocking and Modification

Table Overview

Features

Examples of Filters and Middleware

Use Cases

Key Benefits

Filter Edit View (and example Filter)

Form Sections and Fields

Script Templates

PII Redaction (Request Filter)

Response Guardrails (Pattern Matching)

Response Guardrails (LLM Check)

New Unified Script API

Example Script 1: Blocking Filter (PII Detection)

Example Script 2: Modification Filter (Email Redaction)

Example Script 3: Advanced Modification (Messages Array)

Example Script 4: Combined Blocking + Modification

Available Helper Functions

Message Modification Approaches

Accessing Message Context

Testing Filter Scripts

Action Buttons

Purpose and Benefits

Example Middleware for Tools

Middleware Script: Email Redaction Example

Explanation of the Script

Use Case for Middleware

Benefits of Middleware

Available Tengo Modules

text - String Operations

json - JSON Operations

fmt - Formatting and Printing

tyk - Extended Capabilities (Enterprise)

Compliance Event Reporting

Runtime Limits and Outbound Calls (Enterprise)

Tool Filters

Tool Response Filter Examples

Example 1: Block Tool Responses Containing Errors

Example 2: Redact PII from Tool Responses

Example 3: Filter Based on Tool Name

Filter Execution Order in Chat Sessions

1. User Message Filters (Before RAG)

2. File Content Filters (Before RAG)

3. Tool Response Filters (After Tool Execution)

Best Practices

Migration from Legacy API

Response Filters

Key Characteristics

Configuration

Script API for Response Filters

Example 1: Block Refund Promises (Works for Streaming and Non-Streaming)

Example 2: Block Harmful Content (Streaming with Buffer Check)

Example 3: Combined with LLM Analysis

Response Filter Execution

Best Practices for Response Filters

When to Use Response Filters vs Request Filters