Availability
| Edition | Deployment Type |
|---|---|
| Community & Enterprise | Self-Managed, Hybrid |
Filters: Unified Blocking and Modification
Filters in Midsommar provide comprehensive request/response processing with both blocking and modification capabilities:-
Blocking Filters:
- Purpose: Governance controls that deny requests based on content analysis.
- Behavior:
- Analyze message content, metadata, and context.
- Block requests that violate policies or contain restricted content.
- Example: Block prompts containing PII, sensitive keywords, or unauthorized patterns.
-
Modification Filters:
- Purpose: Transform message content before it reaches the LLM or after tool responses.
- Behavior:
- Redact sensitive information (emails, phone numbers, SSNs).
- Enhance system prompts with safety instructions.
- Normalize or transform content across vendors.
- Example: Automatically redact PII while allowing the request to proceed.
-
Combined Approach:
- Filters can both inspect AND modify in a single script.
- Example: Redact emails from user messages, but block if SSN is detected.
- ✅ Request Filters: Modify/block requests before reaching LLM
- LLM Proxy Requests (before reaching LLM)
- Chat Session Messages (before RAG search and LLM)
- File Content (before RAG indexing)
- Tool Responses (after tool execution)
- ✅ Response Filters: Block LLM responses based on content
- LLM Proxy Responses (REST and streaming)
- Chat Session Responses (regular and streaming)
Table Overview
-
Name:
- The name of the filter or middleware (e.g.,
Anonymize PII (LLM),Fixed PII Filter).
- The name of the filter or middleware (e.g.,
-
Description:
- A brief summary of the filter or middleware’s functionality (e.g., “Uses Regex to remove obvious PII”).
-
Actions:
- A menu (three-dot icon) that allows administrators to:
- Edit the filter or middleware.
- Delete the filter or middleware.
- A menu (three-dot icon) that allows administrators to:
Features
-
Add Filter Button:
- A green button labeled + ADD FILTER, located in the top-right corner. Clicking this button opens a form to create a new filter or middleware.
-
Pagination Dropdown:
- Located at the bottom-left corner, this control allows administrators to adjust the number of entries displayed per page.
Examples of Filters and Middleware
-
Filters:
- PII Detector: A regex-based filter that blocks prompts containing sensitive PII.
- JIRA Field Analysis: Ensures no PII is included in data retrieved from JIRA fields before passing to the LLM.
-
Middleware:
- Anonymize PII (LLM): Uses an LLM to anonymize sensitive data before sending it downstream.
- NER Service Filter: A Named Entity Recognition (NER) microservice that modifies outputs to remove identified entities.
Use Cases
-
Prompt Validation with Filters:
- Ensures that only compliant and secure prompts are sent to LLMs.
- Example: Blocking a prompt with sensitive data that should not be processed by an unapproved vendor.
-
Data Preprocessing with Middleware:
- Prepares data from tools or external sources for safe interaction with LLMs by modifying or anonymizing content.
- Example: Removing sensitive ticket details from a JIRA query before sending to an LLM.
-
Organizational Security:
- Both filters and middleware ensure sensitive information is protected and handled in line with organizational governance policies.
-
Enhanced Tool Interactions:
- Middleware supports tools by transforming their outputs, enabling richer and safer LLM interactions.
Key Benefits
-
Improved Data Governance:
- Filters and middleware work together to enforce strict controls over data flow, protecting sensitive information.
-
Flexibility:
- Middleware allows for data transformation, enhancing interoperability between tools and LLMs.
- Filters ensure compliance without altering user-provided prompts.
-
Compliance and Security:
- Prevent unauthorized or sensitive data from reaching unapproved vendors, ensuring regulatory compliance.
Filter Edit View (and example Filter)
The Filter Edit View enables administrators to create or modify filters using the Tengo scripting language. Filters serve as governance tools that analyze input data (e.g., prompts or files) and decide whether the content is permitted to pass to the upstream LLM. In this example, the filter uses regular expressions (regex) to detect Personally Identifiable Information (PII) and blocks the prompt if any matches are found.Form Sections and Fields
-
Name (Required):
- Specifies the name of the filter (e.g.,
PII Detector).
- Specifies the name of the filter (e.g.,
-
Description (Optional):
- A brief summary of the filter’s purpose and functionality (e.g., “Simple Regex-based PII detector to prevent the wrong data being sent to LLMs”).
-
Script (Required):
- A Tengo script that defines the logic of the filter. The script evaluates input data and determines whether the filter approves or blocks it.
- The example script detects PII using a collection of regex patterns and blocks the data if a match is found.
New Unified Script API
Modern filters use a unified API that provides rich context and supports both blocking and modification: Input Object:Example Script 1: Blocking Filter (PII Detection)
This script blocks requests containing PII patterns:Example Script 2: Modification Filter (Email Redaction)
This script redacts emails while allowing the request to proceed:Example Script 3: Advanced Modification (Messages Array)
This script shows complex message modification using the messages array approach:Example Script 4: Combined Blocking + Modification
This script redacts emails but blocks if SSN is detected:Available Helper Functions
Themidsommar module provides helper functions for common message modification tasks:
redact_pattern(input, pattern, replacement):- Redacts a regex pattern from all messages (system, user, assistant)
- Parameters:
input- The input object provided to your scriptpattern- Regular expression pattern (string)replacement- Replacement string
- Returns: Modified payload as string
- Example:
tyk.redact_pattern(input, "\\d{3}-\\d{2}-\\d{4}", "[SSN]")
Message Modification Approaches
Approach 1: Helper Functions (Simple, recommended for pattern-based redaction)Accessing Message Context
Scripts can access rich contextual information:Action Buttons
-
Update Filter / Create Filter:
- Saves the filter configuration, making it active for future data processing.
-
Back to Filters:
- Returns to the Filters List View without saving changes.
Purpose and Benefits
-
Data Governance:
- Enforces strict control over what data can be sent to LLMs, ensuring compliance with privacy regulations.
-
Flexibility:
- Filters can be tailored to specific organizational needs using custom scripts.
-
Security:
- Prevents sensitive or unauthorized data from leaking to unapproved vendors or external systems.
Example Middleware for Tools
Middleware filters in the Tyk AI Studio modify data coming from tools before passing it to the LLM. These filters are applied to sanitize, anonymize, or enhance the data to ensure it complies with organizational standards and privacy regulations. Below is an example of a middleware filter that sanitizes Personally Identifiable Information (PII), specifically email addresses, from the tool’s output.Middleware Script: Email Redaction Example
Explanation of the Script
-
Module Import:
- The
textmodule is imported to enable regular expression operations (text.re_replace).
- The
-
Regex Pattern:
- A regex pattern is defined to detect email addresses:
- Example pattern:
[\w\.-]+@[\w\.-]+\.\w+ - This pattern matches standard email formats.
- Example pattern:
- A regex pattern is defined to detect email addresses:
-
Filter Function:
- The
filterfunction accepts an input string (e.g., tool output) and:- Uses
text.re_replaceto identify email addresses. - Replaces detected email addresses with
[REDACTED EMAIL].
- Uses
- The
-
Return Processed Output:
- The sanitized output is returned, ensuring that sensitive information like email addresses is redacted before reaching the LLM.
Use Case for Middleware
Tool Example: Imagine a tool, such asSupport Ticket Viewer, which retrieves user tickets from a system. These tickets often contain email addresses. Middleware ensures that no sensitive email information is included in the output sent to the LLM.
-
Input Payload Example:
-
Sanitized Output:
Benefits of Middleware
-
Data Privacy:
- Protects sensitive user information by ensuring it is sanitized before being sent to external systems.
-
Compliance:
- Ensures organizational adherence to privacy laws like GDPR or HIPAA.
-
Enhanced Security:
- Prevents accidental sharing of PII with external vendors or LLMs.
Available Tengo Modules
Filters have access to powerful standard library modules:text - String Operations
json - JSON Operations
fmt - Formatting and Printing
tyk - Extended Capabilities (Enterprise)
Tool Response Filters
Filters can also be applied to tool responses (e.g., API calls, database queries). Tool responses are plain strings, not JSON-structured messages, so they require simpler handling.Example 1: Block Tool Responses Containing Errors
Example 2: Redact PII from Tool Responses
Example 3: Filter Based on Tool Name
tyk.redact_pattern() helper is designed for LLM message structures (JSON format) and will not work with tool responses. For tool responses, use direct string manipulation with the text module as shown above.
Filter Execution Order in Chat Sessions
When a user sends a message in a chat session, filters are executed at multiple points in the pipeline:1. User Message Filters (Before RAG)
When: After preprocessing, before RAG vector search Purpose: Redact PII from user messages before they’re used for vector similarity search Context:input.messages[0].role == "user"
- ✅ RAG vector similarity search
- ✅ Subsequent LLM processing
- ✅ Chat history storage
2. File Content Filters (Before RAG)
When: When files are attached to messages Purpose: Filter sensitive content from uploaded files before indexing Context:input.context.file_ref contains the file reference
3. Tool Response Filters (After Tool Execution)
When: After a tool returns data, before sending to LLM Purpose: Filter sensitive data from external API responses Context:input.messages[0].role == "tool", input.context.tool_name available
Best Practices
- Always define
output- Scripts must set the output variable - Use
tyk.redact_patternfor LLM messages - Handles vendor differences automatically - Use
textmodule for tool responses - Direct string manipulation - Use messages array for complex LLM modifications - Gives you full control
- Provide clear block messages - Help users understand policy violations
- Test across vendors - OpenAI, Anthropic, and Google AI have different formats
- Check message roles - Different logic for system, user, assistant, and tool messages
- Handle edge cases - Empty arrays, missing fields, etc.
- Consider RAG impact - Filters run before RAG, so redactions affect vector search
Migration from Legacy API
Old API (still supported for backward compatibility):Response Filters
Response Filters enable administrators to block LLM responses based on content analysis, providing governance controls on what LLMs can say to end users.Key Characteristics
- Block-Only: Response filters can only block responses, not modify them
- Works on LLM Responses: Applied to LLM responses only (not tool responses)
- Streaming Support: Execute per-chunk during streaming with access to accumulated buffer
- Script-Controlled: Filter scripts decide when to evaluate based on buffer length
Configuration
Enable response filtering by checking “Is this a Response Filter?” when creating or editing a filter in the admin UI.Script API for Response Filters
Response filters use the sameScriptInput/ScriptOutput structure as request filters, with additional response-specific fields:
Input Object (Non-Streaming):
payload and messages fields in output are ignored for response filters.
Example 1: Block Refund Promises (Works for Streaming and Non-Streaming)
Example 2: Block Harmful Content (Streaming with Buffer Check)
Example 3: Combined with LLM Analysis
Use another LLM to analyze the response for policy violations:Response Filter Execution
Proxy (REST):- Executes after response hooks (if any)
- Full response available in
input.raw_input - If blocked: Error returned to client instead of response
- Executes on every chunk
- Access to both
raw_input(current chunk) andcurrent_buffer(accumulated text) - If blocked: Streaming stops, error sent to client
- Executes before adding to chat history
- If blocked: Error published to queue instead of response
- Executes on every chunk before publishing to queue
- If blocked: Error published, streaming callback returns error to stop further chunks
Best Practices for Response Filters
- Buffer Management (Streaming): Script controls evaluation timing based on
len(current_buffer) - Performance: Keep filter logic lightweight (executes per-chunk in streaming)
- Clear Messages: Provide helpful block messages for users
- Fail Open: Filters fail open on script errors (response allowed through)
- LLM-Only: Response filters only work on LLM responses, not tool responses
- Block-Only: Cannot modify responses, only block them
When to Use Response Filters vs Request Filters
Use Request Filters When:- Preventing sensitive data from reaching the LLM
- Redacting PII before LLM processing
- Enforcing input policies
- Preventing LLMs from making commitments (refunds, promises)
- Blocking harmful or inappropriate LLM outputs
- Enforcing corporate communication policies
- Detecting policy violations in LLM responses
This unified filter system demonstrates how flexible and powerful Midsommar’s scripting capabilities are, enabling administrators to enforce strict data governance policies while supporting advanced LLM and tool integration workflows with both blocking and modification capabilities.