HTTP Header Propagation

When running HolmesGPT as a server, HTTP headers from incoming requests can be forwarded to toolsets when they make outgoing API calls. This is useful for passing per-request authentication tokens, tenant identifiers, or other contextual headers through to backend services.

Header propagation is supported across all toolset types: MCP servers, HTTP connectors, custom (YAML) toolsets, and built-in Python toolsets.

Note

Header propagation is only available when running Holmes as a server (Helm deployment). It does not apply when using the CLI directly.

How It Works

1. A client sends an HTTP request to the Holmes server (e.g., /api/investigate)
2. Holmes extracts non-sensitive headers from the request (blocking Authorization, Cookie, and Set-Cookie by default)
3. The extracted headers are available as request_context during tool execution
4. Toolsets that opt in render those templates using the request context and forward the resulting values to their tools at invocation time. Toolsets without the config key are unaffected.

Configuration

The config key is placed inside the config section of each toolset and accepts a dictionary of names mapped to Jinja2 template strings:

• extra_headers -- for MCP servers, HTTP connectors, and Python toolsets (values become HTTP headers)
• For custom (YAML) toolsets, request_context and env are available directly in Jinja2 command/script templates — no extra config key needed

Templates can reference:

• {{ request_context.headers['Header-Name'] }} -- a header from the incoming HTTP request (case-insensitive lookup)
• {{ env.ENV_VAR }} -- an environment variable
• Plain strings -- static values that don't need rendering

Toolset Examples

MCP Servers

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

mcp_servers:
  customer_data:
    description: "Customer data API"
    config:
      url: "http://customer-api:8000/mcp"
      mode: streamable-http
      extra_headers:
        X-Tenant-Id: "{{ request_context.headers['X-Tenant-Id'] }}"
        X-Auth-Token: "{{ request_context.headers['X-Auth-Token'] }}"

After making changes to your configuration, run:

holmes toolset refresh

When using the standalone Holmes Helm Chart, update your values.yaml:

mcp_servers:
  customer_data:
    description: "Customer data API"
    config:
      url: "http://customer-api:8000/mcp"
      mode: streamable-http
      extra_headers:
        X-Tenant-Id: "{{ request_context.headers['X-Tenant-Id'] }}"
        X-Auth-Token: "{{ request_context.headers['X-Auth-Token'] }}"

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  mcp_servers:
    customer_data:
      description: "Customer data API"
      config:
        url: "http://customer-api:8000/mcp"
        mode: streamable-http
        extra_headers:
          X-Tenant-Id: "{{ request_context.headers['X-Tenant-Id'] }}"
          X-Auth-Token: "{{ request_context.headers['X-Auth-Token'] }}"

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

See MCP Servers -- Dynamic Headers for the full MCP configuration reference.

HTTP Connectors

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export INTERNAL_API_KEY=your-internal-api-key

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  internal-api:
    type: http
    enabled: true
    config:
      extra_headers:
        X-Request-Id: "{{ request_context.headers['X-Request-Id'] }}"
        X-Api-Key: "{{ env.INTERNAL_API_KEY }}"
      endpoints:
        - hosts: ["internal-api.corp.net"]
          methods: ["GET"]

After making changes to your configuration, run:

holmes toolset refresh

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-header-propagation \
  --from-literal=INTERNAL_API_KEY=your-internal-api-key \
  -n <namespace>

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-header-propagation

toolsets:
  internal-api:
    type: http
    enabled: true
    config:
      extra_headers:
        X-Request-Id: "{{ request_context.headers['X-Request-Id'] }}"
        X-Api-Key: "{{ env.INTERNAL_API_KEY }}"
      endpoints:
        - hosts: ["internal-api.corp.net"]
          methods: ["GET"]

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-header-propagation \
  --from-literal=INTERNAL_API_KEY=your-internal-api-key \
  -n <namespace>

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-header-propagation

  toolsets:
    internal-api:
      type: http
      enabled: true
      config:
        extra_headers:
          X-Request-Id: "{{ request_context.headers['X-Request-Id'] }}"
          X-Api-Key: "{{ env.INTERNAL_API_KEY }}"
        endpoints:
          - hosts: ["internal-api.corp.net"]
            methods: ["GET"]

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

The rendered headers are merged into every outgoing request after the endpoint's own authentication headers, so they can override defaults when needed.

See HTTP Connectors for the full HTTP connector configuration reference.

Custom (YAML) Toolsets

YAML tool commands and scripts are Jinja2 templates. The variables request_context and env are available directly — no extra config key needed. Use request_context.headers['Header-Name'] to access incoming request headers and env.VAR_NAME to access environment variables.

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Configuration File (toolsets.yaml):

toolsets:
  internal-api:
    description: "Fetch data from the internal API"
    tools:
      - name: "fetch_data"
        description: "Fetch data from internal API"
        command: 'curl -s -H "X-Auth-Token: {{ request_context.headers[''X-Auth-Token''] }}" https://internal-api.corp.net/data'

holmes ask "fetch data from the internal API" --custom-toolsets=toolsets.yaml

When using the standalone Holmes Helm Chart, update your values.yaml:

toolsets:
  internal-api:
    description: "Fetch data from the internal API"
    tools:
      - name: "fetch_data"
        description: "Fetch data from internal API"
        command: 'curl -s -H "X-Auth-Token: {{ request_context.headers[''X-Auth-Token''] }}" https://internal-api.corp.net/data'

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  toolsets:
    internal-api:
      description: "Fetch data from the internal API"
      tools:
        - name: "fetch_data"
          description: "Fetch data from internal API"
          command: 'curl -s -H "X-Auth-Token: {{ request_context.headers[''X-Auth-Token''] }}" https://internal-api.corp.net/data'

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

See Custom Toolsets for the full YAML toolset reference.

Built-in Python Toolsets

Each built-in Python toolset decides individually whether to support extra_headers. Not all toolsets support it — only those whose request method renders extra_headers via render_header_templates() and merges the result into outgoing HTTP calls.

The following example shows how ServiceNow Tables, one toolset that supports header propagation, is configured:

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export SERVICENOW_API_KEY=your-servicenow-api-key

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  servicenow/tables:
    config:
      extra_headers:
        X-Correlation-Id: "{{ request_context.headers['X-Correlation-Id'] }}"
      api_key: "{{ env.SERVICENOW_API_KEY }}"
      api_url: "https://instance.service-now.com"

After making changes to your configuration, run:

holmes toolset refresh

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-header-propagation-servicenow \
  --from-literal=SERVICENOW_API_KEY=your-servicenow-api-key \
  -n <namespace>

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-header-propagation-servicenow

toolsets:
  servicenow/tables:
    config:
      extra_headers:
        X-Correlation-Id: "{{ request_context.headers['X-Correlation-Id'] }}"
      api_key: "{{ env.SERVICENOW_API_KEY }}"
      api_url: "https://instance.service-now.com"

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-header-propagation-servicenow \
  --from-literal=SERVICENOW_API_KEY=your-servicenow-api-key \
  -n <namespace>

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-header-propagation-servicenow

  toolsets:
    servicenow/tables:
      config:
        extra_headers:
          X-Correlation-Id: "{{ request_context.headers['X-Correlation-Id'] }}"
        api_key: "{{ env.SERVICENOW_API_KEY }}"
        api_url: "https://instance.service-now.com"

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

For a reference implementation showing how to add extra_headers support to a Python toolset, see servicenow_tables.py.

Sending Headers to Holmes

Include your custom headers alongside the normal request:

curl -X POST http://holmes-server/api/investigate \
  -H "Content-Type: application/json" \
  -H "X-Auth-Token: your-token-here" \
  -H "X-Tenant-Id: tenant-42" \
  -d '{"question": "Check system status"}'

Blocked Headers

By default, the following headers are not forwarded from the incoming request to the request_context:

• Authorization
• Cookie
• Set-Cookie

You can override this list with the HOLMES_PASSTHROUGH_BLOCKED_HEADERS environment variable (comma-separated, case-insensitive):

# Block only Authorization (allow cookies through)
export HOLMES_PASSTHROUGH_BLOCKED_HEADERS="authorization"

# Block additional headers
export HOLMES_PASSTHROUGH_BLOCKED_HEADERS="authorization,cookie,set-cookie,x-internal-only"

Precedence

When multiple header sources exist, later layers override earlier ones:

1. Toolset's own authentication headers (e.g., API key, bearer token)
2. LLM-provided headers (HTTP connector only, via the headers tool parameter)
3. extra_headers (rendered templates from the config section)