Grafana Dashboards¶

Connect HolmesGPT to Grafana for dashboard analysis, visual rendering, query extraction, and understanding your monitoring setup. When the Grafana Image Renderer is installed, HolmesGPT can visually render dashboards and panels to detect anomalies like spikes, trends, and outliers.

Prerequisites¶

A Grafana service account token with the following permissions:

Basic role → Viewer

For visual rendering, the Grafana Image Renderer plugin must be installed on your Grafana instance and enable_rendering: true must be set in the config. HolmesGPT auto-detects the renderer — if it's not installed, visual rendering tools are simply not registered and everything else works normally.

Configuration¶

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export GRAFANA_API_KEY=your-grafana-service-account-token

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_key: "{{ env.GRAFANA_API_KEY }}"
      api_url: <your grafana url>  # e.g. https://acme-corp.grafana.net, or http://localhost:3000 for a local Grafana
      # Optional: Additional headers for all requests
      # additional_headers:
      #   X-Custom-Header: "custom-value"

After making changes to your configuration, run:

holmes toolset refresh

To test, run:

holmes ask "Show me all dashboards tagged with 'kubernetes'"

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-grafanadashboards \
  --from-literal=GRAFANA_API_KEY=your-grafana-service-account-token \
  -n <namespace>

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-grafanadashboards

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_key: "{{ env.GRAFANA_API_KEY }}"
      api_url: <your grafana url>  # e.g. https://acme-corp.grafana.net, or http://localhost:3000 for a local Grafana
      # Optional: Additional headers for all requests
      # additional_headers:
      #   X-Custom-Header: "custom-value"

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

Create a Kubernetes secret in the namespace Holmes runs in:

kubectl create secret generic holmes-grafanadashboards \
  --from-literal=GRAFANA_API_KEY=your-grafana-service-account-token \
  -n <namespace>

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-grafanadashboards

  toolsets:
    grafana/dashboards:
      enabled: true
      config:
        api_key: "{{ env.GRAFANA_API_KEY }}"
        api_url: <your grafana url>  # e.g. https://acme-corp.grafana.net, or http://localhost:3000 for a local Grafana
        # Optional: Additional headers for all requests
        # additional_headers:
        #   X-Custom-Header: "custom-value"

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Multiple Instances¶

The Grafana toolset can connect to more than one Grafana instance. List each one under instances: with a unique name. Any config field set outside instances: becomes a default that every instance inherits, so shared settings only need to be written once.

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      instances:
        - name: prod
          api_url: <your grafana url>
          api_key: <your grafana service account token>
        - name: staging
          api_url: <your grafana url>
          api_key: <your grafana service account token>

When more than one instance is configured, HolmesGPT automatically adds an instance parameter to every Grafana tool (so it can pick which instance to query) and a grafana_dashboards_list_instances tool to list the configured instances. With a single instance — including the flat config without instances: — the tools are unchanged and fully backwards compatible.

See Multiple Instances for the full behaviour, including global defaults and health reporting.

Visual Rendering¶

When the Grafana Image Renderer is available, HolmesGPT can take screenshots of dashboards and panels and analyze them using the LLM's vision capabilities. This is useful for:

Spotting anomalous spikes or patterns across many panels at once
Analyzing visual dashboard layouts without parsing raw query data
Investigating dashboards that use complex visualizations (heatmaps, gauges, etc.)

The LLM controls all rendering parameters — time range, dimensions, theme, timezone, and template variables — so it can zoom in on specific time windows or adjust the view as needed during investigation.

Rendering is disabled by default. To enable it, add enable_rendering: true to your config:

In Kubernetes, this reuses the holmes-grafanadashboards secret created in the Configuration section above.

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export GRAFANA_API_KEY=your-grafana-service-account-token

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: <your grafana url>
      api_key: "{{ env.GRAFANA_API_KEY }}"
      enable_rendering: true

After making changes to your configuration, run:

holmes toolset refresh

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-grafanadashboards

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: <your grafana url>
      api_key: "{{ env.GRAFANA_API_KEY }}"
      enable_rendering: true

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-grafanadashboards

  toolsets:
    grafana/dashboards:
      enabled: true
      config:
        api_url: <your grafana url>
        api_key: "{{ env.GRAFANA_API_KEY }}"
        enable_rendering: true

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

When rendering a full dashboard, HolmesGPT captures the entire page (all rows) so that panels at the bottom are not cropped.

Advanced Configuration¶

SSL Verification¶

For self-signed certificates, you can disable SSL verification:

In Kubernetes, this reuses the holmes-grafanadashboards secret created in the Configuration section above.

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export GRAFANA_API_KEY=your-grafana-service-account-token

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: https://grafana.internal
      api_key: "{{ env.GRAFANA_API_KEY }}"
      verify_ssl: false  # Disable SSL verification (default: true)

After making changes to your configuration, run:

holmes toolset refresh

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-grafanadashboards

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: https://grafana.internal
      api_key: "{{ env.GRAFANA_API_KEY }}"
      verify_ssl: false  # Disable SSL verification (default: true)

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-grafanadashboards

  toolsets:
    grafana/dashboards:
      enabled: true
      config:
        api_url: https://grafana.internal
        api_key: "{{ env.GRAFANA_API_KEY }}"
        verify_ssl: false  # Disable SSL verification (default: true)

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

External URL¶

If HolmesGPT accesses Grafana through an internal URL but you want clickable links in results to use a different URL:

In Kubernetes, this reuses the holmes-grafanadashboards secret created in the Configuration section above.

Holmes CLIHolmes Helm ChartRobusta Helm Chart

Set the environment variable:

export GRAFANA_API_KEY=your-grafana-service-account-token

Add the following to ~/.holmes/config.yaml. Create the file if it doesn't exist:

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: http://grafana.internal:3000  # Internal URL for API calls
      external_url: https://grafana.example.com  # URL for links in results
      api_key: "{{ env.GRAFANA_API_KEY }}"

After making changes to your configuration, run:

holmes toolset refresh

When using the standalone Holmes Helm Chart, update your values.yaml:

extraEnvVarsSecrets:
  - holmes-grafanadashboards

toolsets:
  grafana/dashboards:
    enabled: true
    config:
      api_url: http://grafana.internal:3000  # Internal URL for API calls
      external_url: https://grafana.example.com  # URL for links in results
      api_key: "{{ env.GRAFANA_API_KEY }}"

Apply the configuration:

helm upgrade holmes robusta/holmes -f values.yaml

When using the Robusta Helm Chart (which includes HolmesGPT), update your generated_values.yaml:

holmes:
  extraEnvVarsSecrets:
    - holmes-grafanadashboards

  toolsets:
    grafana/dashboards:
      enabled: true
      config:
        api_url: http://grafana.internal:3000  # Internal URL for API calls
        external_url: https://grafana.example.com  # URL for links in results
        api_key: "{{ env.GRAFANA_API_KEY }}"

Apply the configuration:

helm upgrade robusta robusta/robusta -f generated_values.yaml --set clusterName=<YOUR_CLUSTER_NAME>

Common Use Cases¶

holmes ask "Find all dashboards tagged with 'production' or 'kubernetes'"

holmes ask "Show me what metrics the 'Node Exporter' dashboard monitors"

holmes ask "Get the CPU usage queries from the Kubernetes cluster dashboard and check if any nodes are throttling"

holmes ask "Look at the Platform Services dashboard and tell me if any panels show anomalous spikes"

holmes ask "Render the checkout latency panel from the last 24 hours and analyze the trend"