Audit reporting¶

Otto Chriek spent six months checking 2,847 deployments by hand before OPA was deployed. He will not do that again. OPA’s decision logging captures every policy evaluation as a structured JSON record: who asked, what they asked for, what the policy decided, and why. These records flow to Graylog via Filebeat, are queryable for compliance reporting, and form the evidentiary basis for Golem Trust’s ISO 27001 certification. Otto generates the monthly compliance report in approximately four minutes; he has timed it. This runbook covers the decision log format, Filebeat forwarding, the Graylog dashboard, and the compliance report generation procedure.

OPA decision logging configuration¶

Decision logging is configured in /etc/opa/config.yaml on the standalone OPA server. Logs are written to a local file and also sent to a remote service:

decision_logs:
  console: false
  plugin: file

plugins:
  file:
    path: /var/log/opa/decisions.log

services:
  graylog-decisions:
    url: https://graylog.golems.internal:12900/api/inputs/opa-decisions
    credentials:
      bearer:
        token_path: /var/run/opa/graylog-token

Each decision log entry is a JSON object on a single line:

{
  "decision_id": "7a3f2c1d-4b8e-41a2-9f00-bb3e12345678",
  "input": {
    "requestor": "angua@golems.internal",
    "resource": {"name": "royalbank-prod-postgres", "environment": "production"},
    "approvals": [...]
  },
  "result": true,
  "timestamp": "2025-11-14T09:15:32.441Z",
  "path": "golem_trust/strongdm/access",
  "requested_by": "strongdm-gateway-01.golems.internal",
  "metrics": {
    "counter_regovm_eval_op_count": 42,
    "timer_rego_query_eval_ns": 183000
  },
  "erased": ["input/approvals/0/approver_password"]
}

The erased field lists input paths that were stripped before logging. This is configured per policy using the OPA masking feature, to avoid logging sensitive fields such as passwords or tokens.

Masking sensitive fields¶

Add a masking rule in the decision log configuration to strip fields that must not appear in audit logs:

decision_logs:
  mask_decision: golem_trust/mask/decision

data:
  golem_trust:
    mask:
      decision:
        - "/input/password"
        - "/input/secret_id"
        - "/input/approvals/*/approver_token"

Filebeat forwarding to Graylog¶

Filebeat runs on each OPA server VM and tails the decision log file. The Filebeat configuration at /etc/filebeat/inputs.d/opa-decisions.yml:

- type: filestream
  id: opa-decisions
  paths:
    - /var/log/opa/decisions.log
  parsers:
    - ndjson:
        target: opa
        add_error_key: true
  fields:
    source: opa-decisions
    environment: production
  fields_under_root: true
  processors:
    - timestamp:
        field: opa.timestamp
        layouts:
          - "2006-01-02T15:04:05.999Z"
        target_field: "@timestamp"

Graylog receives these events on the opa-decisions stream. The stream has a 90-day retention policy, which exceeds the ISO 27001 requirement for audit trail completeness.

Graylog dashboard¶

The “OPA Policy Decisions” dashboard in Graylog provides the operational view. The key widgets and their queries are:

Allow/deny ratio over time:

source:opa-decisions

Chart by result field, last 24 hours.

Top denied policies (last 7 days):

source:opa-decisions AND result:false

Aggregated by path field, top 10.

Policy decision latency percentiles:

source:opa-decisions

Statistical aggregation on metrics.timer_rego_query_eval_ns, converted to milliseconds.

Decisions by requestor:

source:opa-decisions

Aggregated by requested_by, used to identify services generating unexpected policy queries.

Generating compliance reports¶

Otto Chriek’s ISO 27001 evidence requirement is: “show me every production deployment in the past 30 days and confirm that each had at least two approvals.” The report is generated by querying Graylog’s REST API and processing the decision log:

#!/usr/bin/env python3
import requests
import json
from datetime import datetime, timedelta

GRAYLOG_URL = "https://graylog.golems.internal:12900"
STREAM_ID   = "opa-decisions-stream-id"
DAYS        = 30

start = (datetime.utcnow() - timedelta(days=DAYS)).strftime("%Y-%m-%dT%H:%M:%SZ")

resp = requests.get(
    f"{GRAYLOG_URL}/api/search/universal/relative",
    params={
        "query": 'source:opa-decisions AND path:golem_trust\\/strongdm\\/access AND result:true',
        "range": DAYS * 86400,
        "limit": 5000,
        "fields": "timestamp,input.requestor,input.resource.name,result,decision_id"
    },
    auth=("otto.chriek", open("/run/secrets/graylog-token").read().strip()),
    headers={"Accept": "application/json"}
)

decisions = resp.json()["messages"]

report = {
    "generated_at": datetime.utcnow().isoformat() + "Z",
    "period_days": DAYS,
    "total_decisions": len(decisions),
    "production_approvals": [
        {
            "timestamp": m["message"]["timestamp"],
            "requestor": m["message"].get("input.requestor"),
            "resource": m["message"].get("input.resource.name"),
            "decision_id": m["message"]["decision_id"]
        }
        for m in decisions
    ]
}

with open(f"compliance-report-{datetime.utcnow().strftime('%Y-%m')}.json", "w") as f:
    json.dump(report, f, indent=2)

print(f"Report generated: {len(decisions)} approved production access decisions in the past {DAYS} days")

Run this script on the first working day of each month. The output JSON file is uploaded to the iso27001-evidence bucket in Hetzner Object Storage by the CI pipeline job compliance:generate-monthly-report.

Monthly summary report format¶

The report Otto sends to the ISO auditor is a human-readable summary generated from the JSON file. The key sections are:

Total policy decisions evaluated in the period
Number of denied decisions by policy path
Number of approved production deployments with approval counts
Any anomalies: decisions where the approval count was exactly the minimum required, flagged for manual review
Attestation: “No production deployments occurred outside the policy-controlled pipeline during this period, as evidenced by the OPA decision log retained in Graylog”

Otto notes that producing this report used to take three days. It now takes four minutes, of which three are waiting for the Graylog query to return.