log export.md

Export audit logs to your S3 bucket

Stream Barndoor audit events to your own S3-compatible storage in near real time.

Overview

Barndoor can stream every audit event from your tenant — authentication, authorization decisions, AI agent requests, tool calls, policy changes, and more — to an S3-compatible bucket you own. Events are batched, gzipped, and uploaded as JSON Lines files partitioned by date and hour. Once configured, logs land in your bucket within a minute of being generated and remain under your retention and access policies.

This guide walks through configuring a destination, choosing an authentication method, and verifying that events are flowing.

Estimated time: 10–20 minutes (longer if you need to provision a new bucket or IAM role)

Before You Begin

You'll need:

If your bucket is in AWS S3, we strongly recommend IAM-role authentication over access keys. It avoids handing long-lived credentials to Barndoor and is the standard pattern for cross-account access in AWS.

Authentication Methods

Barndoor supports two ways to authenticate to your bucket. Pick one when you configure the destination — you can switch later without recreating the export.

IAM role (AWS only) Access keys
Cloud support AWS S3 only AWS S3, GCS, MinIO, SeaweedFS, any S3-compatible service
Credentials in Barndoor None — we assume your role on every batch via STS Long-lived access_key_id / secret_access_key stored in our secret vault
Rotation Automatic — Barndoor mints short-lived credentials each batch Manual — you rotate the keys and re-paste them into the Barndoor UI
Setup effort Create one IAM role with trust + permissions policies Create one IAM user and generate access keys

Configure with IAM Role Authentication

This is the recommended path for AWS S3 destinations.

Step 1: Start the configuration in Barndoor

  1. Sign in to your Barndoor portal and navigate to Audit Log Export.
  2. Fill in the connection details:
    • Endpoint URL — https://s3.<region>.amazonaws.com (e.g. https://s3.us-east-1.amazonaws.com)
    • Bucket Name — the bucket events will land in
    • Region — matching AWS region (e.g. us-east-1)
    • Path Prefix (optional) — key prefix for every object. Defaults to audit-events/.
  3. Under Authentication, choose IAM role (AWS only).

The UI will display two values you'll need in the next step:

Leave this tab open while you set up the role in AWS.

Step 2: Create the IAM role in your AWS account

In the AWS console, go to IAM → Roles → Create role:

  1. Trusted entity type: Custom trust policy. Paste the snippet below, substituting <barndoor-principal> and <external-id> with the values from the Barndoor UI:
   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Principal": { "AWS": "<barndoor-principal>" },
         "Action": "sts:AssumeRole",
         "Condition": {
           "StringEquals": { "sts:ExternalId": "<external-id>" }
         }
       }
     ]
   }
  1. Permissions: skip the AWS managed-policies screen — we'll add an inline policy after the role is created.
  2. Name: any name you like, e.g. barndoor-audit-export. Copy the role ARN once it's created.
  3. On the role's detail page, Add permissions → Create inline policy. Paste the snippet below, replacing <your-bucket-name>:
   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:PutObject",
           "s3:ListBucket",
           "s3:GetBucketLocation"
         ],
         "Resource": [
           "arn:aws:s3:::<your-bucket-name>",
           "arn:aws:s3:::<your-bucket-name>/*"
         ]
       }
     ]
   }

ListBucket and GetBucketLocation are used by Barndoor's "Check connection" health check. PutObject is what audit-consumer uses at runtime to upload event files.

Step 3: Paste the role ARN back into Barndoor

Back in the Audit Log Export page:

  1. Paste the role's ARN into IAM Role ARN.
  2. Pick the event types to include (see Selecting event types below) or leave the default (all types).
  3. Click Save Configuration. The destination saves in a paused state so you can verify the connection before any events flow.
  4. Click Check Connection. Barndoor assumes your role, performs a HeadBucket against your destination, and reports the result.

If you see Connection Healthy, you're done with setup. If not, jump to Troubleshooting.

Step 4: Start the stream

Once the connection is healthy, click Start in the Current Status card. The badge flips to Streaming Active and new audit events begin flowing to your bucket within ~30 seconds.

Configure with Access Keys

Use this path for non-AWS S3-compatible destinations or when an IAM user fits your team's workflow better.

Step 1: Create the IAM user / equivalent

In AWS:

  1. IAM → Users → Create user, name it (e.g. barndoor-audit-export).
  2. Skip the AWS Management Console access option — this user only needs programmatic access.
  3. Once the user is created, Security credentials → Create access key → Application running outside AWS (or equivalent). Save the access key and secret.
  4. Add inline policy with the same S3 permissions snippet from Step 2 of the IAM-role flow.

For non-AWS S3 services (GCS HMAC, MinIO, SeaweedFS), follow the service's instructions for creating an access key pair scoped to your bucket.

Step 2: Paste the keys into Barndoor

  1. Navigate to Audit Log Export and fill in Endpoint URL, Bucket Name, Region, Path Prefix.
  2. Under Authentication, choose Access keys.
  3. Paste the Access Key ID and Secret Access Key.
  4. Click Save Configuration, then Check Connection, then Start — same flow as the IAM-role path.

The access key and secret never leave Barndoor's encrypted secret store and are never displayed again in the UI. To rotate, edit the destination and paste new values.

Selecting Event Types

The right-hand panel of the configuration form lets you pick which event categories to export. By default all 16 event types are included.

Available categories:

Click a category header to expand it and toggle individual event types. The badge shows how many you've selected out of the total. Select all / Clear all reset the selection for the whole list.

Event-type filters apply only to future events. Changing them doesn't backfill or remove already-exported objects.

Object Format and Partitioning

Exported objects use this layout:

<bucket>/<path-prefix>/dt=YYYY-MM-DD/hour=HH/archive_HHMMSS.<seq>.<batch-id>.json.gz

For example, with the default prefix audit-events/:

my-bucket/audit-events/dt=2026-05-15/hour=07/archive_073825.0000.96568337-7073-4ab6-b431-c7d3d7251877.json.gz

Inside each gzipped file is one JSON object per line, in a shape compatible with Datadog's log-ingestion format. Trace and span IDs are included on events that originated within a traced request so you can correlate audit logs with traces in your observability stack.

Batches flush every 30 seconds or when the buffer reaches 100 events, whichever comes first.

Pausing and Resuming

The Pause / Start button on the destination page is independent of the destination configuration. Pausing buffers new audit events on the Barndoor side; resuming drains the buffer to your destination. You can also delete the destination entirely with Remove — this clears the stored credentials (for access-keys mode) and stops exports until a new destination is configured.

Troubleshooting

"Connection Unhealthy" after Check Connection

Look at the message under the badge. Common causes:

"Last delivery issue: ... AccessDenied" but Check Connection passes

Health check uses HeadBucket (ListBucket / GetBucketLocation permissions). Actual uploads use PutObject. If health check is green but uploads fail with AccessDenied, your role / user is missing s3:PutObject. Add it to the inline permissions policy.

No objects appearing in the bucket even though the stream is "Active"

Audit events are only emitted when traffic flows through Barndoor. If you've just configured an empty test organization and there's no activity, the buffer may genuinely have nothing to upload. Generate some events by logging in/out, navigating around the portal, or making an API call against your tenant.

Switching between IAM-role and access-keys mode

Editing an existing destination to switch modes is supported. When you switch from access keys to IAM role, Barndoor automatically deletes the stored access keys from its vault. When you switch the other way, you'll be prompted to enter new access keys.

Frequently Asked Questions

What happens if my IAM role's trust policy or bucket permissions change?

The next upload attempt will fail. The destination's status banner in the Barndoor UI will show the AWS error message. Re-check the trust and permissions policies.

Can the same IAM role serve multiple Barndoor organizations?

Each Barndoor destination has its own External ID, so even within a single AWS account a single role can support multiple Barndoor tenants by allowing multiple sts:ExternalId values in the trust policy. In practice we recommend one role per Barndoor organization for clean auditing.

Is there a backfill option for historical events?

Not currently — exports start from the moment you click Start. Historical events remain queryable in the Barndoor portal but aren't replayed to your bucket.

What's the data retention on Barndoor's side while paused?

Buffered events are retained for 30 days while the stream is paused. If you resume within that window, all buffered events stream out in order. If you exceed it, older buffered events are dropped.