Amazon S3 Connector

Object storage on AWS. Also supports any S3-compatible service that speaks the same protocol (MinIO, Cloudflare R2, DigitalOcean Spaces, Wasabi, etc.).

Connection form

Set the endpoint to use an S3-compatible service. When endpoint is set, DagFlux automatically switches to path-style URLs, which is what self-hosted S3 services like MinIO require.

Setting up credentials (AWS S3)

Best practice is to create a dedicated IAM user (or role with access keys) scoped to only the buckets DagFlux needs.

Step 1 — Open the AWS IAM console

Go to IAM → Users in the AWS console.

Step 2 — Create a user

1. Click Create user.
2. Name it (e.g. dagflux-s3).
3. Pick Access key — Programmatic access. (Don't enable AWS Management Console access; this user only needs API access.)

Step 3 — Attach a least-privilege policy

Instead of attaching AmazonS3FullAccess, attach a custom policy scoped to the buckets DagFlux should read and write to. Replace my-bucket with your bucket names.

Read + write on a single bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBuckets",
      "Effect": "Allow",
      "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
      "Resource": "*"
    },
    {
      "Sid": "ReadWriteBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetObjectVersion",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

Read-only on a single bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBuckets",
      "Effect": "Allow",
      "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
      "Resource": "*"
    },
    {
      "Sid": "ReadBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3:::my-bucket",
        "arn:aws:s3:::my-bucket/*"
      ]
    }
  ]
}

Restrict to a single prefix (folder) inside a bucket:

Replace the Resource in the read/write block with:

"Resource": [
  "arn:aws:s3:::my-bucket",
  "arn:aws:s3:::my-bucket/dagflux-data/*"
]

…and add a Condition to the s3:ListBucket action that limits the listing to that prefix.

Step 4 — Create the access key

1. Open the user.
2. Go to the Security credentials tab.
3. Under Access keys, click Create access key.
4. Pick Application running outside AWS.
5. Save the Access Key ID and Secret Access Key — the secret is shown only once.

Step 5 — Enter the credentials in DagFlux

• Access Key ID and Secret Access Key — from step 4.
• Region — the default region for your buckets (e.g. us-east-1, eu-west-1).
• Custom Endpoint — leave empty for AWS S3.

Setting up credentials (MinIO / S3-compatible)

For MinIO and other S3-compatible services:

1. In the service's admin console, create an access key and secret access key with the appropriate bucket policies.
2. Note the service's HTTPS endpoint (e.g. https://minio.mycompany.com).
3. In DagFlux, fill in the access key, secret access key, region (often us-east-1 for MinIO), and the Custom Endpoint field.

Behaviour notes

• Object listings use a forward-slash delimiter, so "folders" inside a bucket appear as their own entries.
• Listings paginate transparently — DagFlux pulls additional pages as you scroll.
• Search is performed server-side by combining the prefix with the search term, which is fast even on very large buckets.