# Your own S3 bucket

A storage connection lets the API read sources from your Amazon S3 bucket and write results into it. You create an IAM role in your AWS account that the API may assume; you never share access keys, and the role can reach only the folders (key prefixes) you list.

Before you start:

- Your account must have bought credits. A free account gets `403 not_enabled_for_account`.
- The key needs the `storage` scope (a key made without scopes has it) for the setup calls. Requests that use the connection need only `process`.
- S3 only. For R2, Google Cloud Storage or Azure, write results to a [presigned URL](https://api.watermark.ws/docs/files.md#destinations) instead.
- An account may have 10 connections.

## How it works

1. You create the connection with `POST /v1/storage`: the bucket, its region, the ARN of the role the API should assume, and the prefixes it may use. The API answers with an `external_id` made for this connection and the two policies the role needs.
2. You create the role with those policies. The **trust policy** lets the API's own role assume yours, and only with this connection's `external_id`, so no other Watermark account can point the API at your role. The **permissions policy** allows `s3:GetObject`, `s3:PutObject` and `s3:DeleteObject` under your prefixes and nothing else.
3. You call `POST /v1/storage/{id}/verify`. The API assumes the role and, if the connection may write, under each prefix writes a small object (`.watermark-api-check-…`), reads it back and deletes it. The connection becomes `verified`.
4. Requests name the connection and a key: `{"storage": "stg_…", "key": "photos/IMG_4471.jpg"}`.

The API assumes your role for an hour at a time, and only for this connection's requests.

## Set it up with the AWS CLI

The role does not have to exist when you create the connection: choose its name first, since its ARN is `arn:aws:iam::<your account id>:role/<name>`. These steps need `curl`, `jq` and the AWS CLI signed in to the account that owns the bucket.

**1. Create the connection**, and keep the response:

```bash
curl https://api.watermark.ws/v1/storage \
  -H "Authorization: Bearer $WATERMARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "s3",
    "bucket": "acme-photos",
    "region": "us-east-1",
    "role_arn": "arn:aws:iam::123456789012:role/watermark-api",
    "prefixes": ["photos/", "renders/"]
  }' > storage.json
```

The response, shortened:

```json
{
  "id": "stg_7a",
  "object": "storage",
  "status": "unverified",
  "external_id": "wmx_Qk8pW3c9Lm2Zt7Hs1aB4nV6xR0yE5dFg",
  "trust_policy": {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"AWS": [
        "arn:aws:iam::841837852274:role/laravel-vapor-role",
        "arn:aws:iam::841837852274:role/watermark-fargate-task-role"
      ]},
      "Action": "sts:AssumeRole",
      "Condition": {"StringEquals": {"sts:ExternalId": "wmx_Qk8pW3c9Lm2Zt7Hs1aB4nV6xR0yE5dFg"}}
    }]
  },
  "permissions_policy": {
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": ["arn:aws:s3:::acme-photos/photos/*", "arn:aws:s3:::acme-photos/renders/*"]
    }]
  }
}
```

Use the policies exactly as returned rather than typing them out. The `Principal` lists the roles the API runs as: one for most work, one for long videos. A trust policy missing either one fails those requests.

**2. Create the role** with the trust policy, and give it the permissions policy:

```bash
jq .trust_policy storage.json > trust.json
jq .permissions_policy storage.json > permissions.json

aws iam create-role --role-name watermark-api --assume-role-policy-document file://trust.json
aws iam put-role-policy --role-name watermark-api --policy-name watermark-api-access --policy-document file://permissions.json
```

If the role already exists, replace its trust policy with `aws iam update-assume-role-policy --role-name watermark-api --policy-document file://trust.json`.

**3. Verify:**

```bash
curl -X POST "https://api.watermark.ws/v1/storage/$(jq -r .id storage.json)/verify" \
  -H "Authorization: Bearer $WATERMARK_API_KEY"
```

A `200` with `"status": "verified"` means requests can use it. A new role can take a few seconds to become usable everywhere in AWS: if verify answers `storage_role_refused` straight after `create-role`, wait ten seconds and verify again.

## Set it up in the AWS console

After step 1 above:

1. Open **IAM → Roles → Create role** and choose **Custom trust policy**. Paste `trust_policy` from the response and choose **Next**.
2. Add no permissions here; choose **Next**. Name the role as in your `role_arn` and choose **Create role**.
3. Open the role, choose **Add permissions → Create inline policy**, switch to **JSON** and paste `permissions_policy`. Name it and create it.
4. Verify, as in step 3 above.

## Set it up with Terraform

Create the connection first (step 1 above) and save the two policies as `trust.json` and `permissions.json` next to your configuration:

```hcl
resource "aws_iam_role" "watermark_api" {
  name               = "watermark-api"
  assume_role_policy = file("${path.module}/trust.json")
}

resource "aws_iam_role_policy" "watermark_api" {
  name   = "watermark-api-access"
  role   = aws_iam_role.watermark_api.id
  policy = file("${path.module}/permissions.json")
}
```

Then verify.

## Using the connection

As a source, or a watermark operation's `image`:

```json
{"source": {"storage": "stg_7a", "key": "photos/IMG_4471.jpg"}}
```

As an output's destination:

```json
{
  "source": {"storage": "stg_7a", "key": "photos/IMG_4471.jpg"},
  "operations": [{"type": "watermark", "image": {"storage": "stg_7a", "key": "photos/logo.png"}, "x": 40, "y": 40}],
  "outputs": [{"name": "web", "format": "webp", "destination": {"storage": "stg_7a", "key": "renders/IMG_4471.webp", "cache_control": "public, max-age=86400"}}]
}
```

- A key is a plain path under one of the connection's prefixes: no leading `/`, no `..`, no percent-encoding. Anything else gets `403 key_outside_prefixes`.
- Writing to a key that exists replaces the object.
- A source in your bucket counts against your account's largest source like any other.
- The result lists a file written to your bucket with its `destination` (`storage` and `key`) and the `etag` S3 returned, and no download link.

## Read-only and write-only connections

Send `"read": false` for a connection that only receives results, or `"write": false` for one the API only reads sources from. The permissions policy then allows only what the connection needs. A request that reads through a write-only connection gets `403 storage_read_not_allowed`, and one that writes through a read-only connection `403 storage_write_not_allowed`.

Verify checks a read-only connection by assuming its role, since it has nothing it may write. That proves the trust policy; whether the role may read your keys shows on the first request that reads.

## Buckets with extra rules

- **Encryption with your own KMS key (SSE-KMS).** Also give the role `kms:GenerateDataKey` (to write) and `kms:Decrypt` (to read) on that key, and make sure the key's policy lets the role use it. Buckets encrypted with S3-managed keys (SSE-S3, the default) or the AWS-managed `aws/s3` key need nothing more.
- **A bucket policy with a `Deny`**, such as one that only allows a VPC endpoint or certain IP addresses, also applies to the API and will block it.
- **A bucket in another AWS account than the role.** The bucket's policy must also allow the role the same actions.
- **The region must be the bucket's own.** A wrong region fails with "the bucket refused the request".

## Changing or removing a connection

`PATCH /v1/storage/{id}` changes a connection's `prefixes`, `read` or `write`. Send the whole new list of prefixes, not just the one you add. The API checks the new settings as verify does (assumes the role and, if the connection may write, writes, reads back and deletes a probe under each prefix) before it keeps them. A verified connection whose check fails is left exactly as it was, so requests already using it never break.

To add a folder, give the role access to it first, then send the change:

**1. Put the new permissions policy on the role.** It is the current one with the new prefix added; build it from the connection with `jq`:

```bash
curl https://api.watermark.ws/v1/storage/stg_7a \
  -H "Authorization: Bearer $WATERMARK_API_KEY" \
  | jq '.permissions_policy | .Statement[0].Resource += ["arn:aws:s3:::acme-photos/archive/*"]' > permissions.json

aws iam put-role-policy --role-name watermark-api --policy-name watermark-api-access --policy-document file://permissions.json
```

**2. Send the whole new list:**

```bash
curl -X PATCH https://api.watermark.ws/v1/storage/stg_7a \
  -H "Authorization: Bearer $WATERMARK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"prefixes": ["photos/", "renders/", "archive/"]}'
```

A `200` with `"status": "verified"` means requests can use the new folder now. The response's `permissions_policy` is the policy the role needs for the new settings; compare it with what the role has.

If you send the change before the role allows the new folder, the answer is `422 destination_rejected`, the connection keeps its old prefixes, and `error.permissions_policy` is the policy the new list needs: put that on the role and send the same change again.

```json
{
  "error": {
    "type": "invalid_request",
    "code": "destination_rejected",
    "message": "The connection was not changed: the new settings failed their check. Could not write archive/.watermark-api-check-k3v9q2m8x1pz to acme-photos: the role was denied.",
    "permissions_policy": {
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
        "Resource": ["arn:aws:s3:::acme-photos/photos/*", "arn:aws:s3:::acme-photos/renders/*", "arn:aws:s3:::acme-photos/archive/*"]
      }]
    }
  }
}
```

To remove a folder, or to stop the connection reading or writing, send the change first, then replace the role's policy with the `permissions_policy` in the response.

A connection that is not verified yet (`unverified` or `failed`) takes the new settings even when their check fails, and becomes `failed` with the reason in `last_error`, as a failed verify leaves it; fix the role and verify.

The bucket, region and role cannot be changed (`422 validation_failed`). For those, create a new connection and delete the old one. The new connection has a new `external_id`; while both are in use, the role's trust policy can allow both: `"sts:ExternalId": ["wmx_old…", "wmx_new…"]`.

`DELETE /v1/storage/{id}` stops the API using the connection at once. Nothing in your bucket is touched. To cut the API off on your side, remove the role's permissions policy, which takes effect at once. Removing only the trust policy stops new sessions, but one already started can last up to an hour.

## When something fails

Verify (and a change, see above) answers `422` with one of the codes below, sets the connection's `status` to `failed` and keeps the reason in `last_error`. Fix the cause and verify again. A request that hits the same problem ends `failed` with the code in `error.code`.

| Code | What it means | What to check |
| --- | --- | --- |
| `storage_role_refused` | The API could not assume your role. | The trust policy is the one this connection returned (its `external_id` changes with every new connection); `role_arn` is spelled right; the role was not created seconds ago. |
| `destination_rejected` | "Could not write …: the role was denied." | The permissions policy covers the prefix; the KMS key allows the role; no bucket policy denies it. |
| `source_unreachable` | "Could not read …: the role was denied", or "there is no such object". | The key exists and is under a prefix the connection may read. Without `s3:ListBucket` on the bucket, S3 answers a missing key as denied rather than missing. |
| `source_unreachable` | "… the bucket refused the request." | The connection's `region` is the bucket's region. |
| `storage_not_found` (`404`) | No verified connection with that id on this account. A request naming a connection that is not `verified` gets this too. | The id, that your key belongs to the same account, and the connection's `status`. |
| `storage_limit_reached` (`409`) | The account already has 10 connections. | Delete one you no longer use. |