Bring Your Own Storage (BYOS)

Start here: README · Related: Authentication · Conventions & errors · Text translation · Documents

Bring Your Own Storage routes a tenant's translation data to an Azure Data Lake Storage Gen2 account that the customer owns, instead of the platform store. This page documents the four endpoints that manage a tenant's storage binding — the record that names the account, the credential mode, and the verification state — the read-only usage endpoint that reports a tenant's position against the platform storage cap, plus what data lands in customer storage and what stays on the platform. You need this page if you are onboarding a tenant to their own storage account, automating the bind/verify cycle, or handling the fail-closed errors that appear on translation endpoints when a binding is not verified.

Base URL: https://trueidiom.com

Requests take an optional api-version=2026-09-01 query parameter; omitting it serves the oldest supported version. See Versioning.

Availability

All four binding endpoints require the server flag BYOS_ENABLED=true. When the flag is off, the storage binding store is never constructed and each of them returns:

HTTP/1.1 409 Conflict
Content-Type: application/json

{"detail": "Bring Your Own Storage is not enabled on this service"}

TRANSLATION_ENABLED does not gate these endpoints — binding management stays available while translation is disabled.

GET /api/storage/usage is deliberately not behind the flag. The platform storage cap it reports matters most where BYOS is switched off entirely, because there every tenant is on platform storage; the endpoint answers on any deployment.

sas and connection_string bindings rely on the operator having configured secret storage (KEY_VAULT_URL, or BYOS_SECRET_KEY for the local encrypted store) — but that is a startup condition, not a request-time one. A deployment that turns on BYOS_ENABLED with neither configured fails to boot. So any service that answers these endpoints at all has secret storage behind it, and a PUT carrying a secret never fails for want of it.

Authentication

The generated document declares no securitySchemes and no global security requirement, so the interactive /docs page shows every endpoint as unauthenticated. They are not. Every endpoint on this page resolves the caller's tenant through the same identity path used by /api/translations, and the resolved tenant id is the only thing that selects which binding is read or written — no request field can name a tenant, a binding, or a storage target belonging to someone else.

Identity resolution depends on the deployment's state:

X-Tenant-ID is optional and only narrows API-key authentication to a specific tenant; a mismatch between the key and the named tenant fails authentication.

The curl examples below use $TRUEIDIOM_API_KEY — the same tenant API key the translation and document endpoints take, returned once as tenant_api_key by POST /api/auth/signup/email. See Authentication.

Failure responses:

Any principal that can authenticate for a tenant may manage that tenant's binding; there is no additional role requirement beyond tenant identity. The resolved tenant id is the only selector for which binding is read or written, so these endpoints cannot reach another tenant's binding regardless of what the request body contains.

Every response carries an X-Request-ID header. Send your own X-Request-ID on the request to have it echoed back and bound to the server-side logs for that call.

What a binding is

One binding per tenant, held in the platform control plane (never in customer storage). It has two axes:

• provider — platform means the tenant uses the platform store (the default when no binding exists). adls means the tenant has nominated its own Azure account.
• status — unverified, verified, or error. Data routes to the customer account only when provider="adls" and status="verified".

Routing fails closed. With provider="adls" and any status other than verified, operations that would persist data do not silently fall back to the platform store — they fail. What happens to the job next depends on which pipeline it is: a text translation job is parked as persistence_failed and replayed once the binding verifies, while a document job is marked failed and its scratch purged, so it must be resubmitted after the binding is fixed (see Effects on other endpoints).

Any successful PUT resets status to unverified, clears last_error, and clears verified_at, even when the submitted values are identical to what was stored. Re-run verify after every edit.

Supported backend

Azure Data Lake Storage Gen2 only (provider="adls"). account_url must be the dfs endpoint — https://<account>.dfs.core.windows.net. The account needs a StorageV2 account with hierarchical namespace enabled, and filesystem is the container name. No other cloud or storage backend is accepted by the schema.

Data planes

Within the container, three logical prefixes are used. Defaults come from the server; prefixes overrides them per plane.

Approval of a document job does not write a record to the golden plane in your account. It writes a counts-only audit record — document id, tenant, reviewer, language pair, pair and corrected counts, timestamp, no text — into the platform's own golden store, filed as <document_id>.docapproval-<n>.json alongside the <job_id>.json blobs there. The document's own files land on the documents plane as usual.

Document processing writes to a local scratch directory because the extractor and reconstructor operate on real files; for each finalized document file that is uploaded to the customer account, the scratch file is removed on the same code path, success or failure. Downloads fetch from the customer account once scratch is gone. The exception is the pipeline telemetry log: it is pipeline internals end to end, it is never uploaded to a customer account, and it is retained platform-side only.

What lands in the customer account is the customer's content: the layout JSONs and the quality report are written there with the document text, geometry and review signals (scores, needs_review, terminology compliance), while pipeline execution internals — engine and model identifiers, per-segment step traces, token counts — stay in the platform control plane. The golden plane follows the same rule: the records written to your account are redacted copies, with model names, token usage, reasoning and per-step agent metadata stripped and the source and target text, review fields, scores and decisions kept — the platform's full-fidelity record stays in the control plane.

Stays in the platform control plane regardless of the binding: tenant accounts, users, billing and usage records, job status and job metadata (including checksums and review history), the binding record itself, and any stored credential ciphertext.

BYOS covers data at rest. The translation work itself still runs on platform compute.

Credential model

auth_mode selects how the platform authenticates to the customer account.

Secret handling

secret is write-only. It is accepted on PUT, handed to the secret resolver (Azure Key Vault when KEY_VAULT_URL is set, otherwise a Fernet-encrypted local store), and only the opaque reference is kept on the binding record.

No endpoint on this page returns secret or the stored reference. Reads expose the boolean has_secret and nothing else about the credential. This is verified behavior, not merely intent: GET after storing a SAS token returns has_secret: true with no secret material anywhere in the response body.

Secret lifecycle rules applied by PUT:

• Submitting secret with auth_mode=sas or connection_string replaces the stored secret for that tenant.
• Omitting secret reuses the stored secret only when an existing binding has the same auth_mode and already holds one. Otherwise the request is rejected with 422.
• Switching to auth_mode=managed_identity, switching to provider=platform, or calling DELETE deletes the stored secret.

The binding object

Returned by GET, PUT, and POST .../verify. The OpenAPI document types these responses as a free-form object; the concrete fields below come from the router's serializer.

GET /api/storage/binding

Read the calling tenant's binding and its verification status.

Auth: tenant identity (see Authentication).

No path, query, or body parameters.

curl -sS "https://trueidiom.com/api/storage/binding?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY"

Call this first when setting up keyless access: principal_client_id is the id you pass to az role assignment create --assignee, so a GET before any PUT tells you which principal to authorize.

Tenant with a verified ADLS binding:

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "adls",
  "account_url": "https://acmelocalization.dfs.core.windows.net",
  "filesystem": "translations",
  "prefixes": {
    "golden": "golden",
    "documents": "documents",
    "terminology": "terminology"
  },
  "auth_mode": "managed_identity",
  "status": "verified",
  "last_error": null,
  "verified_at": "2026-07-24T09:14:52.118374+00:00",
  "has_secret": false,
  "principal_client_id": "6b1f0c74-2d9e-4a83-9f5b-1c07ae43d2f8"
}

When no binding has ever been stored, the response is a synthesized platform binding (nothing is persisted by this call):

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "platform",
  "account_url": "",
  "filesystem": "",
  "prefixes": {
    "golden": "golden",
    "documents": "documents",
    "terminology": "terminology"
  },
  "auth_mode": "managed_identity",
  "status": "unverified",
  "last_error": null,
  "verified_at": null,
  "has_secret": false,
  "principal_client_id": "6b1f0c74-2d9e-4a83-9f5b-1c07ae43d2f8"
}

Note that status reads unverified on a platform binding. Treat provider as the switch; status is meaningful only when provider="adls".

Errors

PUT /api/storage/binding

Create or replace the binding. This is a full replace, not a patch: omitted fields take their schema defaults, so omitting filesystem blanks it and omitting prefixes resets all three prefixes to the defaults. Send the complete desired state on every call.

Auth: tenant identity (see Authentication).

Request body (application/json, required) — schema StorageBindingUpsert:

Ignored when provider="platform": account_url, filesystem, prefixes, auth_mode, and secret. The stored binding becomes a bare platform binding.

Keyless bind

curl -sS -X PUT "https://trueidiom.com/api/storage/binding?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "provider": "adls",
        "account_url": "https://acmelocalization.dfs.core.windows.net",
        "filesystem": "translations",
        "auth_mode": "managed_identity"
      }'

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "adls",
  "account_url": "https://acmelocalization.dfs.core.windows.net",
  "filesystem": "translations",
  "prefixes": {
    "golden": "golden",
    "documents": "documents",
    "terminology": "terminology"
  },
  "auth_mode": "managed_identity",
  "status": "unverified",
  "last_error": null,
  "verified_at": null,
  "has_secret": false
}

SAS bind with custom prefixes

curl -sS -X PUT "https://trueidiom.com/api/storage/binding?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "provider": "adls",
        "account_url": "https://acmelocalization.dfs.core.windows.net",
        "filesystem": "translations",
        "prefixes": {
          "golden": "approved",
          "documents": "docs",
          "terminology": "glossaries"
        },
        "auth_mode": "sas",
        "secret": "sv=2025-01-05&ss=b&srt=sco&sp=rwdlac&se=2026-12-31T00:00:00Z&sig=REPLACE_WITH_YOUR_SAS_SIGNATURE"
      }'

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "adls",
  "account_url": "https://acmelocalization.dfs.core.windows.net",
  "filesystem": "translations",
  "prefixes": {
    "golden": "approved",
    "documents": "docs",
    "terminology": "glossaries"
  },
  "auth_mode": "sas",
  "status": "unverified",
  "last_error": null,
  "verified_at": null,
  "has_secret": true
}

Opt back out to platform storage

curl -sS -X PUT "https://trueidiom.com/api/storage/binding?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"provider": "platform"}'

This path provisions the tenant's managed platform container before anything is mutated, so it can fail with 502 when provisioning does not succeed. That failure is clean — the existing binding and its stored secret are untouched, and the tenant stays on whatever they had. DELETE /api/storage/binding is the always-available exit: it never blocks on provisioning, so use it when a PUT back to platform keeps returning 502.

Errors

The two 422 shapes differ: business-rule rejections put a string in detail, schema failures put an array there. Parse defensively.

POST /api/storage/binding/verify

Run a live health probe against the bound account and record the outcome on the binding. A binding is trustworthy only after this succeeds — the resolver refuses to route data to an adls binding in any other status.

Auth: tenant identity (see Authentication).

No path, query, or body parameters. Send no body.

What it checks. Using the binding's credential (DefaultAzureCredential for managed_identity, the decrypted stored secret for sas/connection_string), it performs a real round-trip against the customer account: upload a small probe object at <prefixes.golden>/.byos-probe, read it back, then delete it. Any failure in that sequence — wrong account_url, missing container, missing role assignment, expired SAS, unresolvable secret — is captured.

The probe writes, reads, and deletes under the golden prefix. Grant the platform identity access to every prefix you configure — documents as well as golden — so that document artifacts land as reliably as approved records do.

On success, the binding's store cache is invalidated so the new destination takes effect immediately, and any of this tenant's text translation jobs parked in persistence_failed are replayed to the now-reachable account. Document jobs are not replayed — a document job that failed against a broken binding has to be resubmitted (see Document jobs: failed, not parked).

curl -sS -X POST "https://trueidiom.com/api/storage/binding/verify?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY"

Success (200) — the binding object plus a replay summary:

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "adls",
  "account_url": "https://acmelocalization.dfs.core.windows.net",
  "filesystem": "translations",
  "prefixes": {
    "golden": "golden",
    "documents": "documents",
    "terminology": "terminology"
  },
  "auth_mode": "managed_identity",
  "status": "verified",
  "last_error": null,
  "verified_at": "2026-07-24T09:14:52.118374+00:00",
  "has_secret": false,
  "replay": {
    "tenant_id": "8f21c9d0b47a3e15",
    "replayed": ["job_2c41f0a9", "job_9be7d310"],
    "failed": []
  }
}

replay fields:

Probe failure (200, not an error status) — the request succeeds, the binding is recorded as broken:

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "adls",
  "account_url": "https://acmelocalization.dfs.core.windows.net",
  "filesystem": "translations",
  "prefixes": {
    "golden": "golden",
    "documents": "documents",
    "terminology": "terminology"
  },
  "auth_mode": "managed_identity",
  "status": "error",
  "last_error": "403 AuthorizationPermissionMismatch: This request is not authorized to perform this operation using this permission.",
  "verified_at": null,
  "has_secret": false
}

A failed probe returns HTTP 200. Branch on status, not on the HTTP status code: treat status == "verified" as success and status == "error" as failure, reading last_error for the reason. The replay key is absent on failure.

Role assignments can take several minutes to propagate in Azure; a 403-flavored last_error immediately after granting Storage Blob Data Contributor is often resolved by retrying verify a few minutes later.

Errors

DELETE /api/storage/binding

Remove the binding entirely. Any stored credential secret is deleted, the store cache is invalidated, and the tenant reverts to platform storage for subsequent work.

Auth: tenant identity (see Authentication).

No path, query, or body parameters. Deleting when no binding exists is not an error.

This is the always-available exit. Unlike PUT with {"provider": "platform"}, it is never blocked by platform-container provisioning — a tenant can always walk away from their own account, and the platform destination is provisioned lazily on their next job.

Data already written to the customer account is left untouched — nothing is migrated, copied back, or deleted from the customer's storage.

curl -sS -X DELETE "https://trueidiom.com/api/storage/binding?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY"

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "platform",
  "status": "deleted"
}

This response is not the binding object — it carries only these three fields, and status: "deleted" is a delete-acknowledgement value that never appears on a stored binding. Call GET afterwards if you need the full post-delete state.

Errors

GET /api/storage/usage

Read the calling tenant's position against the platform storage cap: the document artifact bytes resting on the platform plane, the cap that governs them, and whether that cap is actually being enforced.

Auth: tenant identity (see Authentication).

No path, query, or body parameters. This endpoint is not gated by BYOS_ENABLED — see Availability.

curl -sS "https://trueidiom.com/api/storage/usage?api-version=2026-09-01" \
  -H "X-API-Key: $TRUEIDIOM_API_KEY"

{
  "tenant_id": "8f21c9d0b47a3e15",
  "provider": "platform",
  "used_bytes": 41929216,
  "cap_bytes": 5368709120,
  "remaining_bytes": 5326779904,
  "enforced": true
}

The 507 admission response

When the cap is enforced, it is admission control on document submission: POST /api/documents refuses the upload before the body is parsed, against the declared Content-Length.

HTTP/1.1 507 Insufficient Storage

{"detail": "platform storage cap reached (5368709120 of 5368709120 bytes used) — delete document jobs you no longer need (DELETE /api/documents/{document_id}), or bring your own storage (PUT /api/storage/binding) to keep documents in your own account"}

Three things narrow when this can fire:

• It is off by default — PLATFORM_STORAGE_CAP_BYTES defaults to 0, and a cap of 0 is uncapped.
• It is skipped entirely for a tenant with a non-platform binding. Those artifact bytes rest in the customer's own account, so the platform cap has nothing to guard — this is the one place where BYOS changes an admission decision rather than a destination.
• It gates the next submission, not the current one. A job already in flight can finish somewhat past the cap, because pipeline outputs are written after admission.

Text translation is not gated by this cap; only document artifact bytes count toward it.

Errors

Effects on other endpoints

Once a tenant has provider="adls", the binding's status changes how persistence-bearing endpoints behave.

The two pipelines recover differently, and the difference matters operationally: text jobs park and replay, document jobs fail and must be resubmitted.

Text translation: parked and replayed

Binding present but not verified. Persistence fails closed rather than writing to the platform store. Both the autonomous publish path (POST /api/translations when the quality gate passes) and the review-approval path (POST /api/translations/{job_id}/review with action: "approve") return:

HTTP/1.1 409 Conflict

{
  "detail": "workspace storage (BYOS) binding is not verified — verify it in Storage settings and the parked job will publish automatically, or switch the workspace back to platform storage",
  "error_code": "storage_binding_not_verified",
  "request_id": null
}

Note the literal detail says workspace — the web app's word for a tenant. Match on error_code, not on this text.

The job is not lost: it is stored with status persistence_failed and is republished automatically by the next successful POST /api/storage/binding/verify.

Verified binding, write fails at persist time. The response is:

HTTP/1.1 502 Bad Gateway

{
  "detail": "golden persistence failed: <reason> — the job is parked as persistence_failed and can be retried",
  "error_code": "persistence_failed",
  "request_id": "b3d90f5c-7a41-4e6b-9c02-5f18d47ae3c1"
}

Document jobs: failed, not parked

There is no parking or replay on the document path. When a document job cannot upload its artifacts to the bound account — unverified binding, revoked access, unreachable storage — the job is marked failed with the reason on its failure detail, and its local scratch directory is purged so no tenant bytes linger on the platform volume. Nothing is written to the platform store as a fallback, and nothing is held for later.

That job is not recoverable by verifying the binding: a successful POST /api/storage/binding/verify replays parked text jobs only. Fix the binding, verify it, then resubmit the document.

Correlating these errors

Both bodies include error_code; branch on it rather than on the detail text. The full code vocabulary is in Conventions & errors.

The body's request_id echoes the X-Request-ID you sent, and is null when you sent none — as in the 409 above. The X-Request-ID response header is a separate thing and always carries an id, server-generated when your request did not supply one. So send your own X-Request-ID when you want a correlator inside the error body; otherwise read it off the response header.

Onboarding sequence

1. Operator enables BYOS_ENABLED=true. Secret storage (KEY_VAULT_URL or BYOS_SECRET_KEY) is required alongside it — the service will not start without one, even on a keyless-only deployment. Add the platform Azure identity credentials if any tenant will use managed_identity.
2. Customer creates a StorageV2 account with hierarchical namespace enabled and a container.
3. For managed_identity: customer grants the platform identity Storage Blob Data Contributor on that container (or account). For sas/connection_string: customer mints the credential.
4. PUT /api/storage/binding with provider: "adls", the dfs account_url, the container as filesystem, and the chosen auth_mode (plus secret where required).
5. POST /api/storage/binding/verify and confirm the response has status: "verified".
6. Translate and approve as usual. Re-run step 5 after any subsequent PUT, and after rotating a SAS token (PUT the new secret, then verify).