77 lines
5.6 KiB
Markdown
77 lines
5.6 KiB
Markdown
# ARR private object exchange
|
|
|
|
`arr_storage` is the ARR-owned binary exchange boundary for the Opera XML flow. It is infrastructure only: it does not parse XML, invoke an Agent, generate reports, or write business facts.
|
|
|
|
## Frozen object contract
|
|
|
|
Every object is private and immutable. The only accepted key shape is:
|
|
|
|
```text
|
|
{prefix}/jobs/{opaque_job_id}/attempts/{0001..9999}/{staged|committed}/{role}/{canonical_filename}
|
|
```
|
|
|
|
The default prefix is `arr`. Object-key filenames are controlled (`source.xml`, `daily-report.xlsx`, `result.json`, `structured-result.json`, `exception-report.xlsx`), so an uploaded user filename cannot leak guest or booking data through an OSS key. The delivery filename remains separate and is validated before it can enter `ArtifactRef`.
|
|
|
|
Objects carry Schema 1.0 metadata for job, attempt, role, state, SHA-256, exact byte size, and MIME type. ETag is stored only as provider evidence; it is never treated as a content hash. A cloud adapter should preserve a provider version ID when bucket versioning exposes one.
|
|
|
|
Upload is two-step:
|
|
|
|
1. copy the input into a private snapshot while computing SHA-256 and enforcing the role limit;
|
|
2. create `staged` with an if-absent condition, server-side copy it to a fresh `committed` key, download/hash-check the committed object, then delete only the exact staged object.
|
|
|
|
No code path overwrites a committed key. An idempotent retry succeeds only when the existing object has the same metadata and bytes. Only committed keys implement `ArtifactStore.materialize()` and may enter a `DeliveryEnvelope`.
|
|
|
|
## Cloud adapter boundary
|
|
|
|
`CloudObjectBackend` accepts a narrow `CloudClientPort`. `AliyunOssV2Client` implements it with the Aliyun OSS Python SDK V2: conditional upload, head, streamed read, conditional server-side copy, exact-key delete, normalized custom metadata/content type and safe provider error classes. Startup queries bucket info and fails closed unless the region matches, the bucket does not allow anonymous writes, server-side encryption is configured and versioning is off. A `public-read` bucket is supported by explicit deployment decision; every ARR put/copy sets object ACL `private`, which prevents new processing outputs from inheriting public read access.
|
|
|
|
Install `requirements-oss.txt`, then inject `ARR_OSS_REGION`, `ARR_OSS_BUCKET` and optional HTTPS `ARR_OSS_ENDPOINT`. Credentials use the SDK environment credential provider (RAM/STS variables); no access key is accepted from an Agent message or source file. The adapter can be present while live readiness remains false when deployment values are absent.
|
|
|
|
The SDK transport uses a dedicated HTTP session with ambient desktop/system proxies disabled, so OSS credentials are not silently routed through an unrelated proxy. A deployment that requires an outbound proxy must add and review an explicit transport configuration rather than relying on inherited OS settings.
|
|
|
|
Provider requirements:
|
|
|
|
- bucket ACL is `private` or the explicitly approved `public-read` mode, TLS is required, and server-side encryption is enabled; `public-read-write` is always rejected;
|
|
- ARR and the runtime fetch provider use separate least-privilege identities;
|
|
- ARR can put/head/get/copy exact `arr/jobs/` keys; the runtime can only read the resolved committed source object;
|
|
- credentials come from platform secret/instance-role facilities, never prompts, object metadata, URLs, source files, `.env.example`, or logs;
|
|
- bucket versioning status must be `Off`; the adapter fails closed for `Enabled` or `Suspended`, because the required forbid-overwrite condition is not honored in those modes;
|
|
- conditional create must be real provider-side `If-None-Match`/forbid-overwrite behavior, not a head-then-overwrite sequence.
|
|
|
|
## Agent source fetch
|
|
|
|
ARR uploads the XML itself and records the committed object before Agent dispatch. The program message contains one attachment-shaped descriptor with bucket, endpoint, exact object key, hash and byte size. Those values are routing and integrity metadata, not download credentials. The installed `fetch_oss_file` provider owns its OSS credential and must be restricted to read-only access to the ARR source prefix.
|
|
|
|
The Agent must not use an OSS SDK, URL or user-supplied key, and it must hash/size-check the materialized file before invoking the Skill.
|
|
|
|
## Optional short-lived fetch authorization
|
|
|
|
`InMemoryReadGrantBroker` and `PostgresReadGrantBroker` remain available for a future runtime that supports opaque, job-bound, single-read grants. They are not required by the currently installed `fetch_oss_file` contract, which consumes the attachment-shaped OSS descriptor directly.
|
|
|
|
## Lifecycle
|
|
|
|
- incomplete `staged` objects: provider lifecycle deletion after 24 hours;
|
|
- committed source XML and Agent outputs: no automatic deletion within their business month; retain according to the approved data-retention policy after that month;
|
|
- generated monthly/channel downloads: governed by their report archive policy, not by the Agent exchange prefix;
|
|
- short-lived grants: at most 300 seconds and one materialization;
|
|
- logs/outbox: opaque IDs, state, SHA-256, byte size and safe error code only; never signed URLs, raw filenames, XML bytes, guest fields, or credentials.
|
|
|
|
Local wiring:
|
|
|
|
```python
|
|
from pathlib import Path
|
|
|
|
from arr_storage import FilesystemObjectBackend, ManagedObjectStore
|
|
|
|
backend = FilesystemObjectBackend(Path("/private/arr-object-store"))
|
|
store = ManagedObjectStore(backend)
|
|
committed = store.upload_committed(
|
|
job_id="job-001",
|
|
attempt_no=1,
|
|
role="source_xml",
|
|
source=Path("/private/upload/input.xml"),
|
|
original_filename="source.xml",
|
|
)
|
|
source_ref = committed.to_artifact_ref()
|
|
```
|