feat: sync latest ARR implementation
This commit is contained in:
Wyndham ARR
@@ -1,55 +1,13 @@
|
||||
# ARR remote processing orchestration
|
||||
# ARR2 deterministic processing
|
||||
|
||||
`arr_processing` is the ARR-owned orchestration boundary between a registered XML source and the existing validated ingestion pipeline. The runner never sees an OSS credential, local Agent path, XML bytes, guest data or database credential. The input-message builder resolves one ARR-owned committed object and emits only the non-secret OSS routing that the existing `fetch_oss_file` provider needs.
|
||||
The active ARR2 path uses two modules:
|
||||
|
||||
## Request
|
||||
- `policy.py` loads the frozen processor version and rule-set SHA-256 from `arr-opera-daily-ingest`.
|
||||
- `local.py` runs `process_daily.py` in an isolated directory with a bounded timeout, requires process exit status to
|
||||
agree with both JSON artifacts, and permits only basename-confined declared XLSX outputs.
|
||||
|
||||
`ProcessingRequest` serializes exactly six values as canonical JSON:
|
||||
The caller stores the success/failure artifacts and passes a canonical `DeliveryEnvelope` to `arr_ingestion`. Processor
|
||||
stdout/stderr, temporary paths and source bytes never enter the public response or database.
|
||||
|
||||
- contract `arr-opera-daily-request-1`;
|
||||
- opaque `job_id` and `source_file_id`;
|
||||
- attempt number;
|
||||
- approved processor version and rule-set SHA-256.
|
||||
|
||||
The database request remains the six-field opaque identity. Immediately before dispatch, `OssProcessingMessageBuilder` resolves the registered immutable source and builds `arr-opera-daily-program-input-2`: one `oss_attachments` item containing the canonical filename, bucket, endpoint host, exact object key, SHA-256 and byte size, plus one short-lived attempt-bound `arr_submit_processing_result` grant. The grant is excluded from `ProcessingRequest.to_dict()`, its canonical hash and repr; it exists only in the remote program message. No AccessKey, XML bytes, URL, local path, database credential or MCP transport bearer enters that message. `OpenAgentProcessingTransport` sends the canonical JSON through the text Session/Run API with a stable dispatch idempotency key.
|
||||
|
||||
## Runner semantics
|
||||
|
||||
`ProcessingRunner` reserves one request identity, correlates one remote run, and supports start, bounded polling, cancellation and separately delivered callbacks. Retryable 408/429/5xx/network failures reuse the same idempotency key. A 409 is accepted only if the shared ledger already has the exact correlated run; an uncorrelated active run remains a safe retryable error.
|
||||
|
||||
The in-memory ledger is for tests/local slices. `PostgresProcessingState` is the production-facing implementation against migration 008: reservation identity, `remote_run_id`, status and callback replay are read from the shared ARR processing tables rather than process memory. A submission that is rejected, exhausts its transport retries, or returns an invalid start response is recorded as a terminal failed attempt/run; an identical replay is not silently submitted again.
|
||||
|
||||
Remote `success` is not an ingestion decision. Empty `final_content`, unsigned JSON, free-form text, the old internal finalizer object with `local_path`, a mismatched run, or a terminal remote failure can never construct a `DeliveryEnvelope`.
|
||||
|
||||
## Authenticated result
|
||||
|
||||
The trusted runtime/callback adapter—not the Agent—must register each generated file and produce the path-free result defined by `prompts/arr_opera_daily_processing_result.schema.json`:
|
||||
|
||||
- contract `arr-opera-daily-result-1`;
|
||||
- delivery/job/attempt/run correlation;
|
||||
- processor, rule-set and result Schema identities;
|
||||
- business success/failure shape;
|
||||
- opaque handles plus filename, SHA-256, exact byte size and MIME for `daily_report`, `result_json`, `structured_result_json` and `exception_report`.
|
||||
|
||||
`SignedResultCodec` authenticates this payload with HMAC-SHA256, key ID, UTC issue time and nonce. Keys must be at least 32 bytes, injected from platform secret management, rotated by key ID, and unavailable to the Agent. The default acceptance window is ten minutes. Callback replay is idempotent only for identical signed bytes; the same delivery ID with different signed bytes is a conflict.
|
||||
|
||||
## Output registration
|
||||
|
||||
`ProcessingOutputRegistrar` reads every opaque output handle through a controlled `RemoteFilePort`. The deployed implementation, `PrefixRemoteFilePort`, resolves a handle only below `ARR_AGENT_OUTPUT_PREFIX` in the configured private OSS bucket. For every handle it:
|
||||
|
||||
1. materializes into a fresh private directory with the role size cap;
|
||||
2. rejects missing, symlinked, non-regular, oversized or hash-mismatched bytes;
|
||||
3. uploads through the staged/committed `ManagedObjectStore` path;
|
||||
4. converts only committed identities to the strict ARR `DeliveryEnvelope 1.0`.
|
||||
|
||||
ARR then runs the existing `DeliveryValidator` and PostgreSQL transaction. The signed remote result never bypasses independent replay and never writes Finance facts by itself.
|
||||
|
||||
## Runtime writeback adapter
|
||||
|
||||
`python -m arr_processing.runtime_writeback` is the trusted post-Skill adapter. It accepts only an approved output root and the frozen `FROZEN_AGENT_RESULT`, rejects symlinks/path escape and filename mismatches, rehashes all files, publishes them under deterministic opaque handles, creates a stable delivery ID, signs the canonical result with the independent callback HMAC key, and posts the exact same signed bytes on retryable failures.
|
||||
|
||||
The runtime adapter receives OSS and callback credentials from deployment secret injection. They are not placed in the Agent prompt or frozen result. Its contract and required variables are documented in [`AGENT_WRITEBACK_CONTRACT.md`](../AGENT_WRITEBACK_CONTRACT.md).
|
||||
|
||||
## Current production boundary
|
||||
|
||||
The text Open API submit/get/cancel contract, direct OSS attachment-shaped input, PostgreSQL callback state, OSS V2 output exchange, runtime-side signer/callback and ARR-side independent validation/commit path are implemented and tested. Deployment still needs an API-enabled published Agent Profile, the actual OSS runtime permissions, an independent HMAC key, a public callback URL and invocation of the writeback adapter after the Skill finishes. Live end-to-end connectivity must not be claimed until both readiness flags are true and one non-sensitive vertical-slice job has committed.
|
||||
Older remote-run, signature and callback modules remain as ARR1 compatibility/audit code. The ARR2 Web entrypoint does
|
||||
not import them, root requirements do not install their HTTP client, and Compose exposes no corresponding service.
|
||||
|
||||
Reference in New Issue
Block a user