feat: sync latest ARR implementation

.project-docs/20-architecture/data-flow.md

@@ -1,24 +1,61 @@
 # Data Flow
 
-## Primary Flows
+## Web Access Boundary
 
-| Flow | Source | Destination | Notes |
+1. An anonymous browser requesting `/` or `/h5` is redirected to `/login` with an allowlisted return target; anonymous API/download requests receive JSON `401 AUTH_REQUIRED`.
+2. `POST /api/login` accepts strict same-origin JSON, verifies runtime-owned credentials under a bounded per-client attempt ledger and issues a random server-side session plus CSRF token in an `HttpOnly`, `SameSite=Strict` cookie.
+3. Every portal page, detailed health response, read API, upload/mutation, task trace and artifact download validates the authenticated session. Mutations additionally validate the session CSRF token.
+4. `POST /api/logout` validates CSRF, revokes the server-side session and expires the cookie. `/healthz` remains a no-detail anonymous readiness signal for container orchestration.
+
+## Primary XML Flow
+
+| Step | Source | Destination | Gate |
 |---|---|---|---|
-| XML ingestion | User upload | Agent processing | Upload is the user's only required action for this flow. |
-| Structured result commit | Agent output | PostgreSQL Finance facts | Independent validation/replay must succeed before commit. |
-| Automatic monthly trigger | Successful database commit | Monthly-report worker/program | Month and cutoff values are derived internally, not supplied by the user. |
-| Monthly workbook build | Committed Finance facts | XLSX artifact | `TOTAL PRICE` cells use `REAL PRICE × NIGHTS × NO_OF_ROOMS` formulas. |
+| 1 | Browser XML upload | Uploaded-basename task provenance + private canonical source object + queued DB job | Filename/content/size and immutable object identity |
+| 2 | Committed source object | Isolated processor input | Materialization rechecks stored bytes |
+| 3 | Fixed `process_daily.py` | Daily/result/structured or failure artifacts | Timeout, confined paths, exit/JSON agreement |
+| 4 | Local artifacts | Private committed OSS objects | Role, MIME, size and SHA-256 |
+| 5 | Canonical `DeliveryEnvelope` | `DeliveryValidator` | Strict Schema, reconciliation, source/artifact hashes and independent validation |
+| 6 | Verified delivery | PostgreSQL Finance facts | Serializable atomic commit/version activation |
+| 7 | Accepted/failed run | Task trace and outbox | Persisted state is authoritative |
+| 8 | `arr.daily_version_committed` | Dedicated monthly worker | Lease, `SKIP LOCKED`, bounded retry/dead-letter |
+| 9 | Retained Finance `ARRIVAL` facts | Monthly snapshot request | Affected month from event version; “更新至” is max included `ARRIVAL` |
+| 10 | Monthly snapshot | XLSX/result + `reporting` metadata | Formula reopen-validation, current-pin recheck, atomic activation |
+| 11 | Registered active report | Web list/download + outbox acknowledgement | Real report ID; visible monthly tab polls automatically; path/size/SHA-256 recheck before download |
 
 ## State Ownership
 
-- PostgreSQL owns accepted/current Finance facts and processing state.
-- Generated XLSX files are artifacts derived from a pinned database snapshot; they are not an alternative fact source.
+- PostgreSQL owns job/attempt/delivery state, Finance facts, current version and outbox.
+- PostgreSQL `reporting` owns monthly publication versions, daily-version lineage, channel manifest and artifact identities; it does not duplicate monthly business rows.
+- OSS owns immutable source/output bytes; ARR stores only controlled object identities in the database.
+- `processing_runs.uploaded_filename` owns the user-facing browser basename; the source artifact remains canonically
+  named `source.xml` for processing and validation.
+- Temporary processor/validator paths are deleted after each request.
+- Monthly XLSX/result bytes live in immutable local archive paths on the current shared output volume; generated report artifacts are derived outputs, not an alternative business fact source.
 
-## External Interfaces
+## Booking Source And Company-Report Flow
 
-- User-facing: XML upload, processing status, and controlled report download.
-- Internal: Agent structured-result submission and automatic report-task dispatch after commit.
+| Step | Source | Destination | Gate |
+|---|---|---|---|
+| 1 | Authenticated Booking XLSX upload | Bounded parser 2.0 | Requires Tour Code + exact `โรงแรม`, latest-row/cancellation semantics and room-item extraction/review |
+| 2 | Parsed raw items | Private artifact + extraction draft | Live migration 015 preserves source coordinates, confirmed/pending/deleted item state and extracted values |
+| 3 | Human review | Draft item updates | Confirm/edit or atomically soft-delete 1-50 unique selected items while canonical current Booking source remains unchanged |
+| 4 | Fully reviewed draft | Accepted canonical batch + `booking.current_source_batch` | Live 014/015 plus repository zero-pending/atomic activation; real PostgreSQL transaction-only vertical slice passes |
+| 5 | Current Finance facts | Company-report stay segments | Finance supplies company/channel/dates/pricing; normalized Group Code is the Booking lookup key; open draft blocks new job |
+| 6 | Current Booking room summary | `Booking Room` output | Sum by Group Code + room type; missing or unmatched Group Code leaves Booking Room blank |
+
+Migrations 014/015 are formally applied: batch 1 is the current source and the draft tables are empty. The source
+contains a PostgreSQL draft repository, backend routes, browser editor and GET/POST/PATCH/DELETE transport coverage.
+A real-PostgreSQL transaction-only probe created a draft, confirmed its pending item, activated immutable facts and then
+rolled everything back, preserving the current source and leaving no synthetic residue. First activation of an actual
+operator workbook remains a controlled business action.
+
+## Monthly Derivation Boundary
+
+The worker never infers a date from an XML filename or the current clock. It treats the event's `daily_version_id` only
+as a lookup key, derives scope from retained Finance `ARRIVAL`, and verifies that the generated report's maximum arrival
+equals its persisted `as_of_date`. Every data-row `TOTAL PRICE` cell is `=R[row]*C[row]*G[row]`.
 
 ## Last Updated
 
-2026-07-29
+2026-07-31

.project-docs/20-architecture/module-map.md

@@ -1,27 +1,37 @@
 # Module Map
 
-## Source Layout
-
-| Path | Responsibility | Owner Notes |
+| Path | Responsibility | Notes |
 |---|---|---|
-| `arr_web/` | Upload portal and business API | Manual monthly request code conflicts with ADR-001. |
-| `arr_mcp/` | Direct Agent-result submission and ingestion boundary | Post-commit trigger should originate at or after this boundary, not before it. |
-| `arr_ingestion/`, `arr_processing/` | Validate and persist structured processing results | Keep deterministic replay and transaction semantics. |
-| `monthly_reports/` | Database-backed monthly workbook generation | Needs formula-cell output and automatic orchestration integration. |
-| `database/` | Finance facts, views, guards and migrations | Database total remains useful for integrity checks. |
-| `Dockerfile`, `compose.yaml` | Reproducible single-server Web/MCP runtime | Dockerfile default Web CMD and Compose explicitly enable XML processing; direct source CLI defaults and runtime readiness remain fail-closed. |
-| `deploy/` | Caddy TLS/auth boundary, production env template and operator runbook | Only Caddy publishes host ports; never commit the real production env file. |
-| `tests/` | Regression and acceptance coverage | Add post-commit trigger and formula-cell assertions when implementation is authorized. |
+| `arr_web/auth.py` | Login credential verification and bounded attempt ledger | Environment-owned single-operator identity; dependency-free constant-time comparison; no secret logging |
+| `arr_web/programmatic.py` | Upload-to-terminal coordinator | Registers the validated browser basename separately from canonical `source.xml`; no remote/model transport |
+| `arr_web/booking_uploads.py`, `booking_ingestion/excel.py` | Authenticated Booking source upload and bounded raw XLSX parsing | Parser 2.0 reads Tour Code plus exact `โรงแรม`, latest-row cancellations and extracted room items with review state; coordinator/backend route contracts target drafts |
+| `booking_ingestion/excel_postgres.py` | Earlier direct Booking full-source import/activation path | Its one-item immediate-accept implementation does not represent parser 2.0 review and must not be treated as the completed review repository |
+| `booking_ingestion/excel_review.py`, `booking_ingestion/excel_review_postgres.py` | Editable extraction drafts and atomic reviewed-source activation | Single-operator repository uses advisory locking and a zero-pending gate; real PostgreSQL transaction/rollback acceptance passes; actor/reason/revision history is not implemented |
+| `arr_web/processing_runtime.py` | PostgreSQL + OSS + processor composition | Active production processing composition |
+| `arr_processing/local.py` | Bounded subprocess and output manifest extraction | Never exposes stdout/stderr |
+| `arr_processing/policy.py` | Frozen processor/rule identity | Neutral, reusable loader |
+| `arr_ingestion/validation.py` | Strict artifact/result validation | Runs `validate_daily.py` on success |
+| `arr_ingestion/postgres.py` | Atomic Finance commit and lifecycle state | Four retries only for transient SQLSTATEs |
+| `arr_storage/aliyun_oss_v2.py` | Encrypted/unversioned OSS adapter | Writes all objects private |
+| `arr_web/downloads.py` | OSS daily + controlled local report download routing | Rechecks metadata, size and SHA-256 |
+| `arr_web/job_trace.py` | Programmatic persisted-fact trace | No external trace store |
+| `arr_web/app.py`, `arr_web/repository.py`, `arr_web/company_jobs.py` | Authenticated portal routes and paged history reads | Default-deny login gate protects pages/APIs/downloads; daily/monthly counts and rows share a repeatable-read snapshot; company totals/slices share one lock |
+| `arr_web/server.py` | Standard-library HTTP transport | Dispatches GET/POST/PATCH/DELETE with one bounded body reader; real socket tests cover review update/delete and missing/oversized lengths |
+| `arr_web/static/login.html`, `login.css`, `login.js` | Responsive ARR login gateway | Labeled form, password visibility, generic inline failures, safe desktop/H5 return target and reduced-motion support |
+| `arr_web/static/app.js`, `arr_web/static/h5.js` | Authenticated client state, rendering and polling | Desktop includes 50-row Booking draft review/edit, the draft's validated uploaded filename below the review title, individual/all-visible selection, count-aware in-page delete confirmation and activation; company generation keeps the fixed five-company context beside the page title, a four-card upload/period setup row, compact upper-right month control, cumulative CO display labels on the three period cards, short centered period actions and an in-page generation confirmation dialog; session-expiry redirect and CSRF logout remain shared; monthly versions auto-refresh every four seconds |
+| `monthly_reports/worker.py` | Dedicated outbox consumer | Lease/reclaim, retry/dead-letter, success acknowledgement after activation |
+| `monthly_reports/repository.py` | Monthly snapshot and publication repository | Derives scope from `ARRIVAL`; persists metadata/lineage/artifact identities |
+| `monthly_reports/xlsx/build_workbook.mjs` | Monthly XLSX builder and reopen validator | Exact row-relative `TOTAL PRICE` formulas only in column S |
+| `monthly_reports/`, `company_reports/`, `channel_analytics/` | Downstream reports/BI | Consume accepted Finance facts |
+| `database/012_monthly_report_publication.sql` | Additive metadata-only publication schema | Applied after immutable 008–011 baseline |
+| `database/014_booking_current_source_batch.sql` | Booking full-workbook current-source pointer and view scoping | Formally applied on 2026-07-31; batch 1 remains selected |
+| `database/015_booking_excel_review_drafts.sql` | Item-level Booking extraction draft state | Formally applied and empty; basic latest-state review only, with no actor/reason/revision history and no DB-enforced zero-pending activation |
+| `compose.yaml`, `deploy/` | Web-login + Caddy-HTTPS template and worker deployment boundary | Requires Web credentials; only `/healthz` is anonymously probed; no MCP port/domain/service |
+| `tests/test_arr_programmatic.py` | Real success/failure vertical slices | Primary ARR2.0 acceptance proof |
 
-## Dependency Direction
-
-- Upload/Agent/ingestion writes validated facts; report modules depend on committed facts and must not call back into XML parsing.
-- Public traffic enters through Caddy; Web and MCP stay on the internal Compose network and use separate browser/MCP authentication boundaries.
-
-## Risky Or Sensitive Areas
-
-- Commit-to-trigger atomicity/idempotency, monthly snapshot selection, XLSX formula preservation, and PII-safe result metadata.
+Historical ARR1 compatibility modules and migrations remain for audit/tests but are not imported, installed or deployed
+by the active ARR2.0 entrypoint.
 
 ## Last Updated
 
-2026-07-29
+2026-07-31

.project-docs/20-architecture/system-overview.md

@@ -2,32 +2,52 @@
 
 ## Current Architecture
 
-The ARR system accepts an Opera XML upload, stores the private source artifact, invokes an Agent for deterministic processing, validates the returned structured result, and commits accepted Finance facts to PostgreSQL. Downstream analytics and report programs consume those committed facts. The accepted target flow automatically triggers monthly processing after the commit; the current manual monthly UI/API remains a known implementation gap.
+ARR2.0 accepts Opera XML plus a separate Booking Tour Code/`โรงแรม` workbook and owns both deterministic processing
+lifecycles. It writes immutable private
+objects, runs the frozen processor locally, independently validates the complete artifacts, and atomically commits
+accepted facts to PostgreSQL. Downstream analytics and report programs read committed facts.
+Successful daily commits also create a durable outbox event. A separate worker derives the affected month and watermark
+from committed `ARRIVAL` facts, publishes a validated workbook, and records metadata-only report state in PostgreSQL.
 
 ## Main Components
 
-| Component | Responsibility | Notes |
-|---|---|---|
-| `arr_web` | XML upload, job status, controlled downloads and business APIs | The existing manual monthly-generation request is not the accepted primary flow. |
-| SuperAgent + deterministic processing skill | Convert XML to the approved structured result | Must not decide user-facing report parameters. |
-| `arr_mcp` / ingestion service | Independently validate/replay and atomically commit accepted structured results | Automatic downstream trigger begins only after commit success. |
-| PostgreSQL Finance facts | Authoritative current and retained daily facts | Source of truth for downstream reports. |
-| `monthly_reports` | Generate monthly XLSX from committed database facts | Internal month/cutoff inputs are allowed; `TOTAL PRICE` must be an XLSX formula. |
-| `channel_analytics` / `company_reports` | Additional read/report consumers | Must not change the monthly-report trigger contract. |
-| Caddy + Compose deployment boundary | Terminate HTTPS, authenticate Web users, keep Web/MCP ports internal and start the explicit processing profile | MCP still performs its own bearer check and DNS-rebinding Host validation. |
+| Component | Responsibility |
+|---|---|
+| `arr_web.auth` / authenticated `SessionLedger` | Environment-owned operator credentials, bounded login attempts, random server-side sessions, CSRF and logout revocation |
+| `arr_web.programmatic` | Terminal upload orchestration and safe public error mapping |
+| `arr_processing.local` / `arr_processing.policy` | Isolated frozen processor execution and approved version/rule identity |
+| `arr_storage` | Immutable staged/committed OSS workflow; all object ACLs private |
+| `arr_ingestion.validation` | Strict delivery/result/structured validation plus independent daily validation |
+| `arr_ingestion.postgres` | Job lifecycle, artifact registration, atomic Finance version commit and outbox |
+| `booking_ingestion.excel` | Bounded Tour Code/`โรงแรม` extraction, latest-row/cancellation semantics and room-item normalization |
+| `booking_ingestion.excel_review_postgres` | Durable editable draft state and zero-pending atomic current-source activation |
+| `arr_web.booking_uploads` | Private Booking workbook upload, draft review orchestration and privacy-minimized source summaries |
+| `arr_web.job_trace` | Read-only programmatic trace from persisted run/attempt/delivery/version/outbox facts |
+| `monthly_reports.worker` | Dedicated leased outbox consumer with retry/dead-letter behavior |
+| `monthly_reports.repository` / `reporting` schema | ARRIVAL-derived snapshots, report versions, lineage, manifests and artifact identities |
+| `monthly_reports`, `channel_analytics`, `company_reports` | Deterministic downstream consumers of committed facts |
+| Web + Caddy Compose | Web-owned login/session boundary behind Caddy HTTPS; no browser-native Basic Auth and no MCP service |
 
 ## Important Boundaries
 
-- XML processing and Finance fact validation happen before the monthly-report trigger.
-- The user-facing upload workflow does not collect monthly report year, month, or cutoff date.
-- Monthly processing reads validated database facts rather than repeating Agent business logic.
-- Database `total_price` may support validation, but the exported `TOTAL PRICE` cell must remain a formula.
-- Container deployment enables XML processing through both the Dockerfile default Web CMD and the explicit Compose command, and still requires `processing_ready=true`; direct source CLI execution remains default-closed.
+- The uploaded source is re-materialized from committed object storage before processing.
+- Login documents/assets, `POST /api/login` and minimal `/healthz` readiness are the only anonymous Web routes. Portal documents redirect to login; business APIs, detailed health, uploads, traces and downloads return `401` without an authenticated server-side session.
+- Operator credentials come only from runtime `ARR_WEB_USERNAME` / `ARR_WEB_PASSWORD`; the application fails closed when either is absent. Authenticated mutations also require the per-session CSRF token.
+- Processor exit code and JSON status must agree; output paths are confined to an isolated temporary directory.
+- Validation completes before the database transaction; any transaction failure leaves no partial active version.
+- Booking uploads create review drafts first. Only an explicit activation with no retained pending items can atomically
+  create immutable Booking facts and switch `booking.current_source_batch`; an open draft blocks company-report creation.
+- Only recognized transient PostgreSQL concurrency errors receive bounded transaction retries.
+- PostgreSQL state, not HTTP/console output, is authoritative for success.
+- Daily downloads are read from OSS and rechecked; monthly downloads resolve only registered active/superseded local artifacts and recheck path, size and SHA-256.
+- Fresh uploads are new jobs/versions; exact delivery replay is idempotent.
+- Monthly success is acknowledged only after both artifacts are registered and the publication is active; replay of the same snapshot returns the existing report.
 
 ## Related Decisions
 
+- [ADR-004](../10-decisions/ADR-004-arr-owned-programmatic-processing.md)
 - [ADR-001](../10-decisions/ADR-001-automatic-monthly-trigger-and-total-price-formula.md)
 
 ## Last Updated
 
-2026-07-29
+2026-07-31