fix: recognize corroborated empty Oracle associations
This commit is contained in:
1 parent
4559d5a642
commit
de9eb532f2
9 files changed
+735
-28
No files matched your search
@@ -21,10 +21,10 @@
|
||||
## Outcome
|
||||
|
||||
- Production direct reads now omit unsupported server sorting and use the platform's working GET `getProfiles` query. Original acquisition order, complete paging/recheck and response identity checks remain enforced. Legacy collection keeps its original sorting default.
|
||||
- Added source-field completion before deterministic processing: only candidate records' unresolved/invalid fields may be edited; optional absence requires an explicit decision. Original bytes are retained, revisioned decisions record the authenticated actor, and confirmation freezes a derived source. The same download request resumes from its saved acquisition without querying OHIP again. Existing price review, Finance validation, daily generation and monthly outbox behavior remain in use; XML processing is retained.
|
||||
- Added source-field completion before deterministic processing: only candidate records' unresolved/invalid fields may be edited. Normal optional block/package omission is now recognized under the narrow Oracle association policy described below; unresolved, failed or contradictory evidence still needs resolution. Original bytes are retained, revisioned staff decisions record the authenticated actor, and confirmation freezes a derived source. The same download request resumes from its saved acquisition without querying OHIP again. Existing price review, Finance validation, daily generation and monthly outbox behavior remain in use; XML processing is retained.
|
||||
- Browser recovery is scoped to this service/source context and user, preventing an old sandbox request from restoring into the production instance. Lost save/finalize responses, concurrent revisions and interrupted confirmation recover without a second processing intent.
|
||||
- Updated the existing local service at `http://127.0.0.1:8875/` to run this isolated checkout, retaining hotel57106, credentials, private database and output storage. Previous private launcher/config/plist were backed up before restart. No production deployment, remote push or integration into the unowned dirty primary checkout occurred.
|
||||
- Reused the complete, immutable real 2026-10-07 capture; verified all531 inventory files and both original hashes before importing into request `45e0e20ee75a46e098a0e32d650133a9`. Import made zero new Oracle calls. The live task is `needs_data_review`, no processing job exists, and it has26 unresolved fields across19 rows:1 room,18 block codes and7 package lists. No real business values were supplied or implicitly confirmed; no real daily/monthly was generated.
|
||||
- Reused the complete, immutable real 2026-10-07 capture; verified all531 capture files and both original hashes before importing into request `45e0e20ee75a46e098a0e32d650133a9`. Its initial26 source decisions were1 room,18 block codes and7 package lists. The optional-association follow-up below reinterpreted only corroborated normal omissions, preserving all original bytes and staff decisions. Current live review is revision1 with only1 unresolved required room; no processing job, staff confirmations or real daily/monthly exist. Import and reanalysis made zero new Oracle calls.
|
||||
|
||||
## Verification
|
||||
|
||||
@@ -38,7 +38,7 @@
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- Reconcile the API source selection with the original Oracle report before asking staff to resolve the26 currently presented field decisions. The later original-XML audit below proves that15 decisions concern reservations absent from the manual10/7 report and11 concern optional values explicitly blank in that report. Neither these source differences nor the original optional-field rules can be resolved by asking staff to invent values. Subsequent existing price review and real daily/monthly acceptance remain pending. No synthetic room, source confirmation, or status-based exclusion has been introduced.
|
||||
- The user explicitly deferred API/XML population alignment. That issue remains separate and must not block fixing normal optional omission or be represented as resolved by this change. The remaining10/7 room belongs to confirmation300007418, a Cancelled/unassigned source reservation; do not invent a room or silently exclude the reservation. Real report acceptance still requires resolving the actual remaining input and, when resumed by the user, the desired source selection/text mappings. Existing price review and real daily/monthly acceptance remain pending.
|
||||
- Integration owner should promote the accepted behavior and production compatibility evidence, then integrate this branch into the primary project. Keep this checkout while the local service runs from it. It is manually created, not eligible for automatic skill-managed retirement.
|
||||
- Production deployment/configuration and remote publishing are separate follow-ups. This turn only activated the local instance against the existing production read service.
|
||||
|
||||
@@ -46,7 +46,7 @@
|
||||
|
||||
- Update canonical current-state/data-flow/success-criteria with the separate upstream source-field review and its boundary with the existing price-only processor/Finance review. Preserve ADR-006's original scope; decide whether an additional ADR is appropriate at integration.
|
||||
- Promote the four supported direct search parameters, GET profile lookup compatibility, service-context browser recovery and immutable original/manual decision audit into integration-owned documentation.
|
||||
- Record that real10/7 acquisition is complete but its daily/monthly acceptance awaits staff decisions. Avoid claiming successful report generation from synthetic regression evidence.
|
||||
- Record that real10/7 acquisition is complete and normal optional omission is resolved under a constrained source policy; daily/monthly acceptance still awaits the remaining input and separately deferred source selection. Avoid claiming successful report generation from synthetic regression evidence.
|
||||
|
||||
## Concurrent Task Gate
|
||||
|
||||
@@ -125,3 +125,16 @@ Read: memory-index, project-positioning, current-state latest September sections
|
||||
- Public CLI0.5.0 catalog inspection succeeds against the existing Edge service, version0.12.0: searchHotelReservations remains a synchronous POST with an optional OracleDocument body and no typed mapping to res_detail report criteria. Only public metadata was requested; no credentials, grants, environment routing or hotel business data were read or changed. Private evidence: `production-validation-20261007/search-reservations-public-contract-20261008.json`.
|
||||
- Official documentation does not establish the current hotel's historical report status defaults. Oracle's [legacy RES_DETAIL parameters](https://docs.oracle.com/cd/E98457_01/opera_5_6_core_help/res_detail_help.htm) distinguish arrival/stay dates and include options from Reservations=All. Its Checked-In Today option refers to the current business date; this is not proof that a historical Cloud report excludes every InHouse reservation or includes only CheckedOut. The current [Oracle reservation API schema](https://raw.githubusercontent.com/oracle/hospitality-api-docs/main/rest-api-specs/property/v1/rsv.json) independently exposes actualArrivals, expectedArrivals and reservationStatuses, without defining their equivalence to res_detail. An independent read-only official-document audit reached the same limit; current Cloud report summaries do not supply the missing per-hotel saved parameter values.
|
||||
- Outcome: the user-confirmed two-day range explains the extra10/8 XML group only. Request the actual Oracle export parameter page, especially the date-type and Include selections, before choosing API inclusion criteria. Do not ask for the date again, infer status answers, or introduce CheckedOut-only filtering from this one sample. No code, source data, review decisions or service configuration changed, and no actual daily/monthly was generated. Public metadata/documentation reads required no business query or tests; Git whitespace and task documentation boundary checks apply to this documentation-only continuation.
|
||||
|
||||
## Same-task Follow-up: Optional Information Versus Mandatory Anomalies
|
||||
|
||||
- User explicitly defers API/XML reservation-population alignment and asks to resolve the earlier explanation of26 pending fields:25 optional-source confirmations versus1 original mandatory room error. Subsequent “继续” authorizes checking omission semantics and correcting the related product behavior. Do not resume status/date-scope changes as a prerequisite for this narrower work.
|
||||
- Concurrent gate Passed: resumed the same task, ownercodex, feature mode, branchcodex/arr-production-review, existing absolute worktree and base unchanged; no other owners. Existing planning context/ADR-004/006/007 and prior rule/history audits remain applicable; the user's production-read and multiple-agent consent supersede older September restrictions. Primary checkout's unknown changes remain untouched.
|
||||
- Project Context Loaded / Planning gate Passed: direct JSON and XML retain shared deterministic business rules; optional fields may legitimately be blank, required room values cannot be fabricated, original capture/manual decisions remain immutable and auditable. Scope is integrations/ohip optional observations and source-review page explanation/tests. Public contract reads and saved production capture may establish omission behavior; optional schema presence alone must not be claimed as proof that a failed/ambiguous read is empty. No new hotel business read is needed for the initial investigation.
|
||||
- Plan: independently review official/request contracts and existing capture completeness; distinguish proven empty observations from failed/ambiguous/unestablished values without inventing data. Correct the page's explanation of mandatory versus optional unresolved information and acquisition failure; add meaningful regression for affected branches. Activate verified local changes only after checking task activity and preserve actual staff decisions. Report the actual remaining confirmation count honestly, without promising automatic blank handling if contractual evidence is insufficient.
|
||||
- Research outcome: Oracle's official HTNG mapping directly identifies `getReservation` as the operation for booked packages; requested-element semantics and optional block/package models, combined with successful corroborated search/detail responses, support a narrow normal-omission policy. This is a reasoned conditional implementation, not a claim that Oracle explicitly guarantees all omitted structures are empty. The question to the platform colleague is withdrawn; no colleague determination or new grant/key is needed. Evidence and precise conditions are recorded in `50-evidence/topics/20261008-production-review-9e7b__oracle-optional-associations.md`.
|
||||
- Implementation outcome: fixed policy `oracle-optional-association/v1` resolves normal empty block/package associations and retains missing evidence, failed reads, malformed structures and contradictions. Source-review UI now distinguishes original mandatory/optional rules from fetch failure, conflict and invalid format. The original processor/validator/XML and date/status population are unchanged.
|
||||
- Existing-review outcome: added internal fixed-projection reanalysis, no public route. Original acquisition bytes/manifest/checkpoints and all manual decisions remain immutable/preserved; reanalysis is separately hash-pinned and revisioned, not recorded as staff confirmation. Final audit binds original/reanalysis sources and receipts; finalization remains explicit, with concurrency/idempotency/tamper/interrupted-publication tests.
|
||||
- Verification:88 Python tests passed with disposable real local PostgreSQL enabled, no skips (33 source-review/executor/database and55 source acquisition tests);44 JavaScript tests passed. Independent policy review caught malformed indicator-name and contradictory empty-block edge cases, both fixed and retested. Offline real replay verified531 original files and all176 recorded HTTP requests with zero network calls, preserving58 rows, every other source/field/context/related value and order.36 full-source block observations and16 package observations became empty, resolving25 candidate decisions.
|
||||
- Local activation and observed result: dry-run on a private copy reduced26 decisions to1 without manual decisions or finalization. Verified no active queued/downloading/processing task before service stop; backed up review and executor checkpoint to private `source-reanalysis-backup-20261008-105211`, applied the same checked reanalysis, then restarted the existing service. Authenticated API and CUA both show10/7 revision1,0/1 confirmed, only required room for300007418, `can_finalize=false`, no job/daily/monthly. Page retained as deliverable with10/7 selected; XML upload and separate9/17 pending entry remain. The existing9/17 capture/review was not reinterpreted or queried. Screenshot and full replay receipts remain private outside Git.
|
||||
- Follow-up/promotion candidate: promote the documented limited optional-association semantics and product explanations at integration. Remaining10/7 room is not fabricated; its cancelled/unassigned report inclusion remains a separately deferred business-scope question. Real daily/monthly acceptance, primary-checkout integration and remote deployment are not claimed by this task.
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
# Oracle optional reservation associations
|
||||
|
||||
## Question and scope
|
||||
|
||||
The user asked why10/7 showed26 manual field decisions when XML rules allow empty block and package fields, and instructed Codex to research Oracle directly after the platform colleague could not establish the return semantics. The user expressly deferred comparing API and XML reservation populations. This record does not establish report-selection equivalence or authorize status filtering.
|
||||
|
||||
## Official evidence
|
||||
|
||||
- Oracle's [HTNG migration mapping](https://docs.oracle.com/en/industries/hospitality/integration-platform/ohipu/c_htng.htm) maps Retrieve Booked Packages / FetchBookedPackages to `getReservation`, while configuration package details use a different operation.
|
||||
- The official [RSV schema](https://github.com/oracle/hospitality-api-docs/blob/main/rest-api-specs/property/v1/rsv.json) describes cumulative `fetchInstructions`. The current application explicitly requests `Packages`, `Reservation`, `DailySummary` and the other existing elements; `reservationPackages` and block-association structures are optional.
|
||||
- Oracle's [24.1 property API release notes](https://docs.oracle.com/en/industries/hospitality/opera-cloud/24.1/oprnc/c_feature_summary.htm) describe `reservationBlock` information returned for reservations associated with a block.
|
||||
- Public Edge catalog0.12.0 shows `getPackage` requires a known `productCode`; it cannot discover an unknown attached-package list. `getReservationsByPackage` is a consumption-date query, not an equivalent full single-reservation package-list retrieval. No permission or credential change is needed.
|
||||
|
||||
These sources establish operation purpose, requested-element semantics and optional association models. They do not contain a universal literal guarantee that every absent field means none. The implementation is a conditional contract interpretation supported by those semantics and corroborating successful captured responses; failed, ambiguous or malformed reads are not converted to empty.
|
||||
|
||||
## Narrow policy: oracle-optional-association/v1
|
||||
|
||||
- Omitted packages may become empty only after successful, validated reservation detail requested with `Packages`, with an explicit valid search indicator list. Every indicator has a nonempty string name; `PACKAGEITEM` must be absent or occur once with a legal integer zero. Positive, missing, negative, boolean or duplicate package counts remain unresolved; a positive count also contradicts explicit `[]`.
|
||||
- Omitted block association may become empty only when search and arrival-day detail both omit it, with no additional detail-level block structure. Single-sided omission, explicit null/invalid structures and code/identity disagreement remain unresolved. Existing explicit empty-ID handling also checks all available layers for structure, hotel and nonempty block-name contradictions.
|
||||
- An identified block's failed supplementary read stays failed. Other optional fields receive no blanket omission rule. Original whitelist, required-field validation, room/date deduplication, pricing and XML logic are unchanged.
|
||||
- UI descriptions separately identify original required/optional rules, read failure, conflicting information and invalid format.
|
||||
|
||||
## Saved10/7 evidence and safe replay
|
||||
|
||||
Original complete capture:58 ordered rows,176 successful HTTP responses,531 files including manifest.42 detail rows have nonempty package arrays and positive `PACKAGEITEM`;16 omit packages with valid lists and no package indicator.22 search/day pairs contain block associations;36 omit both. No positive package indicator accompanies missing packages.
|
||||
|
||||
Original capture manifest SHA256: `bcfeb6390fc553754b917f9c935a532916e6bd10d4e95ee43450895e760cd9fc`.
|
||||
Original ARR data SHA256: `805799ba2f3edd7f1cf18c14e5304908061d6194a241c5fc255192728b7f388a`.
|
||||
|
||||
Offline-only recorded transport verified every original file hash/size, matched all176 method/path/body requests, replayed exact response bytes, and made zero network calls. New private capture changed only36 missing block observations and16 missing package observations to empty, plus the named policy tag; all other fields, record order, context, source references and related facts were strictly equal. These52 source observations include25 manual candidate decisions (18 block,7 package); noncandidate rows need no manual decisions. Two remaining company gaps are outside the original rate whitelist; only the original required-room issue remains in review.
|
||||
|
||||
Reanalysis manifest SHA256: `d39177c5aad3c5b235c04aab8693fca51580c15f6b5210aa2d4871276d8e3507`.
|
||||
Reanalysis ARR data SHA256: `2abbebb36a86f701722677ce103e4b3e02fffeb0c369bacd0e0c3bbbb9e9246f`.
|
||||
Private evidence directory: `/Users/chillishark/Library/Application Support/ARR2.0/production-validation-20261007/optional-association-reanalysis-e89e46a9e65c`.
|
||||
|
||||
## Existing review preservation and result
|
||||
|
||||
Internal maintenance only, no new HTTP route: `DataFieldReviews.apply_source_reanalysis` accepts the fixed policy and exact missing-reason-to-empty projection, preserving original acquisition/checkpoint hashes and all staff decisions. Reinterpretation bytes and receipt are immutable and hash-referenced; a separate revisioned `source_reanalysis` event is recorded without fabricating staff confirmations. Manual decisions always take priority. Finalization remains explicit and frozen reports include both original and reanalysis evidence. Stale/concurrent updates, unrelated value edits, tampering and changes after finalization are rejected; same evidence retries are idempotent and interrupted publication recovers.
|
||||
|
||||
After dry-run validation on a private copy and confirming zero active queue tasks, the existing local service was stopped, original review/executor checkpoint backed up,10/7 reanalysis applied and the same service restarted. Local authenticated reads and CUA page inspection show revision1,0/1 confirmed, only `DISP_ROOM_NO` for confirmation300007418, disabled generation, no job and both10/7 and9/17 pending entries. The unrelated existing9/17 review is unchanged; historical pending captures are not silently rewritten by changing future acquisition code. No real daily/monthly was generated.
|
||||
|
||||
Verification:88 targeted Python tests passed with real disposable local PostgreSQL and no skips;44 JavaScript tests passed; syntax/whitespace checks passed. Independent source-policy review found two edge cases (malformed indicator name and contradictory explicit empty block association); both were fixed with regression tests and independently rechecked. Screenshot in the private evidence directory: `local-review-after-reanalysis.png`.
|
||||
|
||||
## Promotion candidate
|
||||
|
||||
Integration owner should promote the conditional optional-association policy and required/optional/failure distinction to canonical source-contract/product documentation, with this evidence and its limits. Preserve the separate unresolved cancelled/unassigned reservation selection question and do not claim a successful real daily/monthly or exact XML equivalence. No policy reversal or new human approval is required for normal optional blanks already allowed by the original business rules.
|
||||
+141
-10
@@ -28,6 +28,8 @@ from integrations.ohip.collect_arr_source import json_bytes, require, strict_jso
|
||||
from integrations.ohip.collect_arr_source import CollectionError
|
||||
|
||||
VERSION = "arr-source-field-review/v1"
|
||||
REANALYSIS_POLICY = "oracle-optional-association/v1"
|
||||
REANALYSIS_VERSION = "arr-source-reanalysis/v1"
|
||||
LIMIT = 100 * 1024 * 1024
|
||||
OPTIONAL = frozenset({"BLOCK_CODE", "RES_COMMENT", "PRODUCTS", "ROOM_CATEGORY_LABEL"})
|
||||
LABELS = {
|
||||
@@ -128,6 +130,12 @@ class DataFieldReviews:
|
||||
state = strict_json(state_raw)
|
||||
require(state["identity_sha256"] == fingerprint(meta) and type(state["revision"]) is int
|
||||
and state["status"] in {"editing", "finalized"}, "data_review_state_context_changed")
|
||||
if "source_reanalysis" in state:
|
||||
reference = state["source_reanalysis"]
|
||||
require(type(reference) is dict and reference.get("parent_original_sha256") == meta["original_sha256"]
|
||||
and reference.get("source_manifest_sha256") == meta["source_manifest_sha256"],
|
||||
"data_review_reanalysis_context_changed")
|
||||
self._reanalyzed_source(directory, original, state)
|
||||
return meta, original, state
|
||||
|
||||
def _publish(self, directory, state):
|
||||
@@ -148,8 +156,74 @@ class DataFieldReviews:
|
||||
value = [value]
|
||||
return {"state": "available", "value": value}
|
||||
|
||||
def _derive(self, original, state):
|
||||
data = copy.deepcopy(original)
|
||||
@staticmethod
|
||||
def _reanalysis_projection(original, reanalyzed, policy_id):
|
||||
require(policy_id == REANALYSIS_POLICY and type(reanalyzed) is dict,
|
||||
"data_review_reanalysis_policy_invalid")
|
||||
expected = copy.deepcopy(original)
|
||||
require(type(reanalyzed.get("records")) is list
|
||||
and len(reanalyzed["records"]) == len(original["records"]),
|
||||
"data_review_reanalysis_records_changed")
|
||||
changes = []
|
||||
allowed = {"BLOCK_CODE": ("missing_reservation_block", ""),
|
||||
"PRODUCTS": ("missing_reservation_packages", [])}
|
||||
for index, (old, new) in enumerate(zip(original["records"], reanalyzed["records"])):
|
||||
require(type(new) is dict and type(new.get("fields")) is dict,
|
||||
"data_review_reanalysis_record_invalid")
|
||||
for field, (reason, empty_value) in allowed.items():
|
||||
before, after = old["fields"][field], new["fields"].get(field)
|
||||
if fingerprint(before) == fingerprint(after):
|
||||
continue
|
||||
empty = {"state": "empty", "value": empty_value}
|
||||
require(before.get("state") == "missing" and before.get("reason") == reason
|
||||
and fingerprint(after) == fingerprint(empty),
|
||||
"data_review_reanalysis_observation_invalid")
|
||||
expected["records"][index]["fields"][field] = copy.deepcopy(empty)
|
||||
changes.append({"item_id": f"{old['source_sequence']}:{field}",
|
||||
"source_sequence": old["source_sequence"], "field": field,
|
||||
"original_observation": copy.deepcopy(before), "reanalyzed_observation": copy.deepcopy(empty)})
|
||||
require(bool(changes), "data_review_reanalysis_no_changes")
|
||||
unresolved = any(observation.get("state") != "available"
|
||||
and not (field in OPTIONAL and observation.get("state") == "empty")
|
||||
for row in expected["records"] for field, observation in row["fields"].items())
|
||||
expected["input_complete"] = not unresolved
|
||||
expected["status"] = "collected_with_gaps" if unresolved else "collected"
|
||||
expected["optional_omission_policy"] = REANALYSIS_POLICY
|
||||
require(fingerprint(expected) == fingerprint(reanalyzed), "data_review_reanalysis_source_changed")
|
||||
return changes
|
||||
|
||||
def _reanalyzed_source(self, directory, original, state):
|
||||
reference = state.get("source_reanalysis")
|
||||
if reference is None:
|
||||
return original, None
|
||||
require(type(reference) is dict and reference.get("policy_id") == REANALYSIS_POLICY,
|
||||
"data_review_reanalysis_reference_invalid")
|
||||
digest, receipt_digest = reference.get("sha256", ""), reference.get("receipt_sha256", "")
|
||||
manifest_digest = reference.get("manifest_sha256", "")
|
||||
require(type(digest) is str and bool(re.fullmatch(r"[0-9a-f]{64}", digest))
|
||||
and type(receipt_digest) is str and bool(re.fullmatch(r"[0-9a-f]{64}", receipt_digest))
|
||||
and type(manifest_digest) is str and bool(re.fullmatch(r"[0-9a-f]{64}", manifest_digest))
|
||||
and reference.get("file") == f"reanalyzed-source-{digest}.json"
|
||||
and reference.get("receipt_file") == f"source-reanalysis-{receipt_digest}.json",
|
||||
"data_review_reanalysis_reference_invalid")
|
||||
payload = protected_read(directory / reference["file"], LIMIT)
|
||||
receipt_raw = protected_read(directory / reference["receipt_file"], LIMIT)
|
||||
require(_hash(payload) == digest and _hash(receipt_raw) == receipt_digest,
|
||||
"data_review_reanalysis_changed")
|
||||
reanalyzed, receipt = strict_json(payload), strict_json(receipt_raw)
|
||||
changes = self._reanalysis_projection(original, reanalyzed, REANALYSIS_POLICY)
|
||||
expected_receipt = {"version": REANALYSIS_VERSION, "policy_id": REANALYSIS_POLICY,
|
||||
"request_id": directory.name, "report_date": original["report_date"], "context": self.context,
|
||||
"parent_original_sha256": reference["parent_original_sha256"],
|
||||
"source_manifest_sha256": reference["source_manifest_sha256"],
|
||||
"reanalyzed_sha256": digest, "reanalysis_manifest_sha256": reference["manifest_sha256"],
|
||||
"changes": changes}
|
||||
require(fingerprint(receipt) == fingerprint(expected_receipt), "data_review_reanalysis_receipt_changed")
|
||||
return reanalyzed, receipt
|
||||
|
||||
def _derive(self, directory, original, state):
|
||||
source, _receipt = self._reanalyzed_source(directory, original, state)
|
||||
data = copy.deepcopy(source)
|
||||
for item_id, decision in state["decisions"].items():
|
||||
sequence, field = item_id.split(":")
|
||||
row = data["records"][int(sequence) - 1]
|
||||
@@ -197,7 +271,7 @@ class DataFieldReviews:
|
||||
return value if observation.get("state") == "available" and isinstance(value, (str, int)) else ""
|
||||
|
||||
def _public(self, directory, meta, original, state):
|
||||
data = self._derive(original, state)
|
||||
data = self._derive(directory, original, state)
|
||||
issues = self._issues(directory, data)
|
||||
keys = set(issues) | set(state["decisions"])
|
||||
items = []
|
||||
@@ -209,6 +283,7 @@ class DataFieldReviews:
|
||||
"field_label": LABELS[field], "confirmation_no": self._display(row, "CONFIRMATION_NO"),
|
||||
"room_no": self._display(row, "DISP_ROOM_NO"), "company_name": self._display(row, "COMPANY_NAME"),
|
||||
"rate_code": self._display(row, "RATE_CODE"), "reason_code": issues.get(item_id, "confirmed"),
|
||||
"source_state": row["fields"][field].get("state"),
|
||||
"can_be_empty": field in OPTIONAL, "value": decision["value"] if decision else None,
|
||||
"confirmed": decision is not None and item_id not in issues})
|
||||
return {"request_id": meta["request_id"], "report_date": meta["report_date"],
|
||||
@@ -245,13 +320,66 @@ class DataFieldReviews:
|
||||
require(fingerprint(actual) == fingerprint(meta), "data_review_prepare_conflict")
|
||||
review = self._public(directory, actual, original, state)
|
||||
# A complete ordinary source needs no human step or derived source.
|
||||
return review if review["total_count"] else None
|
||||
# A reanalysis remains an explicit review until finalized, even if
|
||||
# its validated source interpretation resolves every listed gap.
|
||||
return review if review["total_count"] or "source_reanalysis" in state else None
|
||||
|
||||
def get(self, request_id):
|
||||
directory = self._directory(request_id)
|
||||
with self._lock(directory):
|
||||
return self._public(directory, *self._read(directory))
|
||||
|
||||
def apply_source_reanalysis(self, request_id, payload, manifest_sha256, policy_id, *, expected_revision):
|
||||
"""Attach a validated offline reinterpretation, without changing acquisition or staff decisions.
|
||||
|
||||
This internal maintenance operation has no HTTP route. Its caller must
|
||||
establish the policy's omission evidence from the saved complete capture.
|
||||
The projection here prevents any other source or business-value edits.
|
||||
"""
|
||||
require(policy_id == REANALYSIS_POLICY, "data_review_reanalysis_policy_invalid")
|
||||
require(type(payload) is bytes and len(payload) <= LIMIT,
|
||||
"data_review_reanalysis_payload_invalid")
|
||||
require(type(manifest_sha256) is str and bool(re.fullmatch(r"[0-9a-f]{64}", manifest_sha256)),
|
||||
"data_review_reanalysis_manifest_invalid")
|
||||
reanalyzed = strict_json(payload)
|
||||
directory = self._directory(request_id)
|
||||
with self._lock(directory):
|
||||
meta, original, state = self._read(directory)
|
||||
if state["status"] != "editing" or (directory / "finalize-intent.json").exists():
|
||||
raise _error("FROZEN", "已确认生成,来源解释不能再修改", 409)
|
||||
if type(expected_revision) is not int:
|
||||
self._revision(state, expected_revision)
|
||||
digest = _hash(payload)
|
||||
prior = state.get("source_reanalysis")
|
||||
if prior is not None:
|
||||
if prior["sha256"] == digest and prior["manifest_sha256"] == manifest_sha256:
|
||||
return self._public(directory, meta, original, state)
|
||||
raise _error("CONFLICT", "该来源已完成重新解释,请核对现有记录", 409)
|
||||
self._revision(state, expected_revision)
|
||||
changes = self._reanalysis_projection(original, reanalyzed, policy_id)
|
||||
receipt = {"version": REANALYSIS_VERSION, "policy_id": policy_id,
|
||||
"request_id": request_id, "report_date": meta["report_date"], "context": self.context,
|
||||
"parent_original_sha256": meta["original_sha256"],
|
||||
"source_manifest_sha256": meta["source_manifest_sha256"], "reanalyzed_sha256": digest,
|
||||
"reanalysis_manifest_sha256": manifest_sha256, "changes": changes}
|
||||
receipt_raw = json_bytes(receipt)
|
||||
receipt_digest = _hash(receipt_raw)
|
||||
reference = {"policy_id": policy_id, "file": f"reanalyzed-source-{digest}.json", "sha256": digest,
|
||||
"receipt_file": f"source-reanalysis-{receipt_digest}.json", "receipt_sha256": receipt_digest,
|
||||
"parent_original_sha256": meta["original_sha256"],
|
||||
"source_manifest_sha256": meta["source_manifest_sha256"], "manifest_sha256": manifest_sha256}
|
||||
_write_once(directory / reference["file"], payload)
|
||||
_write_once(directory / reference["receipt_file"], receipt_raw)
|
||||
state["source_reanalysis"] = reference
|
||||
state["revision"] += 1
|
||||
state["events"].append({"revision": state["revision"], "action": "source_reanalysis",
|
||||
"policy_id": policy_id, "parent_original_sha256": meta["original_sha256"],
|
||||
"reanalyzed_sha256": digest, "reanalysis_manifest_sha256": manifest_sha256,
|
||||
"receipt_sha256": receipt_digest, "at": datetime.now(timezone.utc).isoformat()})
|
||||
result = self._public(directory, meta, original, state)
|
||||
self._publish(directory, state)
|
||||
return result
|
||||
|
||||
@staticmethod
|
||||
def _revision(state, revision):
|
||||
if type(revision) is not int or revision != state["revision"]:
|
||||
@@ -312,7 +440,7 @@ class DataFieldReviews:
|
||||
raise _error("FROZEN", "已确认生成,字段不能再修改", 409)
|
||||
self._revision(state, revision)
|
||||
actor = self._actor(actor)
|
||||
data = self._derive(original, state)
|
||||
data = self._derive(directory, original, state)
|
||||
if item_id not in self._issues(directory, data) and item_id not in state["decisions"]:
|
||||
raise _error(message="只能完善当前任务列出的异常字段")
|
||||
sequence, field = item_id.split(":")
|
||||
@@ -328,7 +456,7 @@ class DataFieldReviews:
|
||||
self._publish(directory, state)
|
||||
return result
|
||||
|
||||
def _frozen(self, meta, original, state):
|
||||
def _frozen(self, directory, meta, original, state):
|
||||
changes = []
|
||||
for item_id, decision in sorted(state["decisions"].items()):
|
||||
sequence, field = item_id.split(":")
|
||||
@@ -339,7 +467,10 @@ class DataFieldReviews:
|
||||
"context": self.context, "original_sha256": meta["original_sha256"],
|
||||
"source_manifest_sha256": meta["source_manifest_sha256"], "revision": state["revision"],
|
||||
"changes": changes, "events": state["events"]}
|
||||
data = self._derive(original, state)
|
||||
_source, receipt = self._reanalyzed_source(directory, original, state)
|
||||
if receipt is not None:
|
||||
audit["source_reanalysis"] = {**receipt, "receipt_sha256": state["source_reanalysis"]["receipt_sha256"]}
|
||||
data = self._derive(directory, original, state)
|
||||
data["manual_data_review"] = {"manifest": audit, "manifest_sha256": fingerprint(audit)}
|
||||
binding = fingerprint({"source_manifest_sha256": meta["source_manifest_sha256"],
|
||||
"review_manifest_sha256": fingerprint(audit), "data_sha256": _hash(json_bytes(data))})
|
||||
@@ -354,7 +485,7 @@ class DataFieldReviews:
|
||||
return self._public(directory, meta, original, state)
|
||||
self._revision(state, revision)
|
||||
actor = self._actor(actor)
|
||||
if self._issues(directory, self._derive(original, state)):
|
||||
if self._issues(directory, self._derive(directory, original, state)):
|
||||
raise _error("INCOMPLETE", "请先完善并确认所有异常字段", 409)
|
||||
intent = directory / "finalize-intent.json"
|
||||
prior_sha256 = fingerprint(state)
|
||||
@@ -368,14 +499,14 @@ class DataFieldReviews:
|
||||
state["events"].append({"revision": state["revision"], "action": "finalize", "actor": actor,
|
||||
"at": datetime.now(timezone.utc).isoformat()})
|
||||
atomic_json(intent, {"prior_sha256": prior_sha256, "state": state}, replace=False)
|
||||
payload, binding = self._frozen(meta, original, state)
|
||||
payload, binding = self._frozen(directory, meta, original, state)
|
||||
_write_once(directory / "reviewed-source.json", payload)
|
||||
state["derived_sha256"], state["binding_manifest_sha256"] = _hash(payload), binding
|
||||
self._publish(directory, state)
|
||||
return self._public(directory, meta, original, state)
|
||||
|
||||
def _verify_frozen(self, directory, meta, original, state):
|
||||
payload, binding = self._frozen(meta, original, state)
|
||||
payload, binding = self._frozen(directory, meta, original, state)
|
||||
require(_hash(payload) == state["derived_sha256"] and binding == state["binding_manifest_sha256"]
|
||||
and protected_read(directory / "reviewed-source.json", LIMIT) == payload,
|
||||
"data_review_frozen_source_changed")
|
||||
|
||||
+20
-3
@@ -633,6 +633,23 @@
|
||||
return Array.isArray(item.value) ? item.value.join("\n") : String(item.value ?? "");
|
||||
}
|
||||
|
||||
function arrDataReviewReasonKey(item) {
|
||||
if (item.confirmed) return "confirmed_value";
|
||||
const sourceState = String(item.source_state || "").toLowerCase();
|
||||
const reason = String(item.reason_code || "").toUpperCase();
|
||||
// Older review snapshots do not expose source_state. Recognize only the
|
||||
// collector's explicit failure codes; a missing optional field is not a
|
||||
// failed request and does not make the field required.
|
||||
const failedReasons = ["HTTP_FAILURE", "HTTP_PERMISSION_DENIED", "TRANSPORT_RETRY_EXHAUSTED",
|
||||
"REQUEST_LIMIT_EXCEEDED", "RESPONSE_TOO_LARGE", "OPERATION_MISMATCH", "HOTEL_MISMATCH",
|
||||
"INVALID_DATA_ENVELOPE", "MISSING_ORACLE_REQUEST_ID", "SECRET_IN_RESPONSE", "INVALID_JSON",
|
||||
"UPSTREAM_WARNING_OR_ERROR"];
|
||||
if (sourceState === "failed" || failedReasons.includes(reason)) return "fetch_failed";
|
||||
if (sourceState === "ambiguous" || /(?:^|_)(?:CONFLICT|AMBIGUOUS|MISMATCH|DISAGREEMENT|CHANGED)(?:_|$)/.test(reason)) return "conflict_value";
|
||||
if (/(?:^|_)(?:INVALID|MALFORMED)(?:_|$)/.test(reason) || ["XML_NEGATIVE_NIGHTS", "DATA_NEGATIVE_NIGHTS"].includes(reason)) return "invalid_value";
|
||||
return item.can_be_empty ? "source_unconfirmed" : "required_value_missing";
|
||||
}
|
||||
|
||||
function arrDataReviewCanFinalize(review = state.arrDataReview) {
|
||||
return Boolean(arrDownloadNeedsDataReview() && state.arrDownloadTask.request_id === state.arrDataReviewRequestId
|
||||
&& !state.arrDownloadBusy && review && review.status === "editing" && review.can_finalize && Number(review.pending_count) === 0
|
||||
@@ -675,8 +692,8 @@
|
||||
const labelKey = `data_review.field_${field.toLowerCase()}`;
|
||||
const translatedLabel = I18N.t(labelKey);
|
||||
const label = translatedLabel === labelKey ? (item.field_label || field) : translatedLabel;
|
||||
const reason = String(item.reason_code || "").toUpperCase();
|
||||
const reasonKey = item.confirmed ? "confirmed_value" : /CONFLICT|AMBIGUOUS/.test(reason) ? "conflict_value" : /INVALID|MALFORMED/.test(reason) ? "invalid_value" : "missing_value";
|
||||
const reasonKey = arrDataReviewReasonKey(item);
|
||||
const ruleKey = item.can_be_empty ? "optional_field_rule" : "required_field_rule";
|
||||
const inputId = `arr-data-review-value-${index}`;
|
||||
const value = Object.prototype.hasOwnProperty.call(state.arrDataReviewDrafts, item.item_id)
|
||||
? state.arrDataReviewDrafts[item.item_id] : arrDataReviewInputValue(item);
|
||||
@@ -687,7 +704,7 @@
|
||||
: `<input id="${inputId}" data-arr-data-review-value type="text" inputmode="${inputMode}" autocomplete="off" value="${escapeHtml(value)}" placeholder="${escapeHtml(placeholder)}" ${editable ? "" : "disabled"} />`;
|
||||
return `<tr data-arr-data-review-item-id="${escapeHtml(item.item_id)}" class="${item.confirmed ? "is-complete" : ""}">
|
||||
<td data-label="${escapeHtml(I18N.t("data_review.reservation"))}"><strong>${escapeHtml(item.confirmation_no || "—")}</strong><small>${escapeHtml(I18N.t("data_review.source_row", { sequence: item.source_sequence }))} · ${escapeHtml(I18N.t("data_review.room", { room: item.room_no || "—" }))}</small><small>${escapeHtml(item.company_name || "—")} · ${escapeHtml(item.rate_code || "—")}</small></td>
|
||||
<td data-label="${escapeHtml(I18N.t("data_review.field"))}"><label for="${inputId}">${escapeHtml(label)}</label><small>${escapeHtml(I18N.t(`data_review.${reasonKey}`))}</small></td>
|
||||
<td data-label="${escapeHtml(I18N.t("data_review.field"))}"><label for="${inputId}">${escapeHtml(label)}</label><small>${escapeHtml(I18N.t(`data_review.${ruleKey}`))}</small><small>${escapeHtml(I18N.t(`data_review.${reasonKey}`))}</small></td>
|
||||
<td data-label="${escapeHtml(I18N.t("data_review.value"))}">${input}</td>
|
||||
<td class="arr-data-review-row-actions" data-label="${escapeHtml(I18N.t("common.operation"))}"><span class="arr-data-review-item-status">${escapeHtml(I18N.t(item.confirmed ? "data_review.confirmed" : "data_review.pending"))}</span><button type="button" data-arr-data-review-save ${editable ? "" : "disabled"}>${escapeHtml(I18N.t("data_review.save"))}</button>${item.can_be_empty ? `<button type="button" data-arr-data-review-empty ${editable ? "" : "disabled"}>${escapeHtml(I18N.t("data_review.confirm_empty"))}</button>` : ""}</td>
|
||||
</tr>`;
|
||||
|
||||
@@ -146,16 +146,20 @@
|
||||
"arr_download.retry_unavailable": ["该任务不能继续重试,请查看任务日志", "This task cannot be retried. Check the task log.", "ไม่สามารถลองงานนี้อีกครั้ง โปรดดูบันทึกงาน"],
|
||||
"arr_download.other_active": ["该日期已有进行中的下载任务", "A download for this date is already in progress", "มีงานดาวน์โหลดสำหรับวันที่นี้กำลังดำเนินการอยู่"],
|
||||
"data_review.title": ["人工完善报表字段", "Complete report fields", "ตรวจสอบและกรอกข้อมูลรายงาน"],
|
||||
"data_review.copy": ["请核实并逐项保存缺失或有冲突的字段。允许为空的字段也须明确确认,全部完成后才能生成日报。", "Verify and save each missing or conflicting field. Explicitly confirm any allowed empty values, then generate the daily report when all fields are complete.", "ตรวจสอบและบันทึกข้อมูลที่ขาดหายหรือขัดแย้งทีละรายการ ต้องยืนยันค่าที่เว้นว่างได้อย่างชัดเจน ก่อนสร้างรายงานเมื่อครบทุกช่อง"],
|
||||
"data_review.copy": ["请按各项提示核实来源与原处理规则。可空字段不必补成有值;核实无此信息后,可明确确认留空。全部确认后可生成日报。", "Check each field's source and original processing requirement. Optional fields need no value when none exists; explicitly confirm this after verification. Generate the daily report after all items are confirmed.", "ตรวจสอบแหล่งข้อมูลและข้อกำหนดเดิมของแต่ละช่อง ช่องที่เว้นว่างได้ไม่จำเป็นต้องกรอกค่า หากตรวจสอบแล้วว่าไม่มีข้อมูล ให้ยืนยันการเว้นว่าง สร้างรายงานรายวันได้เมื่อยืนยันครบทุกช่อง"],
|
||||
"data_review.progress": ["{date} · 已确认 {completed} / {total} 项", "{date} · {completed} of {total} confirmed", "{date} · ยืนยันแล้ว {completed} / {total} รายการ"],
|
||||
"data_review.reservation": ["预订信息", "Reservation", "ข้อมูลการจอง"],
|
||||
"data_review.field": ["待完善字段", "Field to review", "ข้อมูลที่ต้องตรวจสอบ"],
|
||||
"data_review.value": ["核实后的值", "Verified value", "ค่าที่ตรวจสอบแล้ว"],
|
||||
"data_review.source_row": ["原始记录 {sequence}", "Source row {sequence}", "รายการต้นฉบับ {sequence}"],
|
||||
"data_review.room": ["房号 {room}", "Room {room}", "ห้อง {room}"],
|
||||
"data_review.missing_value": ["来源字段缺失,请核实", "Source value missing. Verify it.", "ไม่มีข้อมูลต้นฉบับ โปรดตรวจสอบ"],
|
||||
"data_review.conflict_value": ["来源值有冲突,请核实", "Source values conflict. Verify them.", "ข้อมูลต้นฉบับขัดแย้งกัน โปรดตรวจสอบ"],
|
||||
"data_review.invalid_value": ["来源值格式无效,请核实", "Source value is invalid. Verify it.", "รูปแบบข้อมูลต้นฉบับไม่ถูกต้อง โปรดตรวจสอบ"],
|
||||
"data_review.required_field_rule": ["原处理规则必填,不可为空。", "Required by the original processing rules; cannot be blank.", "กฎการประมวลผลเดิมกำหนดให้กรอก ห้ามเว้นว่าง"],
|
||||
"data_review.optional_field_rule": ["原处理规则允许为空。", "The original processing rules allow this field to be blank.", "กฎการประมวลผลเดิมอนุญาตให้เว้นช่องนี้ว่างได้"],
|
||||
"data_review.required_value_missing": ["本次未取得该值,请填写核实后的值。", "This value was not obtained. Enter the verified value.", "ครั้งนี้ไม่ได้รับค่านี้ โปรดกรอกค่าที่ตรวจสอบแล้ว"],
|
||||
"data_review.source_unconfirmed": ["来源尚未确认。请核实有无此信息;没有则点击“确认无此信息”。", "The source is unconfirmed. Check whether this information exists; if it does not, choose “Confirm no value”.", "ยังยืนยันข้อมูลจากแหล่งต้นทางไม่ได้ โปรดตรวจสอบว่ามีข้อมูลนี้หรือไม่ หากไม่มี ให้เลือก “ยืนยันว่าไม่มีข้อมูล”"],
|
||||
"data_review.fetch_failed": ["接口读取未完成,请先核实来源,不能直接认定没有此信息。", "The interface read did not complete. Verify the source before concluding that no information exists.", "การอ่านข้อมูลผ่านอินเทอร์เฟซยังไม่เสร็จ โปรดตรวจสอบแหล่งข้อมูลก่อนสรุปว่าไม่มีข้อมูลนี้"],
|
||||
"data_review.conflict_value": ["来源值有冲突,请核实后填写正确的值。", "Source values conflict. Verify and enter the correct value.", "ข้อมูลต้นฉบับขัดแย้งกัน โปรดตรวจสอบและกรอกค่าที่ถูกต้อง"],
|
||||
"data_review.invalid_value": ["来源值格式无效,请核实后填写有效的值。", "The source value is invalid. Verify and enter a valid value.", "รูปแบบข้อมูลต้นฉบับไม่ถูกต้อง โปรดตรวจสอบและกรอกค่าที่ใช้ได้"],
|
||||
"data_review.confirmed_value": ["已使用人工核实值", "Manually verified value saved", "บันทึกค่าที่ตรวจสอบด้วยตนเองแล้ว"],
|
||||
"data_review.confirmed": ["已确认", "Confirmed", "ยืนยันแล้ว"],
|
||||
"data_review.pending": ["待确认", "Awaiting confirmation", "รอยืนยัน"],
|
||||
|
||||
@@ -27,6 +27,7 @@ from .data_client import DataReader, document, json_bytes, typed_id
|
||||
|
||||
|
||||
VERSION = "arr-ohip-data/v1"
|
||||
OPTIONAL_OMISSION_POLICY = "oracle-optional-association/v1"
|
||||
FIELDS = ("BLOCK_CODE", "ADULTS", "CHILDREN", "COMPANY_NAME", "CONFIRMATION_NO", "DISP_ROOM_NO",
|
||||
"EFFECTIVE_RATE_AMOUNT", "FULL_NAME", "RES_COMMENT", "NO_OF_ROOMS", "PRODUCTS", "RATE_CODE",
|
||||
"ROOM_CATEGORY_LABEL", "ARRIVAL", "DEPARTURE")
|
||||
@@ -167,6 +168,14 @@ def _block(search, detail, reader):
|
||||
except base.CollectionError:
|
||||
pass
|
||||
day = _day(detail, reader.options.arrival_date)
|
||||
# Successful Reservation/DailySummary reads omit this optional association
|
||||
# for ordinary unblocked reservations. Require agreement across the search
|
||||
# and arrival-day detail, rather than treating any missing code as empty.
|
||||
searched_stay = search.get("roomStay")
|
||||
if ("reservationBlock" not in day and type(searched_stay) is dict
|
||||
and "reservationBlock" not in searched_stay
|
||||
and "reservationBlock" not in detail["roomStay"]):
|
||||
return ""
|
||||
block = day.get("reservationBlock")
|
||||
require(type(block) is dict, "missing_reservation_block")
|
||||
require("hotelId" not in block or block["hotelId"] == reader.hotel_id, "block_association_hotel_mismatch")
|
||||
@@ -174,6 +183,16 @@ def _block(search, detail, reader):
|
||||
if identifiers == []:
|
||||
search_block = search.get("roomStay", {}).get("reservationBlock")
|
||||
require(type(search_block) is dict and search_block.get("blockIdList") == [], "block_association_changed")
|
||||
associations = [block, search_block]
|
||||
if "reservationBlock" in detail["roomStay"]:
|
||||
associations.append(detail["roomStay"]["reservationBlock"])
|
||||
for association in associations:
|
||||
require(type(association) is dict and association.get("blockIdList") == [],
|
||||
"block_association_changed")
|
||||
require("hotelId" not in association or association["hotelId"] == reader.hotel_id,
|
||||
"block_association_hotel_mismatch")
|
||||
require("blockName" not in association or (type(association["blockName"]) is str
|
||||
and not association["blockName"].strip()), "block_association_changed")
|
||||
return ""
|
||||
identity = typed_id(identifiers, "Block")
|
||||
# An explicit code disagreement must remain visible, not be overwritten by
|
||||
@@ -197,9 +216,28 @@ def _block(search, detail, reader):
|
||||
return code
|
||||
|
||||
|
||||
def _packages(detail, reader, related):
|
||||
def _packages(search, detail, reader, related):
|
||||
# Oracle maps retrieval of booked packages to getReservation with Packages
|
||||
# fetch instructions. Its optional array may be omitted for zero packages.
|
||||
# An explicit, valid search-indicator list must corroborate that reading;
|
||||
# missing indicators, positive/unknown counts and malformed structures are
|
||||
# still unresolved. A failed detail read never reaches this selector.
|
||||
absent = "reservationPackages" not in detail
|
||||
if absent or detail["reservationPackages"] == []:
|
||||
indicators = search.get("reservationIndicators")
|
||||
if absent or indicators is not None:
|
||||
require(type(indicators) is list and all(type(item) is dict for item in indicators),
|
||||
"missing_reservation_packages")
|
||||
require(all(type(item.get("indicatorName")) is str and bool(item["indicatorName"].strip())
|
||||
for item in indicators), "invalid_package_indicators")
|
||||
package_counts = [item.get("count") for item in indicators if item.get("indicatorName") == "PACKAGEITEM"]
|
||||
require(len(package_counts) <= 1, "ambiguous_package_indicator")
|
||||
require(not package_counts or base.integer(package_counts[0], "invalid_package_indicator_count") == 0,
|
||||
"package_indicator_detail_mismatch")
|
||||
if absent:
|
||||
return []
|
||||
packages = detail.get("reservationPackages")
|
||||
require(type(packages) is list and all(type(p) is dict for p in packages), "missing_reservation_packages")
|
||||
require(type(packages) is list and all(type(p) is dict for p in packages), "invalid_reservation_packages")
|
||||
result = []
|
||||
related["reservation_packages"] = result
|
||||
for item in packages:
|
||||
@@ -330,7 +368,7 @@ def _record(search, detail, reader, sequence):
|
||||
"FULL_NAME": observe(lambda: _name(detail, reader)),
|
||||
"RES_COMMENT": observe(lambda: list(fields.reservation_gen_notes(detail))),
|
||||
"NO_OF_ROOMS": observe(lambda: fields.arrival_number_of_units(detail, day)),
|
||||
"PRODUCTS": observe(lambda: _packages(detail, reader, related)),
|
||||
"PRODUCTS": observe(lambda: _packages(search, detail, reader, related)),
|
||||
"RATE_CODE": observe(lambda: fields.arrival_rate_code(detail, day)),
|
||||
"ROOM_CATEGORY_LABEL": observe(lambda: fields.agreed_room_type(search, detail, day)),
|
||||
"ARRIVAL": observe(lambda: dates().arrival.isoformat()),
|
||||
@@ -408,6 +446,7 @@ def collect(reader):
|
||||
payload = {"version": VERSION, "hotel_id": options.hotel_id, "report_date": options.arrival_date,
|
||||
"source_kind": reader.source_kind,
|
||||
"status": status, "collection_complete": complete and not failed, "input_complete": ready and not failed,
|
||||
"optional_omission_policy": OPTIONAL_OMISSION_POLICY,
|
||||
"field_names": list(FIELDS), "trace_included": False, "atomic_snapshot": False, "records": rows}
|
||||
raw = json_bytes(payload)
|
||||
require(len(raw) <= MAX_DATA_BYTES, "data_output_too_large")
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const vm = require('node:vm');
|
||||
const {harness,response} = require('./helpers/arr_ui_harness.cjs');
|
||||
const requestId='a'.repeat(32);
|
||||
const task={request_id:requestId,report_date:'2026-10-07',status:'needs_data_review',job_id:null,can_retry:false};
|
||||
@@ -7,6 +10,65 @@ const item=(extra={})=>({item_id:'1:BLOCK_CODE',source_sequence:1,confirmation_n
|
||||
const review=(items,revision=1)=>({request_id:requestId,report_date:task.report_date,status:'editing',revision,items,pending_count:items.filter(x=>!x.confirmed).length,total_count:items.length,can_finalize:items.every(x=>x.confirmed)});
|
||||
const plain=value=>JSON.parse(JSON.stringify(value));
|
||||
|
||||
for (const fixture of [
|
||||
{name:'missing room retains the original required rule', field:'DISP_ROOM_NO', can_be_empty:false,
|
||||
source_state:'missing', reason_code:'calendar_missing_rooms', expected:'required_value_missing'},
|
||||
{name:'missing block is optional information awaiting confirmation', field:'BLOCK_CODE', can_be_empty:true,
|
||||
source_state:'missing', reason_code:'missing_reservation_block', expected:'source_unconfirmed'},
|
||||
{name:'missing packages are optional information awaiting confirmation', field:'PRODUCTS', can_be_empty:true,
|
||||
source_state:'missing', reason_code:'missing_reservation_packages', expected:'source_unconfirmed'},
|
||||
{name:'failed source state takes precedence over a missing reason', source_state:'failed',
|
||||
reason_code:'missing_reservation_block', expected:'fetch_failed'},
|
||||
{name:'ambiguous source state takes precedence over an invalid reason', source_state:'ambiguous',
|
||||
reason_code:'invalid_package_code', expected:'conflict_value'},
|
||||
{name:'older HTTP failure snapshots remain failures', reason_code:'http_failure', expected:'fetch_failed'},
|
||||
{name:'invalid response JSON is a read failure rather than a field format error',
|
||||
reason_code:'invalid_json', expected:'fetch_failed'},
|
||||
{name:'a block disagreement is shown as conflicting information',
|
||||
reason_code:'block_code_disagreement', expected:'conflict_value'},
|
||||
{name:'a changed block association is shown as conflicting information',
|
||||
reason_code:'block_association_changed', expected:'conflict_value'},
|
||||
{name:'malformed package code has a separate format explanation',
|
||||
reason_code:'invalid_package_code', expected:'invalid_value'},
|
||||
{name:'invalid nights retain a separate format explanation',
|
||||
field:'DEPARTURE', can_be_empty:false, reason_code:'XML_NEGATIVE_NIGHTS', expected:'invalid_value'},
|
||||
{name:'confirmed decisions show the saved confirmation instead of the original problem',
|
||||
source_state:'failed', reason_code:'http_failure', confirmed:true, value:'VERIFIED', expected:'confirmed_value'},
|
||||
]) {
|
||||
test(`field review explanation: ${fixture.name}`,async()=>{
|
||||
const source=item(fixture);
|
||||
const original=plain(source);
|
||||
const h=harness(()=>response(200,review([source])));
|
||||
await h.acceptARRDownloadTask(task,{sync:false});
|
||||
const markup=h.element('#arr-data-review-body').innerHTML;
|
||||
assert.match(markup,new RegExp(`data_review\\.${fixture.expected}\\b`));
|
||||
assert.match(markup,new RegExp(`data_review\\.${source.can_be_empty?'optional':'required'}_field_rule\\b`));
|
||||
assert.equal(markup.includes('data_review.missing_value'),false);
|
||||
assert.equal(markup.includes('data-arr-data-review-empty'),source.can_be_empty);
|
||||
assert.deepEqual(plain(source),original,'rendering explanations never changes a source or confirmation');
|
||||
assert.equal(h.calls.every(call=>call.method==='GET'),true);
|
||||
});
|
||||
}
|
||||
|
||||
test('review explanations have distinct Chinese, English and Thai copy for required, optional and failed reads',()=>{
|
||||
const context=vm.createContext({window:{},document:{readyState:'loading',addEventListener(){}}});
|
||||
vm.runInContext(fs.readFileSync(path.resolve(__dirname,'../../arr_web/static/i18n.js'),'utf8'),context);
|
||||
const catalog=context.window.ARRI18n.CATALOG;
|
||||
const keys=['required_field_rule','optional_field_rule','required_value_missing','source_unconfirmed',
|
||||
'fetch_failed','conflict_value','invalid_value'];
|
||||
for(const key of keys) {
|
||||
const translations=catalog[`data_review.${key}`];
|
||||
assert.equal(translations.length,3);
|
||||
translations.forEach(text=>assert.equal(typeof text==='string' && text.length>0,true));
|
||||
}
|
||||
assert.match(catalog['data_review.required_field_rule'][0],/原处理规则必填/);
|
||||
assert.match(catalog['data_review.optional_field_rule'][0],/允许为空/);
|
||||
assert.match(catalog['data_review.source_unconfirmed'][0],/确认无此信息/);
|
||||
assert.match(catalog['data_review.fetch_failed'][0],/不能直接认定没有此信息/);
|
||||
assert.equal(catalog['data_review.fetch_failed'][0].includes('刷新'),false,
|
||||
'refreshing the review must not imply a retry of Oracle acquisition');
|
||||
});
|
||||
|
||||
test('source review opens before a job exists and another date can start its own download',async()=>{
|
||||
const h=harness((url,options)=>options.method==='POST'
|
||||
? response(202,{...JSON.parse(options.body),status:'queued',job_id:null,can_retry:false})
|
||||
|
||||
@@ -44,6 +44,8 @@ from tests import test_arr_download_postgres_integration as pg_helpers
|
||||
PROJECT = Path(__file__).resolve().parents[1]
|
||||
ACTOR = "operator@example.test"
|
||||
SOURCE_MANIFEST = "c" * 64
|
||||
REANALYSIS_MANIFEST = "e" * 64
|
||||
REANALYSIS_POLICY = "oracle-optional-association/v1"
|
||||
CONTEXT = {"hotel_id": HOTEL, "source_kind": "test_transport"}
|
||||
|
||||
|
||||
@@ -77,6 +79,23 @@ def gap(document, field, sequence=1, state="missing"):
|
||||
document["status"], document["input_complete"] = "collected_with_gaps", False
|
||||
|
||||
|
||||
def optional_omission_source(document):
|
||||
result = copy.deepcopy(document)
|
||||
for row in result["records"]:
|
||||
for field, reason, empty_value in (("BLOCK_CODE", "missing_reservation_block", ""),
|
||||
("PRODUCTS", "missing_reservation_packages", [])):
|
||||
observation = row["fields"][field]
|
||||
if observation.get("state") == "missing" and observation.get("reason") == reason:
|
||||
row["fields"][field] = {"state": "empty", "value": empty_value}
|
||||
unresolved = any(value["state"] != "available"
|
||||
and not (field in review_module.OPTIONAL and value["state"] == "empty")
|
||||
for row in result["records"] for field, value in row["fields"].items())
|
||||
result["input_complete"] = not unresolved
|
||||
result["status"] = "collected_with_gaps" if unresolved else "collected"
|
||||
result["optional_omission_policy"] = REANALYSIS_POLICY
|
||||
return result
|
||||
|
||||
|
||||
class DataFieldReviewTests(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
@@ -103,6 +122,247 @@ class DataFieldReviewTests(unittest.TestCase):
|
||||
self.assertEqual(raised.exception.code, "ARR_DATA_REVIEW_" + code)
|
||||
self.assertEqual(raised.exception.status, status)
|
||||
|
||||
def omission_gaps(self, document=None, sequence=1):
|
||||
document = self.document if document is None else document
|
||||
for field, reason in (("BLOCK_CODE", "missing_reservation_block"),
|
||||
("PRODUCTS", "missing_reservation_packages")):
|
||||
gap(document, field, sequence=sequence)
|
||||
document["records"][sequence - 1]["fields"][field]["reason"] = reason
|
||||
|
||||
def reanalyze(self, review, document=None, **kwargs):
|
||||
return self.service.apply_source_reanalysis(REQUEST,
|
||||
raw(optional_omission_source(self.document) if document is None else document),
|
||||
REANALYSIS_MANIFEST, REANALYSIS_POLICY, expected_revision=review["revision"], **kwargs)
|
||||
|
||||
def test_source_reanalysis_removes_only_25_optional_gaps_without_confirming_a_room(self):
|
||||
template = self.document["records"][0]
|
||||
self.document["records"] = []
|
||||
for sequence in range(1, 19):
|
||||
row = copy.deepcopy(template)
|
||||
row["source_sequence"], row["reservation_id"] = sequence, f"synthetic-{sequence}"
|
||||
row["fields"]["CONFIRMATION_NO"]["value"] = f"SYNTHETIC-{sequence:03d}"
|
||||
row["fields"]["DISP_ROOM_NO"]["value"] = f"ROOM-{sequence:03d}"
|
||||
self.document["records"].append(row)
|
||||
gap(self.document, "BLOCK_CODE", sequence)
|
||||
row["fields"]["BLOCK_CODE"]["reason"] = "missing_reservation_block"
|
||||
if sequence <= 7:
|
||||
gap(self.document, "PRODUCTS", sequence)
|
||||
row["fields"]["PRODUCTS"]["reason"] = "missing_reservation_packages"
|
||||
gap(self.document, "DISP_ROOM_NO")
|
||||
original = raw(self.document)
|
||||
review = self.prepare()
|
||||
self.assertEqual(review["pending_count"], 26)
|
||||
revised = self.reanalyze(review)
|
||||
self.assertEqual((revised["revision"], revised["pending_count"], revised["total_count"]), (1, 1, 1))
|
||||
self.assertEqual([item["field"] for item in revised["items"]], ["DISP_ROOM_NO"])
|
||||
self.assertFalse(revised["can_finalize"])
|
||||
self.assertIsNone(self.service.payload(REQUEST))
|
||||
self.assertEqual(self.service.original(REQUEST), (original, SOURCE_MANIFEST))
|
||||
directory = self.root / REQUEST
|
||||
_, _, state = self.service._read(directory)
|
||||
self.assertEqual(state["decisions"], {})
|
||||
self.assertEqual([event["action"] for event in state["events"]], ["source_reanalysis"])
|
||||
self.assertNotIn("actor", state["events"][0])
|
||||
receipt = json.loads((directory / state["source_reanalysis"]["receipt_file"]).read_bytes())
|
||||
self.assertEqual(len(receipt["changes"]), 25)
|
||||
self.assertEqual(receipt["parent_original_sha256"], hashlib.sha256(original).hexdigest())
|
||||
for path in directory.iterdir():
|
||||
self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600)
|
||||
|
||||
def test_reanalysis_preserves_staff_decisions_and_final_manifest_has_both_source_bindings(self):
|
||||
self.omission_gaps()
|
||||
gap(self.document, "DISP_ROOM_NO")
|
||||
before = raw(self.document)
|
||||
first = self.prepare()
|
||||
staff = self.update(first, "BLOCK_CODE", "STAFF-VERIFIED-BLOCK")
|
||||
payload = raw(optional_omission_source(self.document))
|
||||
revised = self.reanalyze(staff)
|
||||
confirmed = next(item for item in revised["items"] if item["field"] == "BLOCK_CODE")
|
||||
self.assertEqual((confirmed["value"], confirmed["source_state"], confirmed["confirmed"]),
|
||||
("STAFF-VERIFIED-BLOCK", "available", True))
|
||||
self.assertEqual(revised["pending_count"], 1)
|
||||
restored = DataFieldReviews(root=self.root, policy=self.policy, context=CONTEXT)
|
||||
self.assertEqual(restored.get(REQUEST), revised)
|
||||
revised = restored.update(REQUEST, "1:DISP_ROOM_NO", revised["revision"], "STAFF-ROOM", ACTOR)
|
||||
# A lost maintenance response can be retried even after a later staff save.
|
||||
self.assertEqual(restored.apply_source_reanalysis(REQUEST, payload, REANALYSIS_MANIFEST,
|
||||
REANALYSIS_POLICY, expected_revision=staff["revision"]), revised)
|
||||
frozen = restored.finalize(REQUEST, revised["revision"], ACTOR)
|
||||
result, binding = restored.payload(REQUEST)
|
||||
self.assertEqual(restored.original(REQUEST), (before, SOURCE_MANIFEST))
|
||||
derived = json.loads(result)
|
||||
fields = derived["records"][0]["fields"]
|
||||
self.assertEqual(fields["BLOCK_CODE"]["value"], "STAFF-VERIFIED-BLOCK")
|
||||
self.assertEqual(fields["PRODUCTS"], {"state": "empty", "value": []})
|
||||
audit = derived["manual_data_review"]["manifest"]
|
||||
self.assertEqual(audit["original_sha256"], hashlib.sha256(before).hexdigest())
|
||||
self.assertEqual(audit["source_manifest_sha256"], SOURCE_MANIFEST)
|
||||
self.assertEqual(audit["source_reanalysis"]["policy_id"], REANALYSIS_POLICY)
|
||||
self.assertEqual(audit["source_reanalysis"]["reanalyzed_sha256"], hashlib.sha256(payload).hexdigest())
|
||||
self.assertEqual(audit["source_reanalysis"]["reanalysis_manifest_sha256"], REANALYSIS_MANIFEST)
|
||||
self.assertEqual([event["action"] for event in audit["events"]],
|
||||
["set_field", "source_reanalysis", "set_field", "finalize"])
|
||||
self.assertEqual({change["actor"] for change in audit["changes"]}, {ACTOR})
|
||||
self.assertRegex(binding, r"^[0-9a-f]{64}$")
|
||||
self.assertEqual(restored.finalize(REQUEST, revised["revision"], ACTOR), frozen)
|
||||
self.assert_error("FROZEN", lambda: restored.apply_source_reanalysis(REQUEST, payload,
|
||||
REANALYSIS_MANIFEST, REANALYSIS_POLICY, expected_revision=frozen["revision"]), 409)
|
||||
|
||||
def test_reanalysis_rejects_other_source_changes_and_leaves_revision_unchanged(self):
|
||||
self.document = source_document(2)
|
||||
self.omission_gaps()
|
||||
review = self.prepare()
|
||||
valid = optional_omission_source(self.document)
|
||||
mutations = [
|
||||
lambda data: data.update(hotel_id="OTHER"),
|
||||
lambda data: data.update(report_date="2026-09-16"),
|
||||
lambda data: data.update(source_kind="OTHER"),
|
||||
lambda data: data.update(collection_complete=False),
|
||||
lambda data: data.update(input_complete=False),
|
||||
lambda data: data.update(status="collected_with_gaps"),
|
||||
lambda data: data.update(optional_omission_policy="OTHER"),
|
||||
lambda data: data.update(extra="not allowed"),
|
||||
lambda data: data["records"].reverse(),
|
||||
lambda data: data["records"][0].update(reservation_id="OTHER"),
|
||||
lambda data: data["records"][0]["related"].update(extra="not allowed"),
|
||||
lambda data: data["records"][0]["fields"]["FULL_NAME"].update(value="OTHER"),
|
||||
lambda data: data["records"][0]["fields"]["BLOCK_CODE"].update(reason="not allowed"),
|
||||
]
|
||||
for change in mutations:
|
||||
with self.subTest(change=change):
|
||||
document = copy.deepcopy(valid)
|
||||
change(document)
|
||||
with self.assertRaises(CollectionError):
|
||||
self.reanalyze(review, document)
|
||||
self.assertEqual(self.service.get(REQUEST), review)
|
||||
self.assertEqual(list((self.root / REQUEST).glob("*reanalysis*")), [])
|
||||
|
||||
def test_reanalysis_never_clears_failed_ambiguous_available_or_unrecognized_observations(self):
|
||||
for index, (state, reason) in enumerate((("failed", "missing_reservation_packages"),
|
||||
("ambiguous", "missing_reservation_packages"), ("available", "missing_reservation_packages"),
|
||||
("missing", "synthetic_field_gap")), 1):
|
||||
with self.subTest(state=state, reason=reason):
|
||||
document = source_document()
|
||||
self.omission_gaps(document)
|
||||
observation = document["records"][0]["fields"]["PRODUCTS"]
|
||||
observation["state"], observation["reason"] = state, reason
|
||||
if state == "available":
|
||||
observation["value"] = [{"package": {"packageCode": "EXISTING"}}]
|
||||
request_id = f"{index:032x}"
|
||||
review = self.prepare(document, request_id)
|
||||
proposed = optional_omission_source(document)
|
||||
proposed["records"][0]["fields"]["PRODUCTS"] = {"state": "empty", "value": []}
|
||||
proposed["input_complete"], proposed["status"] = True, "collected"
|
||||
with self.assertRaisesRegex(CollectionError, "data_review_reanalysis_observation_invalid"):
|
||||
self.service.apply_source_reanalysis(request_id, raw(proposed), REANALYSIS_MANIFEST,
|
||||
REANALYSIS_POLICY, expected_revision=review["revision"])
|
||||
self.assertEqual(self.service.get(request_id), review)
|
||||
|
||||
def test_reanalysis_is_revision_guarded_and_rejects_other_policy_or_evidence(self):
|
||||
self.omission_gaps()
|
||||
review = self.prepare()
|
||||
payload = raw(optional_omission_source(self.document))
|
||||
staff = self.update(review, "BLOCK_CODE", "STAFF")
|
||||
self.assert_error("CONFLICT", lambda: self.reanalyze(review), 409)
|
||||
with self.assertRaisesRegex(CollectionError, "data_review_reanalysis_policy_invalid"):
|
||||
self.service.apply_source_reanalysis(REQUEST, payload, REANALYSIS_MANIFEST, "OTHER",
|
||||
expected_revision=staff["revision"])
|
||||
with self.assertRaisesRegex(CollectionError, "data_review_reanalysis_manifest_invalid"):
|
||||
self.service.apply_source_reanalysis(REQUEST, payload, "invalid", REANALYSIS_POLICY,
|
||||
expected_revision=staff["revision"])
|
||||
current = self.reanalyze(staff)
|
||||
self.assertEqual(self.reanalyze(staff), current)
|
||||
self.assert_error("CONFLICT", lambda: self.service.apply_source_reanalysis(REQUEST, payload,
|
||||
"f" * 64, REANALYSIS_POLICY, expected_revision=current["revision"]), 409)
|
||||
self.assertEqual(self.service.get(REQUEST), current)
|
||||
|
||||
def test_reanalysis_publication_recovers_after_crash_without_publishing_partial_source(self):
|
||||
self.omission_gaps()
|
||||
review = self.prepare()
|
||||
real_atomic = review_module.atomic_json
|
||||
def interrupted(path, value, **kwargs):
|
||||
if path.name == "state.json":
|
||||
raise OSError("synthetic reanalysis pointer interruption")
|
||||
return real_atomic(path, value, **kwargs)
|
||||
with patch.object(review_module, "atomic_json", side_effect=interrupted):
|
||||
with self.assertRaisesRegex(OSError, "synthetic reanalysis pointer interruption"):
|
||||
self.reanalyze(review)
|
||||
self.assertEqual(self.service.get(REQUEST), review)
|
||||
staff = self.update(review, "BLOCK_CODE", "PRESERVED-AFTER-INTERRUPTION")
|
||||
restored = DataFieldReviews(root=self.root, policy=self.policy, context=CONTEXT)
|
||||
result = restored.apply_source_reanalysis(REQUEST, raw(optional_omission_source(self.document)),
|
||||
REANALYSIS_MANIFEST, REANALYSIS_POLICY, expected_revision=staff["revision"])
|
||||
self.assertEqual(result["revision"], staff["revision"] + 1)
|
||||
self.assertEqual(result["pending_count"], 0)
|
||||
self.assertEqual(result["items"][0]["value"], "PRESERVED-AFTER-INTERRUPTION")
|
||||
_, _, state = restored._read(self.root / REQUEST)
|
||||
self.assertEqual([event["action"] for event in state["events"]], ["set_field", "source_reanalysis"])
|
||||
|
||||
def test_reanalysis_with_no_pending_items_still_requires_finalize_and_recovers_its_intent(self):
|
||||
self.omission_gaps()
|
||||
original = raw(self.document)
|
||||
revised = self.reanalyze(self.prepare())
|
||||
self.assertTrue(revised["can_finalize"])
|
||||
self.assertEqual(revised["items"], [])
|
||||
self.assertEqual(self.prepare(), revised, "executor cannot bypass an unfinalized reanalysis")
|
||||
self.assertIsNone(self.service.payload(REQUEST))
|
||||
real_atomic = review_module.atomic_json
|
||||
def interrupted(path, value, **kwargs):
|
||||
if path.name == "state.json":
|
||||
raise OSError("synthetic finalize pointer interruption")
|
||||
return real_atomic(path, value, **kwargs)
|
||||
with patch.object(review_module, "atomic_json", side_effect=interrupted):
|
||||
with self.assertRaisesRegex(OSError, "synthetic finalize pointer interruption"):
|
||||
self.service.finalize(REQUEST, revised["revision"], ACTOR)
|
||||
frozen_bytes = (self.root / REQUEST / "reviewed-source.json").read_bytes()
|
||||
self.assertEqual(self.service.get(REQUEST), revised)
|
||||
self.assertIsNone(self.service.payload(REQUEST))
|
||||
self.assert_error("FROZEN", lambda: self.reanalyze(revised), 409)
|
||||
restored = DataFieldReviews(root=self.root, policy=self.policy, context=CONTEXT)
|
||||
final = restored.finalize(REQUEST, revised["revision"], "different-operator@example.test")
|
||||
self.assertEqual(final["status"], "finalized")
|
||||
self.assertEqual(restored.payload(REQUEST)[0], frozen_bytes)
|
||||
self.assertEqual(restored.original(REQUEST), (original, SOURCE_MANIFEST))
|
||||
audit = json.loads(frozen_bytes)["manual_data_review"]["manifest"]
|
||||
self.assertEqual(audit["changes"], [])
|
||||
self.assertEqual([event["action"] for event in audit["events"]], ["source_reanalysis", "finalize"])
|
||||
self.assertEqual(audit["events"][-1]["actor"], ACTOR)
|
||||
|
||||
def test_reanalysis_and_staff_save_share_one_revision_winner(self):
|
||||
self.omission_gaps()
|
||||
gap(self.document, "DISP_ROOM_NO")
|
||||
review = self.prepare()
|
||||
start = threading.Barrier(2)
|
||||
def mutate(reanalysis):
|
||||
start.wait(timeout=3)
|
||||
try:
|
||||
return self.reanalyze(review) if reanalysis else self.update(review, "DISP_ROOM_NO", "STAFF-ROOM")
|
||||
except PortalError as error:
|
||||
return error
|
||||
with ThreadPoolExecutor(max_workers=2) as pool:
|
||||
results = list(pool.map(mutate, [True, False]))
|
||||
winners = [result for result in results if isinstance(result, dict)]
|
||||
conflicts = [result for result in results if isinstance(result, PortalError)]
|
||||
self.assertEqual((len(winners), len(conflicts)), (1, 1))
|
||||
self.assertEqual((conflicts[0].code, conflicts[0].status), ("ARR_DATA_REVIEW_CONFLICT", 409))
|
||||
self.assertEqual(self.service.get(REQUEST), winners[0])
|
||||
|
||||
def test_reanalysis_payload_and_receipt_tampering_are_rejected(self):
|
||||
self.omission_gaps()
|
||||
revised = self.reanalyze(self.prepare())
|
||||
directory = self.root / REQUEST
|
||||
_, _, state = self.service._read(directory)
|
||||
reference = state["source_reanalysis"]
|
||||
for name in (reference["file"], reference["receipt_file"]):
|
||||
with self.subTest(name=name):
|
||||
path = directory / name
|
||||
before = path.read_bytes()
|
||||
path.write_bytes(before + b" ")
|
||||
with self.assertRaisesRegex(CollectionError, "data_review_reanalysis_changed"):
|
||||
self.service.get(REQUEST)
|
||||
path.write_bytes(before)
|
||||
self.assertEqual(self.service.get(REQUEST), revised)
|
||||
|
||||
def test_complete_source_and_gaps_only_on_excluded_rows_need_no_review(self):
|
||||
self.assertIsNone(self.prepare())
|
||||
excluded = source_document()
|
||||
@@ -134,6 +394,27 @@ class DataFieldReviewTests(unittest.TestCase):
|
||||
self.assertEqual({item["field"] for item in review["items"]}, set(invalid))
|
||||
self.assertEqual(review["pending_count"], len(invalid))
|
||||
|
||||
def test_review_exposes_source_failure_separately_from_optional_absence(self):
|
||||
gap(self.document, "DISP_ROOM_NO")
|
||||
gap(self.document, "BLOCK_CODE")
|
||||
gap(self.document, "PRODUCTS", state="failed")
|
||||
gap(self.document, "RES_COMMENT", state="ambiguous")
|
||||
before = raw(self.document)
|
||||
review = self.prepare()
|
||||
items = {item["field"]: item for item in review["items"]}
|
||||
self.assertEqual({field: (item["source_state"], item["can_be_empty"])
|
||||
for field, item in items.items()}, {
|
||||
"DISP_ROOM_NO": ("missing", False), "BLOCK_CODE": ("missing", True),
|
||||
"PRODUCTS": ("failed", True), "RES_COMMENT": ("ambiguous", True),
|
||||
})
|
||||
self.assertEqual(review["pending_count"], 4)
|
||||
self.assertFalse(review["can_finalize"])
|
||||
self.assertEqual(self.service.original(REQUEST)[0], before)
|
||||
confirmed = self.update(review, "BLOCK_CODE", "")
|
||||
item = next(item for item in confirmed["items"] if item["field"] == "BLOCK_CODE")
|
||||
self.assertEqual((item["source_state"], item["confirmed"]), ("empty", True))
|
||||
self.assertEqual(confirmed["pending_count"], 3)
|
||||
|
||||
def test_departure_before_arrival_is_an_editable_issue(self):
|
||||
self.document["records"][0]["fields"]["DEPARTURE"]["value"] = "2026-09-14"
|
||||
review = self.prepare()
|
||||
|
||||
+118
-4
@@ -402,12 +402,13 @@ class ARRDataTests(unittest.TestCase):
|
||||
self.assertEqual(result["status"], "collected_with_gaps")
|
||||
self.assertEqual(field["state"], "missing")
|
||||
|
||||
def test_native_optional_absence_requires_review_even_with_zero_package_price(self):
|
||||
def test_successful_optional_omission_is_empty_only_with_association_corroboration(self):
|
||||
row = self.service.rows[0]
|
||||
row.pop("reservationProfiles")
|
||||
row.pop("reservationPackages")
|
||||
row["roomStay"].pop("reservationBlock")
|
||||
row["roomStay"]["roomRates"][0].pop("reservationBlock")
|
||||
row["reservationIndicators"] = []
|
||||
def edit(op, doc, _):
|
||||
if op == "searchRateInfo": doc["data"]["detail"]["totalPackageAmount"] = 0
|
||||
return doc
|
||||
@@ -415,9 +416,122 @@ class ARRDataTests(unittest.TestCase):
|
||||
result, payload = self.fetch()
|
||||
self.assertEqual(result["status"], "collected_with_gaps")
|
||||
self.assertTrue(result["collection_complete"])
|
||||
for name in ("BLOCK_CODE", "COMPANY_NAME", "PRODUCTS"):
|
||||
self.assertEqual(payload["records"][0]["fields"][name]["state"], "missing")
|
||||
self.assertIsNone(payload["records"][0]["fields"][name]["value"])
|
||||
observations = payload["records"][0]["fields"]
|
||||
self.assertEqual(observations["BLOCK_CODE"], {"state": "empty", "value": ""})
|
||||
self.assertEqual(observations["PRODUCTS"], {"state": "empty", "value": []})
|
||||
# Omission of a different, unestablished structure is not accepted by
|
||||
# this narrowly scoped association rule; zero package price is no proof.
|
||||
self.assertEqual(observations["COMPANY_NAME"]["state"], "missing")
|
||||
self.assertIsNone(observations["COMPANY_NAME"]["value"])
|
||||
self.assertEqual(payload["optional_omission_policy"], data.OPTIONAL_OMISSION_POLICY)
|
||||
|
||||
def test_omitted_packages_without_indicator_list_remain_unresolved(self):
|
||||
self.service.rows[0].pop("reservationPackages")
|
||||
result, field = self.field("PRODUCTS")
|
||||
self.assertEqual((result["status"], field["state"], field["reason"]),
|
||||
("collected_with_gaps", "missing", "missing_reservation_packages"))
|
||||
|
||||
def test_omitted_packages_with_explicit_zero_indicator_are_empty(self):
|
||||
row = self.service.rows[0]
|
||||
row.pop("reservationPackages")
|
||||
row["reservationIndicators"] = [{"indicatorName": "PACKAGEITEM", "count": 0}]
|
||||
result, field = self.field("PRODUCTS")
|
||||
self.assertEqual((result["status"], field), ("collected", {"state": "empty", "value": []}))
|
||||
|
||||
def test_invalid_indicator_names_do_not_prove_absent_packages(self):
|
||||
for index, indicator in enumerate(({}, {"indicatorName": None}, {"indicatorName": 1},
|
||||
{"indicatorName": " "})):
|
||||
self.root = Path(self.temp.name) / f"package-invalid-indicator-{index}"
|
||||
self.service = SimulatedOHIP()
|
||||
row = self.service.rows[0]
|
||||
row.pop("reservationPackages")
|
||||
row["reservationIndicators"] = [indicator]
|
||||
with self.subTest(indicator=indicator):
|
||||
_, field = self.field("PRODUCTS")
|
||||
self.assertEqual((field["state"], field["reason"]), ("missing", "invalid_package_indicators"))
|
||||
|
||||
def test_empty_block_ids_do_not_swallow_other_association_evidence(self):
|
||||
cases = (("search", {"blockIdList": [], "hotelId": "OTHER"}),
|
||||
("search", {"blockIdList": [], "blockName": "ASSOCIATED"}),
|
||||
("day", {"blockIdList": [], "blockName": None}),
|
||||
("detail", {"blockIdList": [{"type": "Block", "id": "b1"}]}),
|
||||
("detail", {}), ("detail", None))
|
||||
for index, (location, block) in enumerate(cases):
|
||||
self.root = Path(self.temp.name) / f"block-empty-contradiction-{index}"
|
||||
self.service = SimulatedOHIP()
|
||||
def edit(op, doc, _, location=location, block=block):
|
||||
if op == "getReservation" and location in {"day", "detail"}:
|
||||
stay = doc["data"]["reservations"]["reservation"][0]["roomStay"]
|
||||
target = stay["roomRates"][0] if location == "day" else stay
|
||||
target["reservationBlock"] = block
|
||||
elif op == base.SEARCH and location == "search":
|
||||
doc["data"]["reservations"]["reservationInfo"][0]["roomStay"]["reservationBlock"] = block
|
||||
return doc
|
||||
self.service.edit = edit
|
||||
with self.subTest(location=location, block=block):
|
||||
_, field = self.field("BLOCK_CODE")
|
||||
self.assertEqual(field["state"], "missing")
|
||||
self.assertIsNone(field["value"])
|
||||
|
||||
def test_positive_or_unknown_package_indicator_never_becomes_empty(self):
|
||||
indicators = [
|
||||
[{"indicatorName": "PACKAGEITEM", "count": 1}],
|
||||
[{"indicatorName": "PACKAGEITEM"}],
|
||||
[{"indicatorName": "PACKAGEITEM", "count": True}],
|
||||
[{"indicatorName": "PACKAGEITEM", "count": -1}],
|
||||
[{"indicatorName": "PACKAGEITEM", "count": 0}] * 2,
|
||||
]
|
||||
for index, indicator in enumerate(indicators):
|
||||
for present in (False, True):
|
||||
self.root = Path(self.temp.name) / f"package-contradiction-{index}-{present}"
|
||||
self.service = SimulatedOHIP()
|
||||
row = self.service.rows[0]
|
||||
if not present:
|
||||
row.pop("reservationPackages")
|
||||
row["reservationIndicators"] = indicator
|
||||
with self.subTest(indicator=indicator, explicit_empty=present):
|
||||
result, field = self.field("PRODUCTS")
|
||||
self.assertEqual(result["status"], "collected_with_gaps")
|
||||
self.assertEqual(field["state"], "missing")
|
||||
self.assertIsNone(field["value"])
|
||||
|
||||
def test_malformed_packages_are_not_interpreted_as_normal_omission(self):
|
||||
for index, malformed in enumerate((None, {}, "", [None])):
|
||||
self.root = Path(self.temp.name) / f"package-malformed-{index}"
|
||||
self.service = SimulatedOHIP()
|
||||
self.service.rows[0]["reservationPackages"] = malformed
|
||||
self.service.rows[0]["reservationIndicators"] = []
|
||||
with self.subTest(value=malformed):
|
||||
_, field = self.field("PRODUCTS")
|
||||
self.assertEqual((field["state"], field["reason"]), ("missing", "invalid_reservation_packages"))
|
||||
|
||||
def test_other_block_evidence_prevents_automatic_empty(self):
|
||||
for index, extra in enumerate((None, {}, {"blockIdList": [{"type": "Block", "id": "b1"}]})):
|
||||
self.root = Path(self.temp.name) / f"block-other-evidence-{index}"
|
||||
self.service = SimulatedOHIP()
|
||||
stay = self.service.rows[0]["roomStay"]
|
||||
stay.pop("reservationBlock")
|
||||
stay["roomRates"][0].pop("reservationBlock")
|
||||
# A detail-only extra association must not silently be discarded.
|
||||
def edit(op, doc, _, extra=extra):
|
||||
if op == "getReservation":
|
||||
doc["data"]["reservations"]["reservation"][0]["roomStay"]["reservationBlock"] = extra
|
||||
return doc
|
||||
self.service.edit = edit
|
||||
with self.subTest(value=extra):
|
||||
_, field = self.field("BLOCK_CODE")
|
||||
self.assertEqual(field["state"], "missing")
|
||||
self.assertIsNone(field["value"])
|
||||
|
||||
def test_failed_block_supplement_is_not_normal_empty(self):
|
||||
block = {"blockIdList": [{"type": "Block", "id": "b1"}]}
|
||||
stay = self.service.rows[0]["roomStay"]
|
||||
stay["reservationBlock"] = copy.deepcopy(block)
|
||||
stay["roomRates"][0]["reservationBlock"] = copy.deepcopy(block)
|
||||
self.service.status = lambda op, count: 503 if op == "getBlock" else 200
|
||||
result, field = self.field("BLOCK_CODE")
|
||||
self.assertEqual((result["status"], field["state"], field["reason"]),
|
||||
("failed", "failed", "http_failure"))
|
||||
|
||||
def test_block_supplement_cannot_overwrite_earlier_code_or_hotel(self):
|
||||
for mismatch in ("code", "hotel"):
|
||||
|
||||
Reference in new issue
Block a user