ARR-2.0-0918/arr-opera-daily-ingest/references/field-contracts.md

Field Contracts

XML structure

Root:

/RES_DETAIL

Reservations:

/RES_DETAIL/LIST_G_GROUP_BY1/G_GROUP_BY1/LIST_G_RESERVATION/G_RESERVATION

XML source	Output field
ADULTS	ADULTS
BLOCK_CODE	BLOCK_CODE
CF_CHILDREN	CHILDREN
COMPANY_NAME	COMPANY_NAME
CONFIRMATION_NO	CONFIRMATION_NO
DISP_ROOM_NO	DISP_ROOM_NO
EFFECTIVE_RATE_AMOUNT	EFFECTIVE_RATE_AMOUNT
FULL_NAME	FULL_NAME
first non-empty LIST_G_COMMENT_RESV_NAME_ID/.../RES_COMMENT	RES_COMMENT
first non-empty LIST_G_DEPT_ID/.../TRACE_TEXT	TRACE_TEXT
NO_OF_ROOMS	NO_OF_ROOMS
PRODUCTS	PRODUCTS
RATE_CODE	RATE_CODE
ROOM_CATEGORY_LABEL	ROOM_CATEGORY_LABEL
TRUNC_BEGIN	ARRIVAL
TRUNC_END	DEPARTURE
computed	NIGHTS
price rule	REAL PRICE
computed	TOTAL PRICE

Group date candidates are GROUPBY1_SORT_COL (YYYYMMDD) and GROUPBY1_COL (DD-MM-YY). Both must agree when both exist. Every whitelist candidate ARRIVAL must equal the single group business date.

Emoji handling (processor 4.1.0)

The original source bytes, size and SHA-256 remain immutable. Parsing uses an in-memory copy:

• Remove recognized Unicode Emoji 17.0 sequences from element text and tails before extracting fields, including CDATA and XML numeric character references. The pinned emoji-sequences-17.0.txt covers complete sequences, qualification variants and emoji components; a family/flag/skin-tone/keycap sequence counts as one occurrence.
• If strict XML parsing fails, recognize CESU-8 surrogate-pair encodings only as part of a listed emoji in element text or CDATA, normalize those to UTF-8, and run the strict XML parser again. Never recover markup, attributes, unpaired surrogates, non-emoji surrogate pairs, arbitrary invalid bytes, or a different declared encoding this way.
• Do not remove whole Unicode blocks or use lossy decoding. Plain numbers, punctuation, currency, Chinese (including supplementary characters), Thai and other non-emoji text keep their values.
• XML structure, required fields, numeric/date validation, filtering, deduplication and pricing still apply to the resulting text. An emoji-only required value therefore fails the existing missing-value rule.
• When any emoji was ignored, result.json.input_cleanup.ignored_emoji_count is a positive integer; omit the metadata when the count is zero. The independent validator recomputes the count from the original XML on successful and review-required results. Upload receipts expose the count and the UI shows a localized notice.

This input normalization does not modify the fixed rate whitelist or price reference. The dataset and this policy are included in the processor rule identity. Data provenance/license: emoji-sequences-17.0.txt and unicode-license.txt. The retired, frozen v3 direct-MCP compatibility replay retains its original strict XML/emoji behavior.

Daily XLSX: exactly 19 columns

1. BLOCK_CODE
2. ADULTS
3. CHILDREN
4. COMPANY_NAME
5. CONFIRMATION_NO
6. DISP_ROOM_NO
7. EFFECTIVE_RATE_AMOUNT
8. FULL_NAME
9. RES_COMMENT
10. TRACE_TEXT
11. NO_OF_ROOMS
12. PRODUCTS
13. RATE_CODE
14. ROOM_CATEGORY_LABEL
15. ARRIVAL
16. DEPARTURE
17. NIGHTS
18. REAL PRICE
19. TOTAL PRICE

TOTAL PRICE = REAL PRICE * NO_OF_ROOMS * NIGHTS. All derived values are static numbers.

Direct XML lineage

• source_sequence: one-based XML reservation order.
• source_location: reservation[N].
• source_worksheet: null.
• source_row_no: null.
• channel_key: deterministic downstream channel fact; it is not an XML source worksheet.

Derived keys

normalized_rate_code = upper(trim(rate_code))
group_code_key       = upper(trim(res_comment))
company_key          = deterministic company keyword normalization

When trimmed RES_COMMENT is empty, group_code_key is null and booking_source_match_status is missing_group_code. Never substitute BLOCK_CODE, confirmation number, or another value.

Required whitelist-candidate values

RATE_CODE, COMPANY_NAME, CONFIRMATION_NO, DISP_ROOM_NO, EFFECTIVE_RATE_AMOUNT, FULL_NAME, ADULTS, CF_CHILDREN, NO_OF_ROOMS, ARRIVAL, and DEPARTURE must be present and valid.

BLOCK_CODE, PRODUCTS, ROOM_CATEGORY_LABEL, RES_COMMENT, and TRACE_TEXT may be blank.

The complete record field list and conditional nullability rules are authoritative in structured-result.schema.json.

Direct OHIP data input (processor 4.2.0)

--data-json accepts frozen arr-ohip-data/v1 field observations with complete collection, exact hotel/date context, unique reservation IDs and continuous source order. Classification, whitelist, room/arrival deduplication, pricing, zero-price exceptions, nights, channel assignment and formulas are shared with XML. An unresolved field on a retained candidate fails the batch; it is not treated as a missing price. Non-whitelist rows are excluded before business-field validation, as in XML.

The reservation comment uses the first nonempty note in the supplied order after emoji cleanup. PRODUCTS displays package codes in supplied order, separated by comma and space; duplicates are retained. All notes, package schedules/details and source-response references remain unchanged in the immutable source JSON. This is the explicit new data display contract, not a claim of byte-identical Oracle report formatting. Optional confirmed-empty values display blank. Trace is not fetched and its existing workbook column remains blank.