部署优化
This commit is contained in:
@@ -25,7 +25,7 @@ from committed `ARRIVAL` facts, publishes a validated workbook, and records meta
|
|||||||
| `arr_web.job_trace` | Read-only programmatic trace from persisted run/attempt/delivery/version/outbox facts |
|
| `arr_web.job_trace` | Read-only programmatic trace from persisted run/attempt/delivery/version/outbox facts |
|
||||||
| `monthly_reports.worker` | Dedicated leased outbox consumer with retry/dead-letter behavior |
|
| `monthly_reports.worker` | Dedicated leased outbox consumer with retry/dead-letter behavior |
|
||||||
| `monthly_reports.repository` / `reporting` schema | ARRIVAL-derived snapshots, report versions, lineage, manifests and artifact identities |
|
| `monthly_reports.repository` / `reporting` schema | ARRIVAL-derived snapshots, report versions, lineage, manifests and artifact identities |
|
||||||
| `monthly_reports`, `channel_analytics`, `company_reports` | Deterministic downstream consumers of committed facts |
|
| `monthly_reports`, `channel_analytics`, `company_reports` | Deterministic downstream consumers of committed facts; company reports build official XLSX files in Python/openpyxl inside Web |
|
||||||
| Web + Caddy Compose | Web-owned login/session boundary behind Caddy HTTPS; no browser-native Basic Auth and no MCP service |
|
| Web + Caddy Compose | Web-owned login/session boundary behind Caddy HTTPS; no browser-native Basic Auth and no MCP service |
|
||||||
|
|
||||||
## Important Boundaries
|
## Important Boundaries
|
||||||
@@ -52,4 +52,4 @@ from committed `ARRIVAL` facts, publishes a validated workbook, and records meta
|
|||||||
|
|
||||||
## Last Updated
|
## Last Updated
|
||||||
|
|
||||||
2026-07-31
|
2026-08-03
|
||||||
|
|||||||
@@ -13,8 +13,19 @@ blocked. A not-yet-ended period keeps its fixed C/O cutoff and must be rerun aft
|
|||||||
workbook needs those facts. Duplicate publication of the same semantic snapshot is idempotent: the first validated
|
workbook needs those facts. Duplicate publication of the same semantic snapshot is idempotent: the first validated
|
||||||
archive/result pair remains authoritative even if a retry rebuilds different XLSX bytes.
|
archive/result pair remains authoritative even if a retry rebuilds different XLSX bytes.
|
||||||
|
|
||||||
|
Company-channel XLSX generation is now Web-container deployable without the private `@oai/artifact-tool` npm package:
|
||||||
|
`company_reports.publishing.ArtifactToolBuilder` builds and reopens workbooks with Python/openpyxl, validates sheet
|
||||||
|
names/headers/row counts/no-formulas, and treats blank Excel cells as business-empty strings. Node/artifact-tool remains
|
||||||
|
only a monthly-worker packaging concern.
|
||||||
|
|
||||||
## Completed On 2026-08-03
|
## Completed On 2026-08-03
|
||||||
|
|
||||||
|
- Removed the private npm dependency from the company-channel report deployment path after CentOS Docker build failed
|
||||||
|
with `@oai/artifact-tool` 404. The company workbook builder now uses Python/openpyxl inside Web, keeps formula-injection
|
||||||
|
text escaping, produces no preview PNGs, and validates regenerated workbooks after saving. `compose.yaml` starts Web
|
||||||
|
with `--enable-company-reports`; HTTP/IP test overrides must keep that flag while omitting `--secure-cookies`.
|
||||||
|
Focused company-report and deployment-entrypoint tests pass 19/19. No database migration or live server deployment was
|
||||||
|
performed from this workstation.
|
||||||
- Made the mobile H5 dashboard publicly readable without weakening the desktop/operator boundary. Anonymous users can
|
- Made the mobile H5 dashboard publicly readable without weakening the desktop/operator boundary. Anonymous users can
|
||||||
load `/h5`, its H5 assets and purpose-built `/api/public/h5/months` plus `/api/public/h5/analytics` aggregate routes;
|
load `/h5`, its H5 assets and purpose-built `/api/public/h5/months` plus `/api/public/h5/analytics` aggregate routes;
|
||||||
the public projection omits source hashes and operational metadata. Generic analytics, legacy H5 APIs, desktop pages,
|
the public projection omits source hashes and operational metadata. Generic analytics, legacy H5 APIs, desktop pages,
|
||||||
@@ -439,9 +450,10 @@ archive/result pair remains authoritative even if a retry rebuilds different XLS
|
|||||||
detached and are not reboot-persistent.
|
detached and are not reboot-persistent.
|
||||||
- The latest operator-selected Web credentials were rotated on 2026-07-31, but the password still matches the username.
|
- The latest operator-selected Web credentials were rotated on 2026-07-31, but the password still matches the username.
|
||||||
Rotate it again to a distinct high-entropy value in Keychain, followed by one controlled Web restart.
|
Rotate it again to a distinct high-entropy value in Keychain, followed by one controlled Web restart.
|
||||||
- The workstation runs Web and worker as separate processes. The checked-in Compose image intentionally does not claim
|
- The workstation runs Web and worker as separate processes. The checked-in Compose image now supports company-channel
|
||||||
to run the worker because it does not package the workstation-only artifact-tool module; a production image/process
|
XLSX generation through Python/openpyxl in Web, but intentionally does not claim to run the monthly worker because it
|
||||||
manager must supply Node/artifact-tool and the shared output volume before enabling that service.
|
does not package the workstation-only monthly artifact-tool module; a production image/process manager must supply
|
||||||
|
Node/artifact-tool and the shared output volume before enabling that monthly service.
|
||||||
- A formally controlled no-PII server acceptance run remains appropriate after that deployment packaging is complete.
|
- A formally controlled no-PII server acceptance run remains appropriate after that deployment packaging is complete.
|
||||||
- ARR2.0 now has Git metadata; `main` tracks `origin/main`. Runtime credential values remain outside Git and project
|
- ARR2.0 now has Git metadata; `main` tracks `origin/main`. Runtime credential values remain outside Git and project
|
||||||
files.
|
files.
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
|
|
||||||
| Date | Task | Outcome | Docs Updated |
|
| Date | Task | Outcome | Docs Updated |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
|
| 2026-08-03 | Make company-report XLSX generation deploy without private npm | Replaced the company-report builder's Node/private `@oai/artifact-tool` runtime dependency with Python/openpyxl workbook generation and self-validation, removed the private package dependency, enabled `--enable-company-reports` in Compose Web, and documented that only the monthly worker still needs Node/artifact-tool packaging. Focused company-report and deployment-entrypoint tests pass 19/19; no migration or live deployment was performed | Current state/history, architecture, deployment evidence/index, stale item, runbooks |
|
||||||
| 2026-08-03 | Execute controlled fix and live-accept 2026-08 `01-10` company report | With explicit confirmation, stopped only the stale PID 11176 and started new listener PID 54127 through the existing Keychain-backed launcher. One authorized job `05cc547d…` succeeded 5/5 with row counts `54/18/7/1/15`, captured active Booking batch-7 source metadata, reused the existing version/artifact identities, and passed five HTTP download/hash checks. Logout and temporary-file cleanup completed; no Booking/Finance source or fact mutation occurred | Current state, runtime evidence/index, stale item, commitments, scoped planning record |
|
| 2026-08-03 | Execute controlled fix and live-accept 2026-08 `01-10` company report | With explicit confirmation, stopped only the stale PID 11176 and started new listener PID 54127 through the existing Keychain-backed launcher. One authorized job `05cc547d…` succeeded 5/5 with row counts `54/18/7/1/15`, captured active Booking batch-7 source metadata, reused the existing version/artifact identities, and passed five HTTP download/hash checks. Logout and temporary-file cleanup completed; no Booking/Finance source or fact mutation occurred | Current state, runtime evidence/index, stale item, commitments, scoped planning record |
|
||||||
| 2026-08-03 | Diagnose the latest 2026-08 `01-10` official-Excel save failure | Read-only evidence confirms newest job `087dceca...` built all five companies (`54/18/7/1/15`) and failed only at publish. Port 8766 is still PID 11176 from 2026-08-02, older than the semantic-reuse publisher fix; the earlier successful five-file archive/result/current set remains hash-consistent and intact. The launcher preflight is ready and six publisher tests pass. Repair is an exact controlled listener replacement followed by one authorized rerun; no restart, report write or business-data mutation was performed | Current state, runtime evidence/index, stale item, scoped planning record |
|
| 2026-08-03 | Diagnose the latest 2026-08 `01-10` official-Excel save failure | Read-only evidence confirms newest job `087dceca...` built all five companies (`54/18/7/1/15`) and failed only at publish. Port 8766 is still PID 11176 from 2026-08-02, older than the semantic-reuse publisher fix; the earlier successful five-file archive/result/current set remains hash-consistent and intact. The launcher preflight is ready and six publisher tests pass. Repair is an exact controlled listener replacement followed by one authorized rerun; no restart, report write or business-data mutation was performed | Current state, runtime evidence/index, stale item, scoped planning record |
|
||||||
| 2026-08-03 | Check whether company-report retries show reused-file identity | Confirmed the publisher can reuse an existing archive/result pair, but the outcome, job API and frontend expose only success/version/download; users cannot see that the file was reused or which prior artifact supplied it. No code or business-data change was made | Current state, retry-semantics evidence/index |
|
| 2026-08-03 | Check whether company-report retries show reused-file identity | Confirmed the publisher can reuse an existing archive/result pair, but the outcome, job API and frontend expose only success/version/download; users cannot see that the file was reused or which prior artifact supplied it. No code or business-data change was made | Current state, retry-semantics evidence/index |
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ Use this index for searchable, traceable evidence records.
|
|||||||
|
|
||||||
| Date | Topic | Status | Source | Detail |
|
| Date | Topic | Status | Source | Detail |
|
||||||
|---|---|---|---|---|
|
|---|---|---|---|---|
|
||||||
|
| 2026-08-03 | Company-report openpyxl builder for deployment | Implemented; focused deployment checks pass | [Evidence topic](topics/2026-08-03-company-report-openpyxl-builder.md) | CentOS Docker build failed because public npm cannot install private `@oai/artifact-tool`. Company-report XLSX generation now runs through Python/openpyxl in Web, while monthly worker packaging remains the only Node/artifact-tool concern. Focused company-report/deployment tests pass 19/19. |
|
||||||
| 2026-08-03 | Booking source and company-report retry semantics | Source contract confirmed; reuse visibility gap identified; live acceptance pending restart | [Evidence topic](topics/2026-08-03-company-report-retry-semantics.md) | Byte-identical XLSX uploads reuse the activated source; same rows with different file bytes are new drafts. Same report snapshots reuse complete publication pairs, changed Finance/Booking pins create new versions, and an activated subset changes Booking detail coverage while retaining Finance rows. The current API/UI do not expose whether a retry reused an existing artifact. |
|
| 2026-08-03 | Booking source and company-report retry semantics | Source contract confirmed; reuse visibility gap identified; live acceptance pending restart | [Evidence topic](topics/2026-08-03-company-report-retry-semantics.md) | Byte-identical XLSX uploads reuse the activated source; same rows with different file bytes are new drafts. Same report snapshots reuse complete publication pairs, changed Finance/Booking pins create new versions, and an activated subset changes Booking detail coverage while retaining Finance rows. The current API/UI do not expose whether a retry reused an existing artifact. |
|
||||||
| 2026-08-03 | Company-report retry after reported 8766 restart | Resolved by controlled listener replacement and live 5/5 rerun | [Evidence topic](topics/2026-08-03-company-source-runtime-diagnosis.md) | PID 11176 was replaced by PID 54127. Job `05cc547d…` succeeded all five August `01-10` companies, reused the prior version/artifact identities, and all five authenticated downloads returned hash-matching 200 responses. No Booking/Finance source or fact changed. |
|
| 2026-08-03 | Company-report retry after reported 8766 restart | Resolved by controlled listener replacement and live 5/5 rerun | [Evidence topic](topics/2026-08-03-company-source-runtime-diagnosis.md) | PID 11176 was replaced by PID 54127. Job `05cc547d…` succeeded all five August `01-10` companies, reused the prior version/artifact identities, and all five authenticated downloads returned hash-matching 200 responses. No Booking/Finance source or fact changed. |
|
||||||
| 2026-08-03 | Remove redundant company-report Excel provenance module | Implemented; Web/company checks pass | [Evidence topic](topics/2026-08-03-company-report-status-copy-fix.md) | The job-detail source card and history source column are removed because reports consume the Booking database snapshot; upload/review source UI and server-side provenance remain available. |
|
| 2026-08-03 | Remove redundant company-report Excel provenance module | Implemented; Web/company checks pass | [Evidence topic](topics/2026-08-03-company-report-status-copy-fix.md) | The job-detail source card and history source column are removed because reports consume the Booking database snapshot; upload/review source UI and server-side provenance remain available. |
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Company-Report Openpyxl Builder
|
||||||
|
|
||||||
|
Date: 2026-08-03
|
||||||
|
|
||||||
|
## Finding
|
||||||
|
|
||||||
|
The CentOS test-server Docker build failed at `npm --prefix company_reports/xlsx install` because
|
||||||
|
`@oai/artifact-tool@2.8.33` is not available from the public npm registry. This is a packaging/runtime dependency
|
||||||
|
failure, not a database, migration or Docker memory issue.
|
||||||
|
|
||||||
|
## Change
|
||||||
|
|
||||||
|
- `company_reports.publishing.ArtifactToolBuilder` now builds official company-channel workbooks with Python/openpyxl.
|
||||||
|
- The builder writes the three expected sheets, preserves the existing headers/styles, escapes text that could be
|
||||||
|
interpreted as formulas, reopens the saved workbook and validates sheet names, dimensions, headers, row values and
|
||||||
|
zero formula cells.
|
||||||
|
- Blank cells read back from Excel as `None` are normalized to the business-empty string during validation.
|
||||||
|
- `company_reports/xlsx/package.json` no longer declares the private `@oai/artifact-tool` dependency.
|
||||||
|
- `compose.yaml` enables `--enable-company-reports` for Web. IP/HTTP test overrides should keep that flag but omit
|
||||||
|
`--secure-cookies`.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
- Watched the new builder test fail against the old Node/private-runtime implementation with
|
||||||
|
`BuildError('the XLSX builder runtime is unavailable')`.
|
||||||
|
- Focused verification passed:
|
||||||
|
`python -m unittest tests.test_company_reports_publishing tests.test_company_reports_integration tests.test_company_reports_acceptance tests.test_company_reports_service tests.test_deployment_entrypoints`
|
||||||
|
with 19 tests passing.
|
||||||
|
|
||||||
|
## Remaining Boundaries
|
||||||
|
|
||||||
|
The monthly worker still uses its separate Node/artifact-tool workbook builder. This change makes company-channel detail
|
||||||
|
Excel generation deployable in the Web image; it does not package or supervise the monthly worker.
|
||||||
@@ -16,7 +16,7 @@
|
|||||||
| 2026-07-30 | `booking_test` current Finance projection | The earlier 417-row snapshot included accepted run `mvp-v1-fixture-20260727` from `synthetic.xml`/`local_fixture`. A 2026-07-31 14:17 read-only company-report recheck found 986 current supported-company facts, so the old 416/417 remediation target is no longer a complete description of current Finance state | Re-audit current daily-version pins and source provenance before any fixture retirement; require explicit authorization for version changes, then verify the clean target and watermark |
|
| 2026-07-30 | `booking_test` current Finance projection | The earlier 417-row snapshot included accepted run `mvp-v1-fixture-20260727` from `synthetic.xml`/`local_fixture`. A 2026-07-31 14:17 read-only company-report recheck found 986 current supported-company facts, so the old 416/417 remediation target is no longer a complete description of current Finance state | Re-audit current daily-version pins and source provenance before any fixture retirement; require explicit authorization for version changes, then verify the clean target and watermark |
|
||||||
| 2026-07-30 | Channel BI refresh lifecycle | Resolved 2026-08-03. Desktop and public H5 now check selected-month metadata every five seconds while visible and reload full analytics only after `updated_at` changes; hidden views pause and transient failures preserve the last good snapshot | Refresh the browser once to load the new static assets, then observe the next authorized publication as a live acceptance check |
|
| 2026-07-30 | Channel BI refresh lifecycle | Resolved 2026-08-03. Desktop and public H5 now check selected-month metadata every five seconds while visible and reload full analytics only after `updated_at` changes; hidden views pause and transient failures preserve the last good snapshot | Refresh the browser once to load the new static assets, then observe the next authorized publication as a live acceptance check |
|
||||||
| 2026-07-30 | Channel BI KPI label | The card labeled `公司数` renders worksheet-level `channel_count`; LianTai GROUP/FIT are two channels, so six does not mean six companies | Decide whether to relabel it `渠道/子表数` or implement an explicit five-company aggregation |
|
| 2026-07-30 | Channel BI KPI label | The card labeled `公司数` renders worksheet-level `channel_count`; LianTai GROUP/FIT are two channels, so six does not mean six companies | Decide whether to relabel it `渠道/子表数` or implement an explicit five-company aggregation |
|
||||||
| 2026-07-30 | Production worker packaging | Local Web/worker separation is live, but the checked-in Compose image does not contain the workstation-only Node/artifact-tool runtime required by the XLSX builder | Package an approved builder runtime and shared output volume before adding/enabling the managed production worker service |
|
| 2026-07-30 | Production monthly-worker packaging | Local Web/worker separation is live, and company-report XLSX generation now runs in Web through Python/openpyxl. The checked-in Compose image still does not contain the workstation-only Node/artifact-tool runtime required by the monthly workbook builder | Package an approved monthly builder runtime and shared output volume before adding/enabling the managed production monthly-worker service |
|
||||||
|
|
||||||
## Superseded For ARR2.0
|
## Superseded For ARR2.0
|
||||||
|
|
||||||
|
|||||||
@@ -47,9 +47,7 @@ cp .env.example .env.local
|
|||||||
--port 8765 \
|
--port 8765 \
|
||||||
--enable-processing \
|
--enable-processing \
|
||||||
--enable-monthly-generation \
|
--enable-monthly-generation \
|
||||||
--enable-company-reports \
|
--enable-company-reports
|
||||||
--node-binary /absolute/path/to/node \
|
|
||||||
--artifact-tool-module /absolute/path/to/artifact_tool.mjs
|
|
||||||
```
|
```
|
||||||
|
|
||||||
另开一个进程启动月报消费者:
|
另开一个进程启动月报消费者:
|
||||||
@@ -91,8 +89,9 @@ worker 是独立无端口进程,应由进程管理器单独保活。
|
|||||||
## 部署
|
## 部署
|
||||||
|
|
||||||
生产配置见 [deploy/README.md](deploy/README.md)。当前 Compose 模板仍只打包 `web` 与 `caddy`,Caddy 负责
|
生产配置见 [deploy/README.md](deploy/README.md)。当前 Compose 模板仍只打包 `web` 与 `caddy`,Caddy 负责
|
||||||
HTTPS,Web 负责应用登录和会话;没有 MCP 端口、MCP 域名或 Agent Secret。月报 worker 必须作为独立进程部署,
|
HTTPS,Web 负责应用登录和会话;没有 MCP 端口、MCP 域名或 Agent Secret。公司渠道明细由 Web 进程使用
|
||||||
并使用同一数据库、共享输出卷以及已经打包 Node/artifact-tool 的运行镜像;当前本地工作站已按这一方式运行。
|
Python/openpyxl 生成正式 Excel,不需要 Node/artifact-tool。月报 worker 必须作为独立进程部署,并使用同一
|
||||||
|
数据库、共享输出卷以及已经打包 Node/artifact-tool 的运行镜像;当前本地工作站已按这一方式运行。
|
||||||
|
|
||||||
## 当前月报行为
|
## 当前月报行为
|
||||||
|
|
||||||
|
|||||||
@@ -22,14 +22,13 @@ Supported companies: `LianTai`, `QBD`, `DY-AI-Easy-KB`, `FengRun`, `HanaTour`.
|
|||||||
python3 -m pip install -r requirements-company-reports.txt
|
python3 -m pip install -r requirements-company-reports.txt
|
||||||
|
|
||||||
export COMPANY_REPORT_DATABASE_URL='postgresql://USER:PASSWORD@HOST:PORT/booking_test'
|
export COMPANY_REPORT_DATABASE_URL='postgresql://USER:PASSWORD@HOST:PORT/booking_test'
|
||||||
export COMPANY_REPORT_NODE_BINARY='/absolute/path/to/node'
|
|
||||||
export COMPANY_REPORT_ARTIFACT_TOOL_MODULE='/absolute/path/to/artifact_tool.mjs'
|
|
||||||
|
|
||||||
python3 -m company_reports generate \
|
python3 -m company_reports generate \
|
||||||
--month 2026-07 \
|
--month 2026-07 \
|
||||||
--as-of 2026-07-10
|
--as-of 2026-07-10
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The workbook builder uses Python/openpyxl. No Node.js or `@oai/artifact-tool` package is required for company reports.
|
||||||
The DSN can fall back to `ARR_DATABASE_URL`; never place a real DSN in source, prompts, output JSON or browser code.
|
The DSN can fall back to `ARR_DATABASE_URL`; never place a real DSN in source, prompts, output JSON or browser code.
|
||||||
|
|
||||||
Scheduling convention:
|
Scheduling convention:
|
||||||
|
|||||||
@@ -71,11 +71,11 @@ def _parser() -> SafeArgumentParser:
|
|||||||
)
|
)
|
||||||
generate.add_argument(
|
generate.add_argument(
|
||||||
"--node-binary",
|
"--node-binary",
|
||||||
help="Node.js executable; defaults to COMPANY_REPORT_NODE_BINARY or PATH",
|
help="deprecated compatibility option; ignored by the openpyxl builder",
|
||||||
)
|
)
|
||||||
generate.add_argument(
|
generate.add_argument(
|
||||||
"--artifact-tool-module",
|
"--artifact-tool-module",
|
||||||
help="absolute path to artifact_tool.mjs when package resolution is unavailable",
|
help="deprecated compatibility option; ignored by the openpyxl builder",
|
||||||
)
|
)
|
||||||
return parser
|
return parser
|
||||||
|
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Artifact-tool build adapter and recoverable atomic publication."""
|
"""Workbook build adapter and recoverable atomic publication."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -6,13 +6,17 @@ import hashlib
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
|
||||||
import tempfile
|
import tempfile
|
||||||
import uuid
|
import uuid
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
|
from datetime import date, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Dict, Mapping, Optional, Tuple
|
from typing import Any, Dict, Mapping, Optional, Tuple
|
||||||
|
|
||||||
|
from openpyxl import Workbook, load_workbook
|
||||||
|
from openpyxl.styles import Alignment, Border, Font, PatternFill, Side
|
||||||
|
from openpyxl.worksheet.worksheet import Worksheet
|
||||||
|
|
||||||
from company_reports.contracts import CompanyReport, ErrorCode, RESULT_SCHEMA_VERSION
|
from company_reports.contracts import CompanyReport, ErrorCode, RESULT_SCHEMA_VERSION
|
||||||
from company_reports.repository import (
|
from company_reports.repository import (
|
||||||
FileMetadata,
|
FileMetadata,
|
||||||
@@ -23,6 +27,21 @@ from company_reports.repository import (
|
|||||||
|
|
||||||
|
|
||||||
XLSX_MIME = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
|
XLSX_MIME = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
|
||||||
|
REPORT_HEADERS = (
|
||||||
|
"ARRIVAL",
|
||||||
|
"DEPARTURE",
|
||||||
|
"NIGHTS",
|
||||||
|
"BLOCK_CODE",
|
||||||
|
"RES_COMMENT",
|
||||||
|
"Booking Room",
|
||||||
|
"Total Booking Price",
|
||||||
|
)
|
||||||
|
BODY_FONT = "222222"
|
||||||
|
DUPLICATE_FILL = "FFF2CC"
|
||||||
|
DUPLICATE_FONT = "9C6500"
|
||||||
|
REVIEW_FILL = "FCE4D6"
|
||||||
|
REVIEW_FONT = "C65911"
|
||||||
|
HEADER_FILL = "FFFFFF"
|
||||||
|
|
||||||
|
|
||||||
class BuildError(RuntimeError):
|
class BuildError(RuntimeError):
|
||||||
@@ -89,63 +108,196 @@ class ArtifactToolBuilder:
|
|||||||
artifact_tool_module: Optional[Path] = None,
|
artifact_tool_module: Optional[Path] = None,
|
||||||
timeout_seconds: int = 120,
|
timeout_seconds: int = 120,
|
||||||
) -> None:
|
) -> None:
|
||||||
configured = (node_binary or os.environ.get("COMPANY_REPORT_NODE_BINARY", "")).strip()
|
_ = (node_binary, artifact_tool_module, timeout_seconds)
|
||||||
self._node_binary = configured or shutil.which("node") or ""
|
|
||||||
self._builder_script = builder_script.resolve()
|
self._builder_script = builder_script.resolve()
|
||||||
self._artifact_tool_module = (
|
|
||||||
artifact_tool_module.resolve() if artifact_tool_module else None
|
@staticmethod
|
||||||
|
def _safe_text(value: Any) -> str:
|
||||||
|
text = str(value or "")
|
||||||
|
return "'" + text if text[:1] in {"=", "+", "-", "@"} else text
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _date_key(value: Any) -> str:
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
return value.date().isoformat()
|
||||||
|
if isinstance(value, date):
|
||||||
|
return value.isoformat()
|
||||||
|
if isinstance(value, str) and len(value) >= 10:
|
||||||
|
return value[:10]
|
||||||
|
return ""
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _cell_text(value: Any) -> str:
|
||||||
|
return "" if value is None else str(value)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _semantic_sha256(payload: Mapping[str, Any]) -> str:
|
||||||
|
encoded = json.dumps(
|
||||||
|
{
|
||||||
|
"headers": payload["headers"],
|
||||||
|
"periods": payload["periods"],
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
separators=(",", ":"),
|
||||||
|
).encode("utf-8")
|
||||||
|
return hashlib.sha256(encoded).hexdigest()
|
||||||
|
|
||||||
|
def _write_sheet(
|
||||||
|
self,
|
||||||
|
worksheet: Worksheet,
|
||||||
|
period: Any,
|
||||||
|
) -> None:
|
||||||
|
worksheet.append(list(REPORT_HEADERS))
|
||||||
|
for row in period.rows:
|
||||||
|
worksheet.append(
|
||||||
|
[
|
||||||
|
row.arrival,
|
||||||
|
row.departure,
|
||||||
|
row.nights,
|
||||||
|
self._safe_text(row.block_code),
|
||||||
|
self._safe_text(row.res_comment),
|
||||||
|
self._safe_text(row.booking_room),
|
||||||
|
self._safe_text(row.total_booking_price),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
worksheet.column_dimensions["A"].width = 13
|
||||||
|
worksheet.column_dimensions["B"].width = 13
|
||||||
|
worksheet.column_dimensions["C"].width = 9
|
||||||
|
worksheet.column_dimensions["D"].width = 20
|
||||||
|
worksheet.column_dimensions["E"].width = 25
|
||||||
|
worksheet.column_dimensions["F"].width = 34
|
||||||
|
worksheet.column_dimensions["G"].width = 48
|
||||||
|
worksheet.freeze_panes = "A2"
|
||||||
|
worksheet.auto_filter.ref = f"A1:G{max(1, worksheet.max_row)}"
|
||||||
|
|
||||||
|
thin_bottom = Border(
|
||||||
|
bottom=Side(style="thin", color="7F7F7F"),
|
||||||
)
|
)
|
||||||
self._timeout_seconds = timeout_seconds
|
header_fill = PatternFill("solid", fgColor=HEADER_FILL)
|
||||||
|
duplicate_fill = PatternFill("solid", fgColor=DUPLICATE_FILL)
|
||||||
|
review_fill = PatternFill("solid", fgColor=REVIEW_FILL)
|
||||||
|
for cell in worksheet[1]:
|
||||||
|
cell.fill = header_fill
|
||||||
|
cell.font = Font(name="Arial", size=10, bold=True, color="000000")
|
||||||
|
cell.alignment = Alignment(horizontal="center", vertical="center")
|
||||||
|
cell.border = thin_bottom
|
||||||
|
worksheet.row_dimensions[1].height = 24
|
||||||
|
|
||||||
|
for row_index in range(2, worksheet.max_row + 1):
|
||||||
|
worksheet.row_dimensions[row_index].height = 30
|
||||||
|
for cell in worksheet[row_index]:
|
||||||
|
cell.font = Font(name="Arial", size=10, color=BODY_FONT)
|
||||||
|
cell.alignment = Alignment(vertical="center")
|
||||||
|
for column in ("A", "B", "C", "D"):
|
||||||
|
worksheet[f"{column}{row_index}"].alignment = Alignment(
|
||||||
|
horizontal="center",
|
||||||
|
vertical="center",
|
||||||
|
)
|
||||||
|
for column in ("E", "F", "G"):
|
||||||
|
worksheet[f"{column}{row_index}"].alignment = Alignment(
|
||||||
|
horizontal="left",
|
||||||
|
vertical="center",
|
||||||
|
wrap_text=True,
|
||||||
|
)
|
||||||
|
worksheet[f"A{row_index}"].number_format = "yyyy-mm-dd"
|
||||||
|
worksheet[f"B{row_index}"].number_format = "yyyy-mm-dd"
|
||||||
|
|
||||||
|
for offset, source_row in enumerate(period.rows, start=2):
|
||||||
|
if source_row.duplicate_group:
|
||||||
|
for column in range(1, 8):
|
||||||
|
worksheet.cell(offset, column).fill = duplicate_fill
|
||||||
|
worksheet[f"E{offset}"].font = Font(
|
||||||
|
name="Arial",
|
||||||
|
size=10,
|
||||||
|
bold=True,
|
||||||
|
color=DUPLICATE_FONT,
|
||||||
|
)
|
||||||
|
if source_row.multi_price_review:
|
||||||
|
worksheet[f"G{offset}"].fill = review_fill
|
||||||
|
worksheet[f"G{offset}"].font = Font(
|
||||||
|
name="Arial",
|
||||||
|
size=10,
|
||||||
|
bold=True,
|
||||||
|
color=REVIEW_FONT,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _validate_workbook(self, output_path: Path, report: CompanyReport) -> int:
|
||||||
|
workbook = load_workbook(output_path, data_only=False)
|
||||||
|
expected_names = [period.sheet_name for period in report.periods]
|
||||||
|
if workbook.sheetnames != expected_names:
|
||||||
|
raise ValueError("worksheet names do not match")
|
||||||
|
formula_count = 0
|
||||||
|
for period in report.periods:
|
||||||
|
worksheet = workbook[period.sheet_name]
|
||||||
|
expected_rows = len(period.rows) + 1
|
||||||
|
if worksheet.max_row != expected_rows or worksheet.max_column != 7:
|
||||||
|
raise ValueError("worksheet dimensions do not match")
|
||||||
|
headers = [worksheet.cell(1, column).value for column in range(1, 8)]
|
||||||
|
if headers != list(REPORT_HEADERS):
|
||||||
|
raise ValueError("worksheet headers do not match")
|
||||||
|
for row_index, row in enumerate(period.rows, start=2):
|
||||||
|
expected = [
|
||||||
|
row.arrival.isoformat(),
|
||||||
|
row.departure.isoformat(),
|
||||||
|
row.nights,
|
||||||
|
self._safe_text(row.block_code),
|
||||||
|
self._safe_text(row.res_comment),
|
||||||
|
self._safe_text(row.booking_room),
|
||||||
|
self._safe_text(row.total_booking_price),
|
||||||
|
]
|
||||||
|
actual = [
|
||||||
|
self._date_key(worksheet.cell(row_index, 1).value),
|
||||||
|
self._date_key(worksheet.cell(row_index, 2).value),
|
||||||
|
worksheet.cell(row_index, 3).value,
|
||||||
|
self._cell_text(worksheet.cell(row_index, 4).value),
|
||||||
|
self._cell_text(worksheet.cell(row_index, 5).value),
|
||||||
|
self._cell_text(worksheet.cell(row_index, 6).value),
|
||||||
|
self._cell_text(worksheet.cell(row_index, 7).value),
|
||||||
|
]
|
||||||
|
if actual != expected:
|
||||||
|
raise ValueError("worksheet values do not match")
|
||||||
|
for cells in worksheet.iter_rows():
|
||||||
|
for cell in cells:
|
||||||
|
if cell.data_type == "f":
|
||||||
|
formula_count += 1
|
||||||
|
if formula_count != 0:
|
||||||
|
raise ValueError("workbook must contain no formulas")
|
||||||
|
return formula_count
|
||||||
|
|
||||||
def build(self, report: CompanyReport, work_dir: Path) -> BuiltWorkbook:
|
def build(self, report: CompanyReport, work_dir: Path) -> BuiltWorkbook:
|
||||||
if not self._node_binary or not self._builder_script.is_file():
|
|
||||||
raise BuildError(
|
|
||||||
ErrorCode.OUTPUT_VALIDATION_FAILED,
|
|
||||||
"the XLSX builder runtime is unavailable",
|
|
||||||
)
|
|
||||||
work_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
|
work_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||||
os.chmod(work_dir, 0o700)
|
os.chmod(work_dir, 0o700)
|
||||||
payload_path = work_dir / "workbook-payload.json"
|
payload_path = work_dir / "workbook-payload.json"
|
||||||
output_path = work_dir / report.filename
|
output_path = work_dir / report.filename
|
||||||
preview_dir = work_dir / "previews"
|
preview_dir = work_dir / "previews"
|
||||||
summary_path = work_dir / "workbook-summary.json"
|
summary_path = work_dir / "workbook-summary.json"
|
||||||
_private_json(payload_path, report.to_workbook_payload())
|
|
||||||
try:
|
try:
|
||||||
environment = os.environ.copy()
|
payload = report.to_workbook_payload()
|
||||||
if self._artifact_tool_module is not None:
|
_private_json(payload_path, payload)
|
||||||
environment["COMPANY_REPORT_ARTIFACT_TOOL_MODULE"] = str(
|
preview_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
|
||||||
self._artifact_tool_module
|
workbook = Workbook()
|
||||||
)
|
for index, period in enumerate(report.periods):
|
||||||
completed = subprocess.run(
|
worksheet = workbook.active if index == 0 else workbook.create_sheet()
|
||||||
[
|
worksheet.title = period.sheet_name
|
||||||
self._node_binary,
|
self._write_sheet(worksheet, period)
|
||||||
str(self._builder_script),
|
workbook.save(output_path)
|
||||||
str(payload_path),
|
os.chmod(output_path, 0o600)
|
||||||
str(output_path),
|
formula_count = self._validate_workbook(output_path, report)
|
||||||
str(preview_dir),
|
|
||||||
str(summary_path),
|
|
||||||
],
|
|
||||||
cwd=self._builder_script.parent,
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
timeout=self._timeout_seconds,
|
|
||||||
env=environment,
|
|
||||||
check=False,
|
|
||||||
)
|
|
||||||
except (OSError, subprocess.TimeoutExpired):
|
|
||||||
raise BuildError(
|
|
||||||
ErrorCode.OUTPUT_VALIDATION_FAILED,
|
|
||||||
"the XLSX builder did not complete",
|
|
||||||
) from None
|
|
||||||
if completed.returncode != 0:
|
|
||||||
raise BuildError(
|
|
||||||
ErrorCode.OUTPUT_VALIDATION_FAILED,
|
|
||||||
"the XLSX builder rejected the generated payload",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
summary = json.loads(summary_path.read_text(encoding="utf-8"))
|
|
||||||
expected_rows = [len(period.rows) for period in report.periods]
|
expected_rows = [len(period.rows) for period in report.periods]
|
||||||
expected_names = [period.sheet_name for period in report.periods]
|
expected_names = [period.sheet_name for period in report.periods]
|
||||||
|
summary = {
|
||||||
|
"status": "success",
|
||||||
|
"schema_version": payload["schema_version"],
|
||||||
|
"company": report.company,
|
||||||
|
"filename": report.filename,
|
||||||
|
"sheet_names": expected_names,
|
||||||
|
"row_counts": expected_rows,
|
||||||
|
"formula_count": formula_count,
|
||||||
|
"semantic_sha256": self._semantic_sha256(payload),
|
||||||
|
"preview_count": 0,
|
||||||
|
"byte_size": output_path.stat().st_size,
|
||||||
|
}
|
||||||
if (
|
if (
|
||||||
summary.get("status") != "success"
|
summary.get("status") != "success"
|
||||||
or summary.get("company") != report.company
|
or summary.get("company") != report.company
|
||||||
@@ -155,17 +307,16 @@ class ArtifactToolBuilder:
|
|||||||
or summary.get("formula_count") != 0
|
or summary.get("formula_count") != 0
|
||||||
or not isinstance(summary.get("semantic_sha256"), str)
|
or not isinstance(summary.get("semantic_sha256"), str)
|
||||||
or len(summary.get("semantic_sha256")) != 64
|
or len(summary.get("semantic_sha256")) != 64
|
||||||
or summary.get("preview_count") != 3
|
|
||||||
or not output_path.is_file()
|
or not output_path.is_file()
|
||||||
or output_path.stat().st_size <= 0
|
or output_path.stat().st_size <= 0
|
||||||
):
|
):
|
||||||
raise ValueError("summary mismatch")
|
raise ValueError("summary mismatch")
|
||||||
|
_private_json(summary_path, summary)
|
||||||
except (OSError, ValueError, TypeError, json.JSONDecodeError):
|
except (OSError, ValueError, TypeError, json.JSONDecodeError):
|
||||||
raise BuildError(
|
raise BuildError(
|
||||||
ErrorCode.OUTPUT_VALIDATION_FAILED,
|
ErrorCode.OUTPUT_VALIDATION_FAILED,
|
||||||
"the XLSX builder result could not be validated",
|
"the XLSX builder result could not be validated",
|
||||||
) from None
|
) from None
|
||||||
os.chmod(output_path, 0o600)
|
|
||||||
return BuiltWorkbook(
|
return BuiltWorkbook(
|
||||||
path=output_path,
|
path=output_path,
|
||||||
sha256=sha256_file(output_path),
|
sha256=sha256_file(output_path),
|
||||||
|
|||||||
@@ -1,8 +1,5 @@
|
|||||||
{
|
{
|
||||||
"name": "company-report-xlsx-builder",
|
"name": "company-report-xlsx-builder",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module"
|
||||||
"dependencies": {
|
|
||||||
"@oai/artifact-tool": "2.8.33"
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ services:
|
|||||||
- --port
|
- --port
|
||||||
- "8765"
|
- "8765"
|
||||||
- --enable-processing
|
- --enable-processing
|
||||||
|
- --enable-company-reports
|
||||||
- --secure-cookies
|
- --secure-cookies
|
||||||
environment:
|
environment:
|
||||||
ARR_WEB_USERNAME: "${ARR_WEB_USERNAME:?set ARR_WEB_USERNAME in deploy/.env.production}"
|
ARR_WEB_USERNAME: "${ARR_WEB_USERNAME:?set ARR_WEB_USERNAME in deploy/.env.production}"
|
||||||
|
|||||||
@@ -8,7 +8,8 @@
|
|||||||
- PostgreSQL 15+:目标必须是隔离数据库 `booking_test`,并已应用 008–012 权威迁移;
|
- PostgreSQL 15+:目标必须是隔离数据库 `booking_test`,并已应用 008–012 权威迁移;
|
||||||
- OSS:区域匹配、服务端加密、未启用或暂停 versioning、禁止匿名写;bucket ACL 可为 private 或 public-read;
|
- OSS:区域匹配、服务端加密、未启用或暂停 versioning、禁止匿名写;bucket ACL 可为 private 或 public-read;
|
||||||
- ARR 数据库账号和 OSS RAM/STS 凭据由 Secret 管理器注入,不写入镜像或仓库。
|
- ARR 数据库账号和 OSS RAM/STS 凭据由 Secret 管理器注入,不写入镜像或仓库。
|
||||||
- worker 与 Web 使用同一数据库和共享的 `/app/outputs` 持久卷;worker 镜像还必须包含 Node.js 与配置匹配的 artifact-tool 模块。
|
- 公司渠道明细由 Web 进程使用 Python/openpyxl 生成正式 Excel;不需要 Node.js 或 `@oai/artifact-tool`。
|
||||||
|
- worker 与 Web 使用同一数据库和共享的 `/app/outputs` 持久卷;月报 worker 镜像还必须包含 Node.js 与配置匹配的 artifact-tool 模块。
|
||||||
|
|
||||||
## 配置
|
## 配置
|
||||||
|
|
||||||
|
|||||||
@@ -1 +1,2 @@
|
|||||||
psycopg[binary]==3.2.13
|
psycopg[binary]==3.2.13
|
||||||
|
openpyxl==3.1.5
|
||||||
|
|||||||
@@ -1,7 +1,5 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -19,14 +17,6 @@ from tests.company_reports_acceptance_fixture import (
|
|||||||
|
|
||||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||||
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
|
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
|
||||||
ARTIFACT_PACKAGE = (
|
|
||||||
PROJECT_ROOT
|
|
||||||
/ "company_reports"
|
|
||||||
/ "xlsx"
|
|
||||||
/ "node_modules"
|
|
||||||
/ "@oai"
|
|
||||||
/ "artifact-tool"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def acceptance_reports():
|
def acceptance_reports():
|
||||||
@@ -38,16 +28,6 @@ def acceptance_reports():
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def artifact_tool_available() -> bool:
|
|
||||||
configured = os.environ.get("COMPANY_REPORT_ARTIFACT_TOOL_MODULE", "").strip()
|
|
||||||
return ARTIFACT_PACKAGE.exists() or bool(configured and Path(configured).is_file())
|
|
||||||
|
|
||||||
|
|
||||||
def node_binary() -> str:
|
|
||||||
configured = os.environ.get("COMPANY_REPORT_NODE_BINARY", "").strip()
|
|
||||||
return configured or shutil.which("node") or ""
|
|
||||||
|
|
||||||
|
|
||||||
class CompanyReportAcceptanceCoreTests(unittest.TestCase):
|
class CompanyReportAcceptanceCoreTests(unittest.TestCase):
|
||||||
def test_all_five_companies_have_contract_complete_month_end_reports(self):
|
def test_all_five_companies_have_contract_complete_month_end_reports(self):
|
||||||
reports = acceptance_reports()
|
reports = acceptance_reports()
|
||||||
@@ -106,15 +86,10 @@ class CompanyReportAcceptanceCoreTests(unittest.TestCase):
|
|||||||
self.assertEqual(dy_row.departure.day, 20)
|
self.assertEqual(dy_row.departure.day, 20)
|
||||||
|
|
||||||
|
|
||||||
@unittest.skipUnless(artifact_tool_available(), "artifact-tool dependency is not installed")
|
|
||||||
class CompanyReportAcceptanceWorkbookTests(unittest.TestCase):
|
class CompanyReportAcceptanceWorkbookTests(unittest.TestCase):
|
||||||
def test_real_builder_exports_reopens_and_renders_all_five_workbooks(self):
|
def test_real_builder_exports_and_reopens_all_five_workbooks(self):
|
||||||
node = node_binary()
|
|
||||||
if not node:
|
|
||||||
self.skipTest("Node.js is unavailable")
|
|
||||||
builder = ArtifactToolBuilder(
|
builder = ArtifactToolBuilder(
|
||||||
BUILDER_SCRIPT,
|
BUILDER_SCRIPT,
|
||||||
node_binary=node,
|
|
||||||
timeout_seconds=180,
|
timeout_seconds=180,
|
||||||
)
|
)
|
||||||
with tempfile.TemporaryDirectory(prefix="company-report-acceptance-") as temp_dir:
|
with tempfile.TemporaryDirectory(prefix="company-report-acceptance-") as temp_dir:
|
||||||
@@ -125,15 +100,12 @@ class CompanyReportAcceptanceWorkbookTests(unittest.TestCase):
|
|||||||
self.assertTrue(built.path.is_file())
|
self.assertTrue(built.path.is_file())
|
||||||
self.assertEqual(built.path.name, report.filename)
|
self.assertEqual(built.path.name, report.filename)
|
||||||
self.assertEqual(built.summary["formula_count"], 0)
|
self.assertEqual(built.summary["formula_count"], 0)
|
||||||
self.assertEqual(built.summary["preview_count"], 3)
|
self.assertEqual(built.summary["preview_count"], 0)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
built.summary["row_counts"],
|
built.summary["row_counts"],
|
||||||
[len(period.rows) for period in report.periods],
|
[len(period.rows) for period in report.periods],
|
||||||
)
|
)
|
||||||
self.assertEqual(
|
self.assertEqual(len(list((root / report.company / "previews").glob("*"))), 0)
|
||||||
len(list((root / report.company / "previews").glob("sheet-*.png"))),
|
|
||||||
3,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import shutil
|
|
||||||
import tempfile
|
import tempfile
|
||||||
import unittest
|
import unittest
|
||||||
from dataclasses import replace
|
from dataclasses import replace
|
||||||
@@ -22,33 +21,10 @@ from tests.test_company_reports_service import FakeRepository, synthetic_snapsho
|
|||||||
|
|
||||||
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
PROJECT_ROOT = Path(__file__).resolve().parents[1]
|
||||||
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
|
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
|
||||||
ARTIFACT_PACKAGE = (
|
|
||||||
PROJECT_ROOT
|
|
||||||
/ "company_reports"
|
|
||||||
/ "xlsx"
|
|
||||||
/ "node_modules"
|
|
||||||
/ "@oai"
|
|
||||||
/ "artifact-tool"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def artifact_tool_available() -> bool:
|
|
||||||
configured = os.environ.get("COMPANY_REPORT_ARTIFACT_TOOL_MODULE", "").strip()
|
|
||||||
return ARTIFACT_PACKAGE.exists() or bool(configured and Path(configured).is_file())
|
|
||||||
|
|
||||||
|
|
||||||
def node_binary() -> str:
|
|
||||||
configured = os.environ.get("COMPANY_REPORT_NODE_BINARY", "").strip()
|
|
||||||
return configured or shutil.which("node") or ""
|
|
||||||
|
|
||||||
|
|
||||||
@unittest.skipUnless(artifact_tool_available(), "artifact-tool dependency is not installed")
|
|
||||||
class CompanyReportIntegrationTests(unittest.TestCase):
|
class CompanyReportIntegrationTests(unittest.TestCase):
|
||||||
def test_missing_and_unmatched_group_codes_export_blank_booking_room(self):
|
def test_missing_and_unmatched_group_codes_export_blank_booking_room(self):
|
||||||
node = node_binary()
|
|
||||||
if not node:
|
|
||||||
self.skipTest("Node.js is unavailable")
|
|
||||||
|
|
||||||
class CapturingBuilder:
|
class CapturingBuilder:
|
||||||
def __init__(self, delegate):
|
def __init__(self, delegate):
|
||||||
self.delegate = delegate
|
self.delegate = delegate
|
||||||
@@ -70,7 +46,7 @@ class CompanyReportIntegrationTests(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
builder = CapturingBuilder(
|
builder = CapturingBuilder(
|
||||||
ArtifactToolBuilder(BUILDER_SCRIPT, node_binary=node, timeout_seconds=90)
|
ArtifactToolBuilder(BUILDER_SCRIPT, timeout_seconds=90)
|
||||||
)
|
)
|
||||||
publisher = AtomicReportPublisher(project_root, output_root)
|
publisher = AtomicReportPublisher(project_root, output_root)
|
||||||
service = CompanyReportService(
|
service = CompanyReportService(
|
||||||
@@ -101,10 +77,6 @@ class CompanyReportIntegrationTests(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
|
|
||||||
def test_same_snapshot_rerun_reuses_publication_when_binary_changes(self):
|
def test_same_snapshot_rerun_reuses_publication_when_binary_changes(self):
|
||||||
node = node_binary()
|
|
||||||
if not node:
|
|
||||||
self.skipTest("Node.js is unavailable")
|
|
||||||
|
|
||||||
class StableVersionRepository(FakeRepository):
|
class StableVersionRepository(FakeRepository):
|
||||||
def reserve_report(self, report):
|
def reserve_report(self, report):
|
||||||
reservation = ReservedReport(9001, 1, report.company)
|
reservation = ReservedReport(9001, 1, report.company)
|
||||||
@@ -142,7 +114,7 @@ class CompanyReportIntegrationTests(unittest.TestCase):
|
|||||||
project_root.mkdir(parents=True)
|
project_root.mkdir(parents=True)
|
||||||
repository = StableVersionRepository(synthetic_snapshot())
|
repository = StableVersionRepository(synthetic_snapshot())
|
||||||
builder = RebuildingBuilder(
|
builder = RebuildingBuilder(
|
||||||
ArtifactToolBuilder(BUILDER_SCRIPT, node_binary=node, timeout_seconds=90)
|
ArtifactToolBuilder(BUILDER_SCRIPT, timeout_seconds=90)
|
||||||
)
|
)
|
||||||
publisher = AtomicReportPublisher(project_root, output_root)
|
publisher = AtomicReportPublisher(project_root, output_root)
|
||||||
service = CompanyReportService(
|
service = CompanyReportService(
|
||||||
|
|||||||
@@ -7,9 +7,19 @@ from datetime import date
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Optional
|
from typing import Optional
|
||||||
|
|
||||||
from company_reports.contracts import BatchSnapshot, ErrorCode
|
from openpyxl import load_workbook
|
||||||
|
|
||||||
|
from company_reports.contracts import (
|
||||||
|
BatchSnapshot,
|
||||||
|
CompanyReport,
|
||||||
|
DailyVersionPin,
|
||||||
|
ErrorCode,
|
||||||
|
PeriodReport,
|
||||||
|
ReportRow,
|
||||||
|
)
|
||||||
from company_reports.core import build_company_report
|
from company_reports.core import build_company_report
|
||||||
from company_reports.publishing import (
|
from company_reports.publishing import (
|
||||||
|
ArtifactToolBuilder,
|
||||||
AtomicReportPublisher,
|
AtomicReportPublisher,
|
||||||
BuiltWorkbook,
|
BuiltWorkbook,
|
||||||
PublicationError,
|
PublicationError,
|
||||||
@@ -53,6 +63,38 @@ def empty_report():
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def workbook_report() -> CompanyReport:
|
||||||
|
row = ReportRow(
|
||||||
|
arrival=date(2026, 7, 8),
|
||||||
|
departure=date(2026, 7, 10),
|
||||||
|
nights=2,
|
||||||
|
block_code="=BLOCK-A",
|
||||||
|
res_comment="SYN-GROUP-A",
|
||||||
|
booking_room="【DBL】1",
|
||||||
|
total_booking_price="+900",
|
||||||
|
normalized_group_code="SYN-GROUP-A",
|
||||||
|
record_ids=(1,),
|
||||||
|
duplicate_group=True,
|
||||||
|
multi_price_review=True,
|
||||||
|
)
|
||||||
|
return CompanyReport(
|
||||||
|
company="QBD",
|
||||||
|
report_year=2026,
|
||||||
|
report_month=7,
|
||||||
|
as_of_date=date(2026, 7, 10),
|
||||||
|
filename="QBD-July-2026.xlsx",
|
||||||
|
periods=(
|
||||||
|
PeriodReport(1, 10, True, "QBD 01-10 Jul 2026", (row,)),
|
||||||
|
PeriodReport(11, 20, False, "QBD 11-20 Jul 2026", ()),
|
||||||
|
PeriodReport(21, 31, False, "QBD 21-31 Jul 2026", ()),
|
||||||
|
),
|
||||||
|
warnings=(),
|
||||||
|
errors=(),
|
||||||
|
daily_versions=(DailyVersionPin(date(2026, 7, 8), 100),),
|
||||||
|
booking_versions={"SYN-GROUP-A": 700},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def synthetic_built_workbook(
|
def synthetic_built_workbook(
|
||||||
work_dir: Path,
|
work_dir: Path,
|
||||||
filename: str,
|
filename: str,
|
||||||
@@ -70,6 +112,41 @@ def synthetic_built_workbook(
|
|||||||
|
|
||||||
|
|
||||||
class CompanyReportPublishingTests(unittest.TestCase):
|
class CompanyReportPublishingTests(unittest.TestCase):
|
||||||
|
def test_builder_creates_xlsx_without_private_artifact_tool_runtime(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="company-report-builder-test-") as temp_dir:
|
||||||
|
root = Path(temp_dir)
|
||||||
|
report = workbook_report()
|
||||||
|
builder = ArtifactToolBuilder(
|
||||||
|
Path("/private/artifact-tool/build_workbook.mjs"),
|
||||||
|
node_binary="/definitely/not/node",
|
||||||
|
)
|
||||||
|
|
||||||
|
built = builder.build(report, root)
|
||||||
|
|
||||||
|
self.assertTrue(built.path.is_file())
|
||||||
|
self.assertEqual(built.path.stat().st_mode & 0o777, 0o600)
|
||||||
|
self.assertEqual(built.summary["status"], "success")
|
||||||
|
self.assertEqual(built.summary["row_counts"], [1, 0, 0])
|
||||||
|
self.assertEqual(built.summary["formula_count"], 0)
|
||||||
|
self.assertEqual(len(built.summary["semantic_sha256"]), 64)
|
||||||
|
workbook = load_workbook(built.path)
|
||||||
|
self.assertEqual(workbook.sheetnames, [period.sheet_name for period in report.periods])
|
||||||
|
sheet = workbook[report.periods[0].sheet_name]
|
||||||
|
self.assertEqual(
|
||||||
|
[cell.value for cell in sheet[1]],
|
||||||
|
[
|
||||||
|
"ARRIVAL",
|
||||||
|
"DEPARTURE",
|
||||||
|
"NIGHTS",
|
||||||
|
"BLOCK_CODE",
|
||||||
|
"RES_COMMENT",
|
||||||
|
"Booking Room",
|
||||||
|
"Total Booking Price",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
self.assertEqual(sheet["D2"].value, "'=BLOCK-A")
|
||||||
|
self.assertEqual(sheet["G2"].value, "'+900")
|
||||||
|
|
||||||
def test_publish_archives_prior_current_and_activates_new_version(self):
|
def test_publish_archives_prior_current_and_activates_new_version(self):
|
||||||
with tempfile.TemporaryDirectory(prefix="company-report-publish-test-") as temp_dir:
|
with tempfile.TemporaryDirectory(prefix="company-report-publish-test-") as temp_dir:
|
||||||
project_root = Path(temp_dir) / "project"
|
project_root = Path(temp_dir) / "project"
|
||||||
|
|||||||
@@ -33,8 +33,12 @@ class DeploymentEntrypointTests(unittest.TestCase):
|
|||||||
root_requirements = (PROJECT_ROOT / "requirements.txt").read_text(
|
root_requirements = (PROJECT_ROOT / "requirements.txt").read_text(
|
||||||
encoding="utf-8"
|
encoding="utf-8"
|
||||||
)
|
)
|
||||||
|
company_requirements = (
|
||||||
|
PROJECT_ROOT / "requirements-company-reports.txt"
|
||||||
|
).read_text(encoding="utf-8")
|
||||||
|
|
||||||
self.assertIn("--enable-processing", compose)
|
self.assertIn("--enable-processing", compose)
|
||||||
|
self.assertIn("--enable-company-reports", compose)
|
||||||
self.assertNotIn("\n mcp:", compose)
|
self.assertNotIn("\n mcp:", compose)
|
||||||
self.assertNotIn("DEERFLOW", compose)
|
self.assertNotIn("DEERFLOW", compose)
|
||||||
self.assertNotIn("ARR_MCP", compose)
|
self.assertNotIn("ARR_MCP", compose)
|
||||||
@@ -47,6 +51,13 @@ class DeploymentEntrypointTests(unittest.TestCase):
|
|||||||
self.assertNotIn("basic_auth", caddy)
|
self.assertNotIn("basic_auth", caddy)
|
||||||
self.assertNotIn("requirements-agent-integration", root_requirements)
|
self.assertNotIn("requirements-agent-integration", root_requirements)
|
||||||
self.assertNotIn("requirements-arr-mcp", root_requirements)
|
self.assertNotIn("requirements-arr-mcp", root_requirements)
|
||||||
|
self.assertIn("openpyxl", company_requirements)
|
||||||
|
company_package = json.loads(
|
||||||
|
(PROJECT_ROOT / "company_reports" / "xlsx" / "package.json").read_text(
|
||||||
|
encoding="utf-8"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
self.assertNotIn("@oai/artifact-tool", json.dumps(company_package))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
Reference in New Issue
Block a user