部署优化

This commit is contained in:
andy
2026-08-03 20:59:59 +08:00
parent 5294edab89
commit 727643f1f2
18 changed files with 358 additions and 130 deletions

View File

@@ -25,7 +25,7 @@ from committed `ARRIVAL` facts, publishes a validated workbook, and records meta
| `arr_web.job_trace` | Read-only programmatic trace from persisted run/attempt/delivery/version/outbox facts | | `arr_web.job_trace` | Read-only programmatic trace from persisted run/attempt/delivery/version/outbox facts |
| `monthly_reports.worker` | Dedicated leased outbox consumer with retry/dead-letter behavior | | `monthly_reports.worker` | Dedicated leased outbox consumer with retry/dead-letter behavior |
| `monthly_reports.repository` / `reporting` schema | ARRIVAL-derived snapshots, report versions, lineage, manifests and artifact identities | | `monthly_reports.repository` / `reporting` schema | ARRIVAL-derived snapshots, report versions, lineage, manifests and artifact identities |
| `monthly_reports`, `channel_analytics`, `company_reports` | Deterministic downstream consumers of committed facts | | `monthly_reports`, `channel_analytics`, `company_reports` | Deterministic downstream consumers of committed facts; company reports build official XLSX files in Python/openpyxl inside Web |
| Web + Caddy Compose | Web-owned login/session boundary behind Caddy HTTPS; no browser-native Basic Auth and no MCP service | | Web + Caddy Compose | Web-owned login/session boundary behind Caddy HTTPS; no browser-native Basic Auth and no MCP service |
## Important Boundaries ## Important Boundaries
@@ -52,4 +52,4 @@ from committed `ARRIVAL` facts, publishes a validated workbook, and records meta
## Last Updated ## Last Updated
2026-07-31 2026-08-03

View File

@@ -13,8 +13,19 @@ blocked. A not-yet-ended period keeps its fixed C/O cutoff and must be rerun aft
workbook needs those facts. Duplicate publication of the same semantic snapshot is idempotent: the first validated workbook needs those facts. Duplicate publication of the same semantic snapshot is idempotent: the first validated
archive/result pair remains authoritative even if a retry rebuilds different XLSX bytes. archive/result pair remains authoritative even if a retry rebuilds different XLSX bytes.
Company-channel XLSX generation is now Web-container deployable without the private `@oai/artifact-tool` npm package:
`company_reports.publishing.ArtifactToolBuilder` builds and reopens workbooks with Python/openpyxl, validates sheet
names/headers/row counts/no-formulas, and treats blank Excel cells as business-empty strings. Node/artifact-tool remains
only a monthly-worker packaging concern.
## Completed On 2026-08-03 ## Completed On 2026-08-03
- Removed the private npm dependency from the company-channel report deployment path after CentOS Docker build failed
with `@oai/artifact-tool` 404. The company workbook builder now uses Python/openpyxl inside Web, keeps formula-injection
text escaping, produces no preview PNGs, and validates regenerated workbooks after saving. `compose.yaml` starts Web
with `--enable-company-reports`; HTTP/IP test overrides must keep that flag while omitting `--secure-cookies`.
Focused company-report and deployment-entrypoint tests pass 19/19. No database migration or live server deployment was
performed from this workstation.
- Made the mobile H5 dashboard publicly readable without weakening the desktop/operator boundary. Anonymous users can - Made the mobile H5 dashboard publicly readable without weakening the desktop/operator boundary. Anonymous users can
load `/h5`, its H5 assets and purpose-built `/api/public/h5/months` plus `/api/public/h5/analytics` aggregate routes; load `/h5`, its H5 assets and purpose-built `/api/public/h5/months` plus `/api/public/h5/analytics` aggregate routes;
the public projection omits source hashes and operational metadata. Generic analytics, legacy H5 APIs, desktop pages, the public projection omits source hashes and operational metadata. Generic analytics, legacy H5 APIs, desktop pages,
@@ -439,9 +450,10 @@ archive/result pair remains authoritative even if a retry rebuilds different XLS
detached and are not reboot-persistent. detached and are not reboot-persistent.
- The latest operator-selected Web credentials were rotated on 2026-07-31, but the password still matches the username. - The latest operator-selected Web credentials were rotated on 2026-07-31, but the password still matches the username.
Rotate it again to a distinct high-entropy value in Keychain, followed by one controlled Web restart. Rotate it again to a distinct high-entropy value in Keychain, followed by one controlled Web restart.
- The workstation runs Web and worker as separate processes. The checked-in Compose image intentionally does not claim - The workstation runs Web and worker as separate processes. The checked-in Compose image now supports company-channel
to run the worker because it does not package the workstation-only artifact-tool module; a production image/process XLSX generation through Python/openpyxl in Web, but intentionally does not claim to run the monthly worker because it
manager must supply Node/artifact-tool and the shared output volume before enabling that service. does not package the workstation-only monthly artifact-tool module; a production image/process manager must supply
Node/artifact-tool and the shared output volume before enabling that monthly service.
- A formally controlled no-PII server acceptance run remains appropriate after that deployment packaging is complete. - A formally controlled no-PII server acceptance run remains appropriate after that deployment packaging is complete.
- ARR2.0 now has Git metadata; `main` tracks `origin/main`. Runtime credential values remain outside Git and project - ARR2.0 now has Git metadata; `main` tracks `origin/main`. Runtime credential values remain outside Git and project
files. files.

View File

@@ -4,6 +4,7 @@
| Date | Task | Outcome | Docs Updated | | Date | Task | Outcome | Docs Updated |
|---|---|---|---| |---|---|---|---|
| 2026-08-03 | Make company-report XLSX generation deploy without private npm | Replaced the company-report builder's Node/private `@oai/artifact-tool` runtime dependency with Python/openpyxl workbook generation and self-validation, removed the private package dependency, enabled `--enable-company-reports` in Compose Web, and documented that only the monthly worker still needs Node/artifact-tool packaging. Focused company-report and deployment-entrypoint tests pass 19/19; no migration or live deployment was performed | Current state/history, architecture, deployment evidence/index, stale item, runbooks |
| 2026-08-03 | Execute controlled fix and live-accept 2026-08 `01-10` company report | With explicit confirmation, stopped only the stale PID 11176 and started new listener PID 54127 through the existing Keychain-backed launcher. One authorized job `05cc547d…` succeeded 5/5 with row counts `54/18/7/1/15`, captured active Booking batch-7 source metadata, reused the existing version/artifact identities, and passed five HTTP download/hash checks. Logout and temporary-file cleanup completed; no Booking/Finance source or fact mutation occurred | Current state, runtime evidence/index, stale item, commitments, scoped planning record | | 2026-08-03 | Execute controlled fix and live-accept 2026-08 `01-10` company report | With explicit confirmation, stopped only the stale PID 11176 and started new listener PID 54127 through the existing Keychain-backed launcher. One authorized job `05cc547d…` succeeded 5/5 with row counts `54/18/7/1/15`, captured active Booking batch-7 source metadata, reused the existing version/artifact identities, and passed five HTTP download/hash checks. Logout and temporary-file cleanup completed; no Booking/Finance source or fact mutation occurred | Current state, runtime evidence/index, stale item, commitments, scoped planning record |
| 2026-08-03 | Diagnose the latest 2026-08 `01-10` official-Excel save failure | Read-only evidence confirms newest job `087dceca...` built all five companies (`54/18/7/1/15`) and failed only at publish. Port 8766 is still PID 11176 from 2026-08-02, older than the semantic-reuse publisher fix; the earlier successful five-file archive/result/current set remains hash-consistent and intact. The launcher preflight is ready and six publisher tests pass. Repair is an exact controlled listener replacement followed by one authorized rerun; no restart, report write or business-data mutation was performed | Current state, runtime evidence/index, stale item, scoped planning record | | 2026-08-03 | Diagnose the latest 2026-08 `01-10` official-Excel save failure | Read-only evidence confirms newest job `087dceca...` built all five companies (`54/18/7/1/15`) and failed only at publish. Port 8766 is still PID 11176 from 2026-08-02, older than the semantic-reuse publisher fix; the earlier successful five-file archive/result/current set remains hash-consistent and intact. The launcher preflight is ready and six publisher tests pass. Repair is an exact controlled listener replacement followed by one authorized rerun; no restart, report write or business-data mutation was performed | Current state, runtime evidence/index, stale item, scoped planning record |
| 2026-08-03 | Check whether company-report retries show reused-file identity | Confirmed the publisher can reuse an existing archive/result pair, but the outcome, job API and frontend expose only success/version/download; users cannot see that the file was reused or which prior artifact supplied it. No code or business-data change was made | Current state, retry-semantics evidence/index | | 2026-08-03 | Check whether company-report retries show reused-file identity | Confirmed the publisher can reuse an existing archive/result pair, but the outcome, job API and frontend expose only success/version/download; users cannot see that the file was reused or which prior artifact supplied it. No code or business-data change was made | Current state, retry-semantics evidence/index |

View File

@@ -4,6 +4,7 @@ Use this index for searchable, traceable evidence records.
| Date | Topic | Status | Source | Detail | | Date | Topic | Status | Source | Detail |
|---|---|---|---|---| |---|---|---|---|---|
| 2026-08-03 | Company-report openpyxl builder for deployment | Implemented; focused deployment checks pass | [Evidence topic](topics/2026-08-03-company-report-openpyxl-builder.md) | CentOS Docker build failed because public npm cannot install private `@oai/artifact-tool`. Company-report XLSX generation now runs through Python/openpyxl in Web, while monthly worker packaging remains the only Node/artifact-tool concern. Focused company-report/deployment tests pass 19/19. |
| 2026-08-03 | Booking source and company-report retry semantics | Source contract confirmed; reuse visibility gap identified; live acceptance pending restart | [Evidence topic](topics/2026-08-03-company-report-retry-semantics.md) | Byte-identical XLSX uploads reuse the activated source; same rows with different file bytes are new drafts. Same report snapshots reuse complete publication pairs, changed Finance/Booking pins create new versions, and an activated subset changes Booking detail coverage while retaining Finance rows. The current API/UI do not expose whether a retry reused an existing artifact. | | 2026-08-03 | Booking source and company-report retry semantics | Source contract confirmed; reuse visibility gap identified; live acceptance pending restart | [Evidence topic](topics/2026-08-03-company-report-retry-semantics.md) | Byte-identical XLSX uploads reuse the activated source; same rows with different file bytes are new drafts. Same report snapshots reuse complete publication pairs, changed Finance/Booking pins create new versions, and an activated subset changes Booking detail coverage while retaining Finance rows. The current API/UI do not expose whether a retry reused an existing artifact. |
| 2026-08-03 | Company-report retry after reported 8766 restart | Resolved by controlled listener replacement and live 5/5 rerun | [Evidence topic](topics/2026-08-03-company-source-runtime-diagnosis.md) | PID 11176 was replaced by PID 54127. Job `05cc547d…` succeeded all five August `01-10` companies, reused the prior version/artifact identities, and all five authenticated downloads returned hash-matching 200 responses. No Booking/Finance source or fact changed. | | 2026-08-03 | Company-report retry after reported 8766 restart | Resolved by controlled listener replacement and live 5/5 rerun | [Evidence topic](topics/2026-08-03-company-source-runtime-diagnosis.md) | PID 11176 was replaced by PID 54127. Job `05cc547d…` succeeded all five August `01-10` companies, reused the prior version/artifact identities, and all five authenticated downloads returned hash-matching 200 responses. No Booking/Finance source or fact changed. |
| 2026-08-03 | Remove redundant company-report Excel provenance module | Implemented; Web/company checks pass | [Evidence topic](topics/2026-08-03-company-report-status-copy-fix.md) | The job-detail source card and history source column are removed because reports consume the Booking database snapshot; upload/review source UI and server-side provenance remain available. | | 2026-08-03 | Remove redundant company-report Excel provenance module | Implemented; Web/company checks pass | [Evidence topic](topics/2026-08-03-company-report-status-copy-fix.md) | The job-detail source card and history source column are removed because reports consume the Booking database snapshot; upload/review source UI and server-side provenance remain available. |

View File

@@ -0,0 +1,33 @@
# Company-Report Openpyxl Builder
Date: 2026-08-03
## Finding
The CentOS test-server Docker build failed at `npm --prefix company_reports/xlsx install` because
`@oai/artifact-tool@2.8.33` is not available from the public npm registry. This is a packaging/runtime dependency
failure, not a database, migration or Docker memory issue.
## Change
- `company_reports.publishing.ArtifactToolBuilder` now builds official company-channel workbooks with Python/openpyxl.
- The builder writes the three expected sheets, preserves the existing headers/styles, escapes text that could be
interpreted as formulas, reopens the saved workbook and validates sheet names, dimensions, headers, row values and
zero formula cells.
- Blank cells read back from Excel as `None` are normalized to the business-empty string during validation.
- `company_reports/xlsx/package.json` no longer declares the private `@oai/artifact-tool` dependency.
- `compose.yaml` enables `--enable-company-reports` for Web. IP/HTTP test overrides should keep that flag but omit
`--secure-cookies`.
## Verification
- Watched the new builder test fail against the old Node/private-runtime implementation with
`BuildError('the XLSX builder runtime is unavailable')`.
- Focused verification passed:
`python -m unittest tests.test_company_reports_publishing tests.test_company_reports_integration tests.test_company_reports_acceptance tests.test_company_reports_service tests.test_deployment_entrypoints`
with 19 tests passing.
## Remaining Boundaries
The monthly worker still uses its separate Node/artifact-tool workbook builder. This change makes company-channel detail
Excel generation deployable in the Web image; it does not package or supervise the monthly worker.

View File

@@ -16,7 +16,7 @@
| 2026-07-30 | `booking_test` current Finance projection | The earlier 417-row snapshot included accepted run `mvp-v1-fixture-20260727` from `synthetic.xml`/`local_fixture`. A 2026-07-31 14:17 read-only company-report recheck found 986 current supported-company facts, so the old 416/417 remediation target is no longer a complete description of current Finance state | Re-audit current daily-version pins and source provenance before any fixture retirement; require explicit authorization for version changes, then verify the clean target and watermark | | 2026-07-30 | `booking_test` current Finance projection | The earlier 417-row snapshot included accepted run `mvp-v1-fixture-20260727` from `synthetic.xml`/`local_fixture`. A 2026-07-31 14:17 read-only company-report recheck found 986 current supported-company facts, so the old 416/417 remediation target is no longer a complete description of current Finance state | Re-audit current daily-version pins and source provenance before any fixture retirement; require explicit authorization for version changes, then verify the clean target and watermark |
| 2026-07-30 | Channel BI refresh lifecycle | Resolved 2026-08-03. Desktop and public H5 now check selected-month metadata every five seconds while visible and reload full analytics only after `updated_at` changes; hidden views pause and transient failures preserve the last good snapshot | Refresh the browser once to load the new static assets, then observe the next authorized publication as a live acceptance check | | 2026-07-30 | Channel BI refresh lifecycle | Resolved 2026-08-03. Desktop and public H5 now check selected-month metadata every five seconds while visible and reload full analytics only after `updated_at` changes; hidden views pause and transient failures preserve the last good snapshot | Refresh the browser once to load the new static assets, then observe the next authorized publication as a live acceptance check |
| 2026-07-30 | Channel BI KPI label | The card labeled `公司数` renders worksheet-level `channel_count`; LianTai GROUP/FIT are two channels, so six does not mean six companies | Decide whether to relabel it `渠道/子表数` or implement an explicit five-company aggregation | | 2026-07-30 | Channel BI KPI label | The card labeled `公司数` renders worksheet-level `channel_count`; LianTai GROUP/FIT are two channels, so six does not mean six companies | Decide whether to relabel it `渠道/子表数` or implement an explicit five-company aggregation |
| 2026-07-30 | Production worker packaging | Local Web/worker separation is live, but the checked-in Compose image does not contain the workstation-only Node/artifact-tool runtime required by the XLSX builder | Package an approved builder runtime and shared output volume before adding/enabling the managed production worker service | | 2026-07-30 | Production monthly-worker packaging | Local Web/worker separation is live, and company-report XLSX generation now runs in Web through Python/openpyxl. The checked-in Compose image still does not contain the workstation-only Node/artifact-tool runtime required by the monthly workbook builder | Package an approved monthly builder runtime and shared output volume before adding/enabling the managed production monthly-worker service |
## Superseded For ARR2.0 ## Superseded For ARR2.0

View File

@@ -47,9 +47,7 @@ cp .env.example .env.local
--port 8765 \ --port 8765 \
--enable-processing \ --enable-processing \
--enable-monthly-generation \ --enable-monthly-generation \
--enable-company-reports \ --enable-company-reports
--node-binary /absolute/path/to/node \
--artifact-tool-module /absolute/path/to/artifact_tool.mjs
``` ```
另开一个进程启动月报消费者: 另开一个进程启动月报消费者:
@@ -91,8 +89,9 @@ worker 是独立无端口进程,应由进程管理器单独保活。
## 部署 ## 部署
生产配置见 [deploy/README.md](deploy/README.md)。当前 Compose 模板仍只打包 `web``caddy`Caddy 负责 生产配置见 [deploy/README.md](deploy/README.md)。当前 Compose 模板仍只打包 `web``caddy`Caddy 负责
HTTPSWeb 负责应用登录和会话;没有 MCP 端口、MCP 域名或 Agent Secret。月报 worker 必须作为独立进程部署, HTTPSWeb 负责应用登录和会话;没有 MCP 端口、MCP 域名或 Agent Secret。公司渠道明细由 Web 进程使用
并使用同一数据库、共享输出卷以及已经打包 Node/artifact-tool 的运行镜像;当前本地工作站已按这一方式运行。 Python/openpyxl 生成正式 Excel不需要 Node/artifact-tool。月报 worker 必须作为独立进程部署,并使用同一
数据库、共享输出卷以及已经打包 Node/artifact-tool 的运行镜像;当前本地工作站已按这一方式运行。
## 当前月报行为 ## 当前月报行为

View File

@@ -22,14 +22,13 @@ Supported companies: `LianTai`, `QBD`, `DY-AI-Easy-KB`, `FengRun`, `HanaTour`.
python3 -m pip install -r requirements-company-reports.txt python3 -m pip install -r requirements-company-reports.txt
export COMPANY_REPORT_DATABASE_URL='postgresql://USER:PASSWORD@HOST:PORT/booking_test' export COMPANY_REPORT_DATABASE_URL='postgresql://USER:PASSWORD@HOST:PORT/booking_test'
export COMPANY_REPORT_NODE_BINARY='/absolute/path/to/node'
export COMPANY_REPORT_ARTIFACT_TOOL_MODULE='/absolute/path/to/artifact_tool.mjs'
python3 -m company_reports generate \ python3 -m company_reports generate \
--month 2026-07 \ --month 2026-07 \
--as-of 2026-07-10 --as-of 2026-07-10
``` ```
The workbook builder uses Python/openpyxl. No Node.js or `@oai/artifact-tool` package is required for company reports.
The DSN can fall back to `ARR_DATABASE_URL`; never place a real DSN in source, prompts, output JSON or browser code. The DSN can fall back to `ARR_DATABASE_URL`; never place a real DSN in source, prompts, output JSON or browser code.
Scheduling convention: Scheduling convention:

View File

@@ -71,11 +71,11 @@ def _parser() -> SafeArgumentParser:
) )
generate.add_argument( generate.add_argument(
"--node-binary", "--node-binary",
help="Node.js executable; defaults to COMPANY_REPORT_NODE_BINARY or PATH", help="deprecated compatibility option; ignored by the openpyxl builder",
) )
generate.add_argument( generate.add_argument(
"--artifact-tool-module", "--artifact-tool-module",
help="absolute path to artifact_tool.mjs when package resolution is unavailable", help="deprecated compatibility option; ignored by the openpyxl builder",
) )
return parser return parser

View File

@@ -1,4 +1,4 @@
"""Artifact-tool build adapter and recoverable atomic publication.""" """Workbook build adapter and recoverable atomic publication."""
from __future__ import annotations from __future__ import annotations
@@ -6,13 +6,17 @@ import hashlib
import json import json
import os import os
import shutil import shutil
import subprocess
import tempfile import tempfile
import uuid import uuid
from dataclasses import dataclass from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path from pathlib import Path
from typing import Any, Dict, Mapping, Optional, Tuple from typing import Any, Dict, Mapping, Optional, Tuple
from openpyxl import Workbook, load_workbook
from openpyxl.styles import Alignment, Border, Font, PatternFill, Side
from openpyxl.worksheet.worksheet import Worksheet
from company_reports.contracts import CompanyReport, ErrorCode, RESULT_SCHEMA_VERSION from company_reports.contracts import CompanyReport, ErrorCode, RESULT_SCHEMA_VERSION
from company_reports.repository import ( from company_reports.repository import (
FileMetadata, FileMetadata,
@@ -23,6 +27,21 @@ from company_reports.repository import (
XLSX_MIME = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" XLSX_MIME = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
REPORT_HEADERS = (
"ARRIVAL",
"DEPARTURE",
"NIGHTS",
"BLOCK_CODE",
"RES_COMMENT",
"Booking Room",
"Total Booking Price",
)
BODY_FONT = "222222"
DUPLICATE_FILL = "FFF2CC"
DUPLICATE_FONT = "9C6500"
REVIEW_FILL = "FCE4D6"
REVIEW_FONT = "C65911"
HEADER_FILL = "FFFFFF"
class BuildError(RuntimeError): class BuildError(RuntimeError):
@@ -89,63 +108,196 @@ class ArtifactToolBuilder:
artifact_tool_module: Optional[Path] = None, artifact_tool_module: Optional[Path] = None,
timeout_seconds: int = 120, timeout_seconds: int = 120,
) -> None: ) -> None:
configured = (node_binary or os.environ.get("COMPANY_REPORT_NODE_BINARY", "")).strip() _ = (node_binary, artifact_tool_module, timeout_seconds)
self._node_binary = configured or shutil.which("node") or ""
self._builder_script = builder_script.resolve() self._builder_script = builder_script.resolve()
self._artifact_tool_module = (
artifact_tool_module.resolve() if artifact_tool_module else None @staticmethod
def _safe_text(value: Any) -> str:
text = str(value or "")
return "'" + text if text[:1] in {"=", "+", "-", "@"} else text
@staticmethod
def _date_key(value: Any) -> str:
if isinstance(value, datetime):
return value.date().isoformat()
if isinstance(value, date):
return value.isoformat()
if isinstance(value, str) and len(value) >= 10:
return value[:10]
return ""
@staticmethod
def _cell_text(value: Any) -> str:
return "" if value is None else str(value)
@staticmethod
def _semantic_sha256(payload: Mapping[str, Any]) -> str:
encoded = json.dumps(
{
"headers": payload["headers"],
"periods": payload["periods"],
},
ensure_ascii=False,
separators=(",", ":"),
).encode("utf-8")
return hashlib.sha256(encoded).hexdigest()
def _write_sheet(
self,
worksheet: Worksheet,
period: Any,
) -> None:
worksheet.append(list(REPORT_HEADERS))
for row in period.rows:
worksheet.append(
[
row.arrival,
row.departure,
row.nights,
self._safe_text(row.block_code),
self._safe_text(row.res_comment),
self._safe_text(row.booking_room),
self._safe_text(row.total_booking_price),
]
)
worksheet.column_dimensions["A"].width = 13
worksheet.column_dimensions["B"].width = 13
worksheet.column_dimensions["C"].width = 9
worksheet.column_dimensions["D"].width = 20
worksheet.column_dimensions["E"].width = 25
worksheet.column_dimensions["F"].width = 34
worksheet.column_dimensions["G"].width = 48
worksheet.freeze_panes = "A2"
worksheet.auto_filter.ref = f"A1:G{max(1, worksheet.max_row)}"
thin_bottom = Border(
bottom=Side(style="thin", color="7F7F7F"),
) )
self._timeout_seconds = timeout_seconds header_fill = PatternFill("solid", fgColor=HEADER_FILL)
duplicate_fill = PatternFill("solid", fgColor=DUPLICATE_FILL)
review_fill = PatternFill("solid", fgColor=REVIEW_FILL)
for cell in worksheet[1]:
cell.fill = header_fill
cell.font = Font(name="Arial", size=10, bold=True, color="000000")
cell.alignment = Alignment(horizontal="center", vertical="center")
cell.border = thin_bottom
worksheet.row_dimensions[1].height = 24
for row_index in range(2, worksheet.max_row + 1):
worksheet.row_dimensions[row_index].height = 30
for cell in worksheet[row_index]:
cell.font = Font(name="Arial", size=10, color=BODY_FONT)
cell.alignment = Alignment(vertical="center")
for column in ("A", "B", "C", "D"):
worksheet[f"{column}{row_index}"].alignment = Alignment(
horizontal="center",
vertical="center",
)
for column in ("E", "F", "G"):
worksheet[f"{column}{row_index}"].alignment = Alignment(
horizontal="left",
vertical="center",
wrap_text=True,
)
worksheet[f"A{row_index}"].number_format = "yyyy-mm-dd"
worksheet[f"B{row_index}"].number_format = "yyyy-mm-dd"
for offset, source_row in enumerate(period.rows, start=2):
if source_row.duplicate_group:
for column in range(1, 8):
worksheet.cell(offset, column).fill = duplicate_fill
worksheet[f"E{offset}"].font = Font(
name="Arial",
size=10,
bold=True,
color=DUPLICATE_FONT,
)
if source_row.multi_price_review:
worksheet[f"G{offset}"].fill = review_fill
worksheet[f"G{offset}"].font = Font(
name="Arial",
size=10,
bold=True,
color=REVIEW_FONT,
)
def _validate_workbook(self, output_path: Path, report: CompanyReport) -> int:
workbook = load_workbook(output_path, data_only=False)
expected_names = [period.sheet_name for period in report.periods]
if workbook.sheetnames != expected_names:
raise ValueError("worksheet names do not match")
formula_count = 0
for period in report.periods:
worksheet = workbook[period.sheet_name]
expected_rows = len(period.rows) + 1
if worksheet.max_row != expected_rows or worksheet.max_column != 7:
raise ValueError("worksheet dimensions do not match")
headers = [worksheet.cell(1, column).value for column in range(1, 8)]
if headers != list(REPORT_HEADERS):
raise ValueError("worksheet headers do not match")
for row_index, row in enumerate(period.rows, start=2):
expected = [
row.arrival.isoformat(),
row.departure.isoformat(),
row.nights,
self._safe_text(row.block_code),
self._safe_text(row.res_comment),
self._safe_text(row.booking_room),
self._safe_text(row.total_booking_price),
]
actual = [
self._date_key(worksheet.cell(row_index, 1).value),
self._date_key(worksheet.cell(row_index, 2).value),
worksheet.cell(row_index, 3).value,
self._cell_text(worksheet.cell(row_index, 4).value),
self._cell_text(worksheet.cell(row_index, 5).value),
self._cell_text(worksheet.cell(row_index, 6).value),
self._cell_text(worksheet.cell(row_index, 7).value),
]
if actual != expected:
raise ValueError("worksheet values do not match")
for cells in worksheet.iter_rows():
for cell in cells:
if cell.data_type == "f":
formula_count += 1
if formula_count != 0:
raise ValueError("workbook must contain no formulas")
return formula_count
def build(self, report: CompanyReport, work_dir: Path) -> BuiltWorkbook: def build(self, report: CompanyReport, work_dir: Path) -> BuiltWorkbook:
if not self._node_binary or not self._builder_script.is_file():
raise BuildError(
ErrorCode.OUTPUT_VALIDATION_FAILED,
"the XLSX builder runtime is unavailable",
)
work_dir.mkdir(parents=True, exist_ok=True, mode=0o700) work_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
os.chmod(work_dir, 0o700) os.chmod(work_dir, 0o700)
payload_path = work_dir / "workbook-payload.json" payload_path = work_dir / "workbook-payload.json"
output_path = work_dir / report.filename output_path = work_dir / report.filename
preview_dir = work_dir / "previews" preview_dir = work_dir / "previews"
summary_path = work_dir / "workbook-summary.json" summary_path = work_dir / "workbook-summary.json"
_private_json(payload_path, report.to_workbook_payload())
try: try:
environment = os.environ.copy() payload = report.to_workbook_payload()
if self._artifact_tool_module is not None: _private_json(payload_path, payload)
environment["COMPANY_REPORT_ARTIFACT_TOOL_MODULE"] = str( preview_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
self._artifact_tool_module workbook = Workbook()
) for index, period in enumerate(report.periods):
completed = subprocess.run( worksheet = workbook.active if index == 0 else workbook.create_sheet()
[ worksheet.title = period.sheet_name
self._node_binary, self._write_sheet(worksheet, period)
str(self._builder_script), workbook.save(output_path)
str(payload_path), os.chmod(output_path, 0o600)
str(output_path), formula_count = self._validate_workbook(output_path, report)
str(preview_dir),
str(summary_path),
],
cwd=self._builder_script.parent,
capture_output=True,
text=True,
timeout=self._timeout_seconds,
env=environment,
check=False,
)
except (OSError, subprocess.TimeoutExpired):
raise BuildError(
ErrorCode.OUTPUT_VALIDATION_FAILED,
"the XLSX builder did not complete",
) from None
if completed.returncode != 0:
raise BuildError(
ErrorCode.OUTPUT_VALIDATION_FAILED,
"the XLSX builder rejected the generated payload",
)
try:
summary = json.loads(summary_path.read_text(encoding="utf-8"))
expected_rows = [len(period.rows) for period in report.periods] expected_rows = [len(period.rows) for period in report.periods]
expected_names = [period.sheet_name for period in report.periods] expected_names = [period.sheet_name for period in report.periods]
summary = {
"status": "success",
"schema_version": payload["schema_version"],
"company": report.company,
"filename": report.filename,
"sheet_names": expected_names,
"row_counts": expected_rows,
"formula_count": formula_count,
"semantic_sha256": self._semantic_sha256(payload),
"preview_count": 0,
"byte_size": output_path.stat().st_size,
}
if ( if (
summary.get("status") != "success" summary.get("status") != "success"
or summary.get("company") != report.company or summary.get("company") != report.company
@@ -155,17 +307,16 @@ class ArtifactToolBuilder:
or summary.get("formula_count") != 0 or summary.get("formula_count") != 0
or not isinstance(summary.get("semantic_sha256"), str) or not isinstance(summary.get("semantic_sha256"), str)
or len(summary.get("semantic_sha256")) != 64 or len(summary.get("semantic_sha256")) != 64
or summary.get("preview_count") != 3
or not output_path.is_file() or not output_path.is_file()
or output_path.stat().st_size <= 0 or output_path.stat().st_size <= 0
): ):
raise ValueError("summary mismatch") raise ValueError("summary mismatch")
_private_json(summary_path, summary)
except (OSError, ValueError, TypeError, json.JSONDecodeError): except (OSError, ValueError, TypeError, json.JSONDecodeError):
raise BuildError( raise BuildError(
ErrorCode.OUTPUT_VALIDATION_FAILED, ErrorCode.OUTPUT_VALIDATION_FAILED,
"the XLSX builder result could not be validated", "the XLSX builder result could not be validated",
) from None ) from None
os.chmod(output_path, 0o600)
return BuiltWorkbook( return BuiltWorkbook(
path=output_path, path=output_path,
sha256=sha256_file(output_path), sha256=sha256_file(output_path),

View File

@@ -1,8 +1,5 @@
{ {
"name": "company-report-xlsx-builder", "name": "company-report-xlsx-builder",
"private": true, "private": true,
"type": "module", "type": "module"
"dependencies": {
"@oai/artifact-tool": "2.8.33"
}
} }

View File

@@ -16,6 +16,7 @@ services:
- --port - --port
- "8765" - "8765"
- --enable-processing - --enable-processing
- --enable-company-reports
- --secure-cookies - --secure-cookies
environment: environment:
ARR_WEB_USERNAME: "${ARR_WEB_USERNAME:?set ARR_WEB_USERNAME in deploy/.env.production}" ARR_WEB_USERNAME: "${ARR_WEB_USERNAME:?set ARR_WEB_USERNAME in deploy/.env.production}"

View File

@@ -8,7 +8,8 @@
- PostgreSQL 15+:目标必须是隔离数据库 `booking_test`,并已应用 008012 权威迁移; - PostgreSQL 15+:目标必须是隔离数据库 `booking_test`,并已应用 008012 权威迁移;
- OSS区域匹配、服务端加密、未启用或暂停 versioning、禁止匿名写bucket ACL 可为 private 或 public-read - OSS区域匹配、服务端加密、未启用或暂停 versioning、禁止匿名写bucket ACL 可为 private 或 public-read
- ARR 数据库账号和 OSS RAM/STS 凭据由 Secret 管理器注入,不写入镜像或仓库。 - ARR 数据库账号和 OSS RAM/STS 凭据由 Secret 管理器注入,不写入镜像或仓库。
- worker 与 Web 使用同一数据库和共享的 `/app/outputs` 持久卷worker 镜像还必须包含 Node.js 与配置匹配的 artifact-tool 模块 - 公司渠道明细由 Web 进程使用 Python/openpyxl 生成正式 Excel不需要 Node.js 或 `@oai/artifact-tool`
- worker 与 Web 使用同一数据库和共享的 `/app/outputs` 持久卷;月报 worker 镜像还必须包含 Node.js 与配置匹配的 artifact-tool 模块。
## 配置 ## 配置

View File

@@ -1 +1,2 @@
psycopg[binary]==3.2.13 psycopg[binary]==3.2.13
openpyxl==3.1.5

View File

@@ -1,7 +1,5 @@
from __future__ import annotations from __future__ import annotations
import os
import shutil
import tempfile import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
@@ -19,14 +17,6 @@ from tests.company_reports_acceptance_fixture import (
PROJECT_ROOT = Path(__file__).resolve().parents[1] PROJECT_ROOT = Path(__file__).resolve().parents[1]
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs" BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
ARTIFACT_PACKAGE = (
PROJECT_ROOT
/ "company_reports"
/ "xlsx"
/ "node_modules"
/ "@oai"
/ "artifact-tool"
)
def acceptance_reports(): def acceptance_reports():
@@ -38,16 +28,6 @@ def acceptance_reports():
) )
def artifact_tool_available() -> bool:
configured = os.environ.get("COMPANY_REPORT_ARTIFACT_TOOL_MODULE", "").strip()
return ARTIFACT_PACKAGE.exists() or bool(configured and Path(configured).is_file())
def node_binary() -> str:
configured = os.environ.get("COMPANY_REPORT_NODE_BINARY", "").strip()
return configured or shutil.which("node") or ""
class CompanyReportAcceptanceCoreTests(unittest.TestCase): class CompanyReportAcceptanceCoreTests(unittest.TestCase):
def test_all_five_companies_have_contract_complete_month_end_reports(self): def test_all_five_companies_have_contract_complete_month_end_reports(self):
reports = acceptance_reports() reports = acceptance_reports()
@@ -106,15 +86,10 @@ class CompanyReportAcceptanceCoreTests(unittest.TestCase):
self.assertEqual(dy_row.departure.day, 20) self.assertEqual(dy_row.departure.day, 20)
@unittest.skipUnless(artifact_tool_available(), "artifact-tool dependency is not installed")
class CompanyReportAcceptanceWorkbookTests(unittest.TestCase): class CompanyReportAcceptanceWorkbookTests(unittest.TestCase):
def test_real_builder_exports_reopens_and_renders_all_five_workbooks(self): def test_real_builder_exports_and_reopens_all_five_workbooks(self):
node = node_binary()
if not node:
self.skipTest("Node.js is unavailable")
builder = ArtifactToolBuilder( builder = ArtifactToolBuilder(
BUILDER_SCRIPT, BUILDER_SCRIPT,
node_binary=node,
timeout_seconds=180, timeout_seconds=180,
) )
with tempfile.TemporaryDirectory(prefix="company-report-acceptance-") as temp_dir: with tempfile.TemporaryDirectory(prefix="company-report-acceptance-") as temp_dir:
@@ -125,15 +100,12 @@ class CompanyReportAcceptanceWorkbookTests(unittest.TestCase):
self.assertTrue(built.path.is_file()) self.assertTrue(built.path.is_file())
self.assertEqual(built.path.name, report.filename) self.assertEqual(built.path.name, report.filename)
self.assertEqual(built.summary["formula_count"], 0) self.assertEqual(built.summary["formula_count"], 0)
self.assertEqual(built.summary["preview_count"], 3) self.assertEqual(built.summary["preview_count"], 0)
self.assertEqual( self.assertEqual(
built.summary["row_counts"], built.summary["row_counts"],
[len(period.rows) for period in report.periods], [len(period.rows) for period in report.periods],
) )
self.assertEqual( self.assertEqual(len(list((root / report.company / "previews").glob("*"))), 0)
len(list((root / report.company / "previews").glob("sheet-*.png"))),
3,
)
if __name__ == "__main__": if __name__ == "__main__":

View File

@@ -2,7 +2,6 @@ from __future__ import annotations
import json import json
import os import os
import shutil
import tempfile import tempfile
import unittest import unittest
from dataclasses import replace from dataclasses import replace
@@ -22,33 +21,10 @@ from tests.test_company_reports_service import FakeRepository, synthetic_snapsho
PROJECT_ROOT = Path(__file__).resolve().parents[1] PROJECT_ROOT = Path(__file__).resolve().parents[1]
BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs" BUILDER_SCRIPT = PROJECT_ROOT / "company_reports" / "xlsx" / "build_workbook.mjs"
ARTIFACT_PACKAGE = (
PROJECT_ROOT
/ "company_reports"
/ "xlsx"
/ "node_modules"
/ "@oai"
/ "artifact-tool"
)
def artifact_tool_available() -> bool:
configured = os.environ.get("COMPANY_REPORT_ARTIFACT_TOOL_MODULE", "").strip()
return ARTIFACT_PACKAGE.exists() or bool(configured and Path(configured).is_file())
def node_binary() -> str:
configured = os.environ.get("COMPANY_REPORT_NODE_BINARY", "").strip()
return configured or shutil.which("node") or ""
@unittest.skipUnless(artifact_tool_available(), "artifact-tool dependency is not installed")
class CompanyReportIntegrationTests(unittest.TestCase): class CompanyReportIntegrationTests(unittest.TestCase):
def test_missing_and_unmatched_group_codes_export_blank_booking_room(self): def test_missing_and_unmatched_group_codes_export_blank_booking_room(self):
node = node_binary()
if not node:
self.skipTest("Node.js is unavailable")
class CapturingBuilder: class CapturingBuilder:
def __init__(self, delegate): def __init__(self, delegate):
self.delegate = delegate self.delegate = delegate
@@ -70,7 +46,7 @@ class CompanyReportIntegrationTests(unittest.TestCase):
) )
) )
builder = CapturingBuilder( builder = CapturingBuilder(
ArtifactToolBuilder(BUILDER_SCRIPT, node_binary=node, timeout_seconds=90) ArtifactToolBuilder(BUILDER_SCRIPT, timeout_seconds=90)
) )
publisher = AtomicReportPublisher(project_root, output_root) publisher = AtomicReportPublisher(project_root, output_root)
service = CompanyReportService( service = CompanyReportService(
@@ -101,10 +77,6 @@ class CompanyReportIntegrationTests(unittest.TestCase):
) )
def test_same_snapshot_rerun_reuses_publication_when_binary_changes(self): def test_same_snapshot_rerun_reuses_publication_when_binary_changes(self):
node = node_binary()
if not node:
self.skipTest("Node.js is unavailable")
class StableVersionRepository(FakeRepository): class StableVersionRepository(FakeRepository):
def reserve_report(self, report): def reserve_report(self, report):
reservation = ReservedReport(9001, 1, report.company) reservation = ReservedReport(9001, 1, report.company)
@@ -142,7 +114,7 @@ class CompanyReportIntegrationTests(unittest.TestCase):
project_root.mkdir(parents=True) project_root.mkdir(parents=True)
repository = StableVersionRepository(synthetic_snapshot()) repository = StableVersionRepository(synthetic_snapshot())
builder = RebuildingBuilder( builder = RebuildingBuilder(
ArtifactToolBuilder(BUILDER_SCRIPT, node_binary=node, timeout_seconds=90) ArtifactToolBuilder(BUILDER_SCRIPT, timeout_seconds=90)
) )
publisher = AtomicReportPublisher(project_root, output_root) publisher = AtomicReportPublisher(project_root, output_root)
service = CompanyReportService( service = CompanyReportService(

View File

@@ -7,9 +7,19 @@ from datetime import date
from pathlib import Path from pathlib import Path
from typing import Optional from typing import Optional
from company_reports.contracts import BatchSnapshot, ErrorCode from openpyxl import load_workbook
from company_reports.contracts import (
BatchSnapshot,
CompanyReport,
DailyVersionPin,
ErrorCode,
PeriodReport,
ReportRow,
)
from company_reports.core import build_company_report from company_reports.core import build_company_report
from company_reports.publishing import ( from company_reports.publishing import (
ArtifactToolBuilder,
AtomicReportPublisher, AtomicReportPublisher,
BuiltWorkbook, BuiltWorkbook,
PublicationError, PublicationError,
@@ -53,6 +63,38 @@ def empty_report():
) )
def workbook_report() -> CompanyReport:
row = ReportRow(
arrival=date(2026, 7, 8),
departure=date(2026, 7, 10),
nights=2,
block_code="=BLOCK-A",
res_comment="SYN-GROUP-A",
booking_room="【DBL】1",
total_booking_price="+900",
normalized_group_code="SYN-GROUP-A",
record_ids=(1,),
duplicate_group=True,
multi_price_review=True,
)
return CompanyReport(
company="QBD",
report_year=2026,
report_month=7,
as_of_date=date(2026, 7, 10),
filename="QBD-July-2026.xlsx",
periods=(
PeriodReport(1, 10, True, "QBD 01-10 Jul 2026", (row,)),
PeriodReport(11, 20, False, "QBD 11-20 Jul 2026", ()),
PeriodReport(21, 31, False, "QBD 21-31 Jul 2026", ()),
),
warnings=(),
errors=(),
daily_versions=(DailyVersionPin(date(2026, 7, 8), 100),),
booking_versions={"SYN-GROUP-A": 700},
)
def synthetic_built_workbook( def synthetic_built_workbook(
work_dir: Path, work_dir: Path,
filename: str, filename: str,
@@ -70,6 +112,41 @@ def synthetic_built_workbook(
class CompanyReportPublishingTests(unittest.TestCase): class CompanyReportPublishingTests(unittest.TestCase):
def test_builder_creates_xlsx_without_private_artifact_tool_runtime(self):
with tempfile.TemporaryDirectory(prefix="company-report-builder-test-") as temp_dir:
root = Path(temp_dir)
report = workbook_report()
builder = ArtifactToolBuilder(
Path("/private/artifact-tool/build_workbook.mjs"),
node_binary="/definitely/not/node",
)
built = builder.build(report, root)
self.assertTrue(built.path.is_file())
self.assertEqual(built.path.stat().st_mode & 0o777, 0o600)
self.assertEqual(built.summary["status"], "success")
self.assertEqual(built.summary["row_counts"], [1, 0, 0])
self.assertEqual(built.summary["formula_count"], 0)
self.assertEqual(len(built.summary["semantic_sha256"]), 64)
workbook = load_workbook(built.path)
self.assertEqual(workbook.sheetnames, [period.sheet_name for period in report.periods])
sheet = workbook[report.periods[0].sheet_name]
self.assertEqual(
[cell.value for cell in sheet[1]],
[
"ARRIVAL",
"DEPARTURE",
"NIGHTS",
"BLOCK_CODE",
"RES_COMMENT",
"Booking Room",
"Total Booking Price",
],
)
self.assertEqual(sheet["D2"].value, "'=BLOCK-A")
self.assertEqual(sheet["G2"].value, "'+900")
def test_publish_archives_prior_current_and_activates_new_version(self): def test_publish_archives_prior_current_and_activates_new_version(self):
with tempfile.TemporaryDirectory(prefix="company-report-publish-test-") as temp_dir: with tempfile.TemporaryDirectory(prefix="company-report-publish-test-") as temp_dir:
project_root = Path(temp_dir) / "project" project_root = Path(temp_dir) / "project"

View File

@@ -33,8 +33,12 @@ class DeploymentEntrypointTests(unittest.TestCase):
root_requirements = (PROJECT_ROOT / "requirements.txt").read_text( root_requirements = (PROJECT_ROOT / "requirements.txt").read_text(
encoding="utf-8" encoding="utf-8"
) )
company_requirements = (
PROJECT_ROOT / "requirements-company-reports.txt"
).read_text(encoding="utf-8")
self.assertIn("--enable-processing", compose) self.assertIn("--enable-processing", compose)
self.assertIn("--enable-company-reports", compose)
self.assertNotIn("\n mcp:", compose) self.assertNotIn("\n mcp:", compose)
self.assertNotIn("DEERFLOW", compose) self.assertNotIn("DEERFLOW", compose)
self.assertNotIn("ARR_MCP", compose) self.assertNotIn("ARR_MCP", compose)
@@ -47,6 +51,13 @@ class DeploymentEntrypointTests(unittest.TestCase):
self.assertNotIn("basic_auth", caddy) self.assertNotIn("basic_auth", caddy)
self.assertNotIn("requirements-agent-integration", root_requirements) self.assertNotIn("requirements-agent-integration", root_requirements)
self.assertNotIn("requirements-arr-mcp", root_requirements) self.assertNotIn("requirements-arr-mcp", root_requirements)
self.assertIn("openpyxl", company_requirements)
company_package = json.loads(
(PROJECT_ROOT / "company_reports" / "xlsx" / "package.json").read_text(
encoding="utf-8"
)
)
self.assertNotIn("@oai/artifact-tool", json.dumps(company_package))
if __name__ == "__main__": if __name__ == "__main__":