63 lines
5.4 KiB
Markdown
63 lines
5.4 KiB
Markdown
# Task: Fix server XLS roster conversion
|
|
|
|
## Identity
|
|
|
|
- Task ID: 20260831-fix-xls-conversion-83b17d6a
|
|
- Mode: Feature
|
|
- Branch: main
|
|
- Worktree: /Users/inmanx/Documents/lwltAPI
|
|
- Base commit: 4ba2d437102f5f2bbe0381c0f1069686f3c04efb
|
|
- Owner: codex
|
|
- Status: Ready for integration
|
|
|
|
## Scope
|
|
|
|
- Inspect the user-supplied production task event and server-log excerpt for `roster_workbook_conversion_failed` after an internal AgentBus `.xls` download.
|
|
- Trace the legacy XLS normalization path through the container image, document converter, workbook validator, task event, and attachment idempotency logic.
|
|
- Restore production `.xls` conversion support, expose a privacy-safe conversion subcode, and allow a new message to revalidate previously rejected bytes.
|
|
- Add regression coverage and run the full repository gates.
|
|
|
|
## Intent And Constraints
|
|
|
|
- Continue accepting one genuine legacy `.xls` or `.xlsx` roster while preserving format/MIME/magic checks, macro rejection, bounded isolated conversion, XLSX archive inspection, deterministic template validation, and encrypted normalized storage.
|
|
- Never log or persist workbook bytes, plaintext file names, cell values, converter paths, or raw converter output.
|
|
- A replay of the same AgentBus message must remain idempotent; a later newly delivered message may revalidate a rejected digest because runtime or converter fixes can change the result.
|
|
- Do not deploy, restart services, mutate Kubernetes, access ERP, read secrets, retry the live task, or send an external message.
|
|
|
|
## Outcome
|
|
|
|
- Confirmed that attachment correlation and internal HTTPS download now work. The supplied event kept the original task in `awaiting_attachment`, recorded 31,744 downloaded bytes, and failed only at `roster_workbook_conversion_failed`; no new business task or ERP execution was started.
|
|
- The workbook reached legacy conversion only after passing the `.xls` extension, accepted MIME, and OLE compound-file magic checks. The accompanying 99-line log excerpt contains only healthy HTTP polling after the event and no converter stderr, so it does not establish a malformed workbook.
|
|
- Identified the production image defect: the converter requests the Calc-only `Calc MS Excel 2007 XML` export filter, while the runtime image installed only `libreoffice-writer`. Debian packages Calc as the separate spreadsheet component and Writer does not depend on it.
|
|
- Added `libreoffice-calc` to the runtime image alongside Writer and added a repository gate requiring both components.
|
|
- Added a typed, bounded conversion error-code contract and carries its safe value into the roster rejection event as `conversion_error_code`; expected values distinguish unavailable converter, timeout, process failure, missing/invalid output, and output-size failure without exposing paths or content.
|
|
- Preserved an explicit workbook conversion error through the normalizer instead of collapsing it to a generic code.
|
|
- Changed rejected-attachment deduplication so a later message revalidates the same SHA-256 and can upgrade the existing rejected record to normalized after a server fix. The same AgentBus idempotency key still short-circuits as a duplicate, and already normalized bytes remain deduplicated.
|
|
- No deployment, restart, Kubernetes mutation, ERP access, secret read, live-task retry, or external message occurred.
|
|
|
|
## Verification
|
|
|
|
- Focused converter, workbook, intake, and retry tests: 20/20 passed.
|
|
- `node --run check:repo`: 10/10 passed, including the new runtime Calc/Writer dependency gate.
|
|
- `node --run check`: passed.
|
|
- `node --run test:control-plane`: 137/137 passed.
|
|
- `node --run test:legacy`: 256/256 passed.
|
|
- `node --run build`: passed.
|
|
- `check_project_docs.py`: passed.
|
|
- `git diff --check`: passed.
|
|
- Docker and Podman CLIs are unavailable on this workstation, so an actual image build was not performed locally; runtime package installation remains to be exercised by the authorized deployment pipeline.
|
|
|
|
## Follow-ups
|
|
|
|
- Build and deploy the new image only under separate authorization, set `DEPLOYMENT_REVISION`, and confirm the image contains `libreoffice-calc`.
|
|
- After deployment, resend the same workbook in a new attachment message. The expected path is conversion success, a normalized row count, and transition from `awaiting_attachment` to `parse_queued`; no file modification is required to change its digest.
|
|
|
|
## Promotion Candidates
|
|
|
|
- Target canonical documents: `.project-docs/30-worklog/current-state.md` and `.project-docs/50-evidence/evidence-index.md`.
|
|
- Proposal: record that the live environment progressed through the internal download fix to a runtime spreadsheet-converter packaging failure, and that the repository now requires both Writer and Calc plus safe conversion subcodes and rejected-byte revalidation.
|
|
- Evidence: the user-supplied 2026-08-31 task event, current Dockerfile/package boundary, official Debian package descriptions, and this task's focused/full verification.
|
|
- Future impact: deployment checks should treat the Calc component and a non-`unknown` deployment revision as required evidence before live `.xls` verification.
|
|
- Semantic conflicts: the integrated snapshot still says the internal attachment fix is not active; the supplied runtime behavior proves the download boundary advanced, but `deployment_revision` remains `unknown`, so the exact deployed commit cannot be identified.
|
|
- Human confirmation required: no for recording the bounded runtime observation; deployment itself remains separately authorized.
|