merge: integrate attachment correlation and server diagnostics

.project-docs/30-worklog/tasks/20260830-server-diagnostics-c4d8a1f2.md

@@ -0,0 +1,70 @@
+# Task: Add privacy-safe server diagnostics logging
+
+## Identity
+
+- Task ID: 20260830-server-diagnostics-c4d8a1f2
+- Mode: Feature
+- Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
+- Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
+- Base commit: 23605066076f6c7c463564281379cc28bb27ec06
+- Owner: codex
+- Status: Ready for integration
+
+## Scope
+
+- Add one privacy-safe structured diagnostic schema for control-plane service lifecycle, HTTP requests, task/audit state, parser workers, AgentBus, roster attachment ingress, database/runtime failures, and artifact cleanup.
+- Preserve correlation across `request_id`, `task_id`, AgentBus frame/channel/conversation identifiers, diagnostic event/stage, bounded error code/fingerprint, outcome, and duration without logging request bodies, attachment URLs/bytes, roster values, credentials, cookies, or tokens.
+- Make production logs directly usable from the server through bounded Docker JSON-log retention and a read-only diagnostic command that reports container state, readiness, and filtered recent logs.
+- Add regression coverage for redaction/error fingerprints, stable request IDs, production payload-log blocking, structured event fields, and deployment log rotation.
+- Update active control-plane, environment-example, and deployment documentation.
+
+## Intent And Constraints
+
+- Build on attachment-correlation commit `23605066076f6c7c463564281379cc28bb27ec06`; retain all of its strict attachment and SSRF controls.
+- "Complete" means complete lifecycle and correlation metadata, not raw sensitive payload capture. Production must reject `AGENTBUS_LOG_PAYLOADS=true`.
+- Keep business audit rows authoritative while mirroring only event type, identifiers, state, and metadata keys into operational logs; never duplicate encrypted business data into logs.
+- Logging failures must never change task processing, attachment download, or ERP behavior.
+- Use stdout/stderr as the application log sink and Docker's existing log collection boundary; add bounded rotation rather than introducing a second mutable log database or repository log files.
+- Do not change parser contracts, Schema, mapping, ERP, Chrome extension, or release artifacts. Do not read secrets, deploy, restart services, mutate live tasks, access ERP, or send external messages.
+
+## Plan
+
+1. Add reusable diagnostic helpers for safe error codes, fingerprints, stack frames, request IDs, paths, durations, and emergency JSON stderr records.
+2. Wire structured service/HTTP/process/task/audit/parser events into the control plane and replace raw exception logging with safe descriptors.
+3. Add privacy-safe attachment metadata, DNS, redirect, response, completion, and failure milestones to the existing AgentBus event stream.
+4. Add Docker log rotation plus a read-only server diagnostic script and document filtering by request, task, frame, conversation, channel, or error code.
+5. Add focused regression tests, run all repository gates, record the outcome, and leave the branch ready for integration without deployment.
+
+## Outcome
+
+- Added a shared privacy-safe diagnostic layer with bounded error codes, fingerprints, sanitized stack frames, request IDs, paths, durations, metadata-key summaries, and emergency JSON stderr records.
+- Replaced raw exception logging across service startup, HTTP handling, parser workers, AgentBus, channel management, database runtime failures, CLI jobs, and OSS cleanup. Operational logging remains non-authoritative and cannot change business state.
+- Added correlated lifecycle events for service, HTTP, task state, audit staging, parser queue/worker/persistence, readiness, AgentBus channel/frame delivery, roster attachment ingress, artifact cleanup, and process shutdown.
+- Added detailed attachment milestones for metadata, DNS, IP family/count, redirects, HTTPS response, byte/hash verification, completion, failure, and duration without logging URL, hostname, IP, file name, bytes, roster values, or message bodies.
+- Production now rejects raw AgentBus payload logging. Pino redaction covers common credential fields, and HTTP automatic request serialization is disabled in favor of the bounded diagnostic schema.
+- Added bounded Docker `json-file` retention (default 20 MB × 10 files per container) and `infra/diagnose-server.sh`, a read-only command for container state, readiness, and literal correlation filtering.
+- Updated deployment examples and operator documentation. No live service, database, ERP, external channel, release artifact, or deployment state was changed.
+
+## Verification
+
+- `node --run check:repo` — passed, 9/9.
+- `node --run check` — passed.
+- `node --run test:control-plane` — passed, 135/135.
+- `node --run test:legacy` — passed, 248/248.
+- `node --run build` — passed.
+- `sh -n infra/diagnose-server.sh infra/predeploy-check.sh` — passed.
+- `sh infra/diagnose-server.sh --help` — passed; invalid unbounded `--since` was rejected with exit 2.
+- `git diff --check` — passed.
+- Ruby/Psych parse of `docker-compose.yml` with aliases enabled — passed.
+- `docker compose --env-file .env.production.example config --quiet` — not available on this workstation because the Docker CLI is not installed; Compose structure and rotation bindings are covered by the repository hygiene test.
+
+## Follow-ups
+
+- At integration/deployment time, set `DEPLOYMENT_REVISION` to the deployed commit or release identifier and keep `AGENTBUS_LOG_PAYLOADS=false`.
+- After separately authorized deployment/restart, run `sh infra/diagnose-server.sh --since 10m` and verify `service.listening`, readiness, current deployment revision, and the live AgentBus attachment path.
+- Existing pre-deployment container logs do not gain the new schema retroactively.
+
+## Promotion Candidates
+
+- Candidate: promote the privacy-safe diagnostic field contract and stdout/Docker retention boundary into canonical architecture/data-flow memory after integration acceptance. Target: `.project-docs/10-architecture/system-architecture.md` and the relevant canonical data-flow record. Evidence: this task record and the passing gates above. Human decision: not required unless canonical maintainers want a different log-retention policy.
+- Candidate: promote the AgentBus attachment diagnostic privacy rule (stage metadata only; never URL, hostname, IP, file name, bytes, payload, or roster values) into canonical business constraints after integration acceptance. Target: relevant AgentBus/business rules memory. Evidence: `control-plane/src/input-attachment.ts`, `control-plane/src/agentbus.ts`, and their regression tests. Human decision: not required.

.project-docs/30-worklog/tasks/20260830-wechat-attachment-correlation-9f3a2c.md

@@ -0,0 +1,69 @@
+# Task: Fix WeChat attachment task correlation
+
+## Identity
+
+- Task ID: 20260830-wechat-attachment-correlation-9f3a2c
+- Mode: Feature
+- Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
+- Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
+- Base commit: e7aa58a203f9c05850a3d10e681b8800b856ee12
+- Owner: codex
+- Status: Ready for Integration
+
+## Scope
+
+- Diagnose the deployed WeChat/AgentBus roster-attachment failure shown at 2026-08-30 15:19.
+- Keep an attachment-only WeChat message from entering the ordinary new-task path when the bridge supplied only its text placeholder.
+- Use the exact WeChat envelope `Conversation:` value for task correlation when no explicit AgentBus conversation field is present.
+- Return safe, actionable attachment-ingress errors instead of collapsing metadata, URL, DNS, download, size, or digest failures into the generic task-processing message.
+- Add focused regression coverage and update the active AgentBus transport documentation.
+
+## Intent And Constraints
+
+- Preserve the existing Program-only roster workflow: one `.xls/.xlsx` attachment must become `payload.attachments[]` with a bounded public HTTPS URL before it can resume an `awaiting_attachment` task.
+- Do not weaken HTTPS, credential, redirect, DNS, private/reserved-network, size, or SHA-256 checks; absent file bytes must fail closed and leave the original task waiting.
+- Do not change parser, operation, Schema, mapping, ERP, Chrome-extension, or release-artifact behavior.
+- Do not read local secrets, deploy, restart services, mutate live tasks, access ERP, or send external messages.
+- Treat the screenshots as evidence only. The raw production frame and server error code are unavailable, so distinguish the two supported failure branches in logs and user replies instead of asserting one without evidence.
+
+## Plan
+
+1. Add strict parsing for the observed WeChat transport envelope, including its conversation identifier and attachment-only placeholder.
+2. Reject placeholder-only attachment messages before `TaskService.ingestMessage`, and surface the safe `InputAttachmentError` reason and code.
+3. Add listener tests proving conversation correlation, no accidental task creation, actionable failure text, and unchanged strict-envelope behavior.
+4. Update the AgentBus contract/readme, then run targeted and full repository verification.
+
+## Outcome
+
+- Confirmed the failure had two distinct ingress problems before task selection: the strict WeChat transport envelope exposed a `Conversation:` value that was not promoted to the AgentBus frame's conversation key, while an attachment card could arrive as placeholder text without the required `payload.attachments[]` file metadata.
+- Added strict envelope parsing that preserves explicit AgentBus `conversation_id` precedence and otherwise uses the observed WeChat `Conversation:` value. A real structured attachment can therefore select the unique `awaiting_attachment` task in the same channel and conversation through the existing `TaskService` path.
+- Added a fail-closed guard for `[WeChat attachment: ...]` placeholder-only frames. They now stop before `TaskService.ingestMessage`, do not create a second business task, leave the original roster task waiting, and return an actionable safe message that the file content never reached the platform.
+- Preserved all existing attachment download controls. Pre-ingest `InputAttachmentError` failures now return their predefined safe summary and log a bounded error code instead of being collapsed into the generic task-processing failure; URLs, file bytes, and roster values remain absent from replies and logs.
+- Updated the active AgentBus transport contract and control-plane README. No parser, Schema, mapping, ERP, Chrome-extension, release artifact, deployment, service process, live task, or external system was changed.
+
+## Verification
+
+- Focused AgentBus listener regression: 14/14 passed, including envelope conversation precedence, placeholder-only fail-closed behavior, zero task-ingestion calls, bounded error logging, and listener teardown.
+- `node --run check:repo`: 9/9 passed.
+- `node --run check`: passed.
+- `node --run test:control-plane`: 128/128 passed.
+- `node --run test:legacy`: 248/248 passed.
+- `node --run build`: passed.
+- `check_project_docs.py`: passed.
+- `check_doc_drift.py --task-id 20260830-wechat-attachment-correlation-9f3a2c`: passed.
+- `git diff --check`: passed.
+
+## Follow-ups
+
+- The external WeChat bridge must deliver each workbook as one real `payload.attachments[]` entry with the documented name, size/type/hash metadata and a control-plane-reachable public HTTPS URL. Placeholder text alone cannot supply file bytes and is intentionally not converted into an attachment.
+- The raw 2026-08-30 production frame and its original server-side exception were not available. After this branch is integrated and deployed under separate authorization, observe the bounded `error_code` to distinguish missing metadata from URL, DNS, download, size, or digest failure without inspecting sensitive payloads.
+- Integration, deployment, restart, and a live WeChat retry remain separate authorized actions; this Feature task performs none of them.
+
+## Promotion Candidates
+
+- Target: `.project-docs/20-architecture/data-flow.md` and `.project-docs/40-domain/business-rules.md`.
+  Proposal: record that a strict WeChat envelope may supply the AgentBus conversation fallback, but a roster attachment exists only when the frame carries a validated `payload.attachments[]` entry; placeholder text must never create a new task and must leave the prior roster task awaiting its file.
+  Evidence: `control-plane/src/agentbus.ts`, `control-plane/test/agentbus.test.ts`, `control-plane/README.md`, and `agent设计规范/agentbus-reply-contract.md` on this task branch; focused 14/14, control-plane 128/128, and legacy 248/248 tests passed.
+  Future impact: future bridge or channel adapters must preserve stable conversation identity and send file metadata/URL separately from user-visible attachment placeholder text; operational diagnosis should use bounded attachment error codes rather than payload logging.
+  Semantic conflicts: none found in active contracts; the proposal makes the existing structured-attachment requirement and task-correlation boundary explicit.
+  Human confirmation: not required for the rule itself because it preserves the existing fail-closed attachment contract, but Integration mode is required before writing canonical project memory.

.project-docs/30-worklog/tasks/20260831-push-server-diagnostics-7c4e91a2.md

@@ -0,0 +1,43 @@
+# Task: Push server diagnostics branch
+
+## Identity
+
+- Task ID: 20260831-push-server-diagnostics-7c4e91a2
+- Mode: Feature
+- Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
+- Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
+- Base commit: a3963ad0f629818edfa921261acfeb28dde9c9dd
+- Owner: codex
+- Status: Ready for integration
+
+## Scope
+
+- Push the existing, tested server-diagnostics feature branch to the configured `origin` remote and establish upstream tracking.
+- Preserve the exact local commits and publish them under the same feature-branch name.
+- Record and verify the remote ref without changing source code, canonical project memory, release artifacts, or deployment state.
+
+## Intent And Constraints
+
+- The requested operation authorizes a Git remote push only; it does not authorize merging, rebasing, force-pushing, changing `main`, deploying, restarting services, or mutating live systems.
+- Remote `main` advanced independently to `98282d1` and diverges from this branch at `7b5d855`; publish the feature branch as-is so later integration can reconcile the histories explicitly.
+- Push only to `origin` (`LWLT-AIBOT.git`), not the separately configured legacy repository.
+- Preserve the validated server-diagnostics payload commit `a3963ad0f629818edfa921261acfeb28dde9c9dd`.
+
+## Outcome
+
+- The current feature branch was published to the same-named branch on `origin` with upstream tracking.
+- Remote `main`, the legacy remote, deployment state, and live services were not changed.
+
+## Verification
+
+- `git fetch origin` completed and showed remote `main` at `98282d127c989a7e9ca1d688c9d3d2fadd73e8bc`.
+- Pre-push `git ls-remote --heads origin <feature-branch>` returned no existing branch, so the push creates a new remote feature ref rather than overwriting shared history.
+- After push, verify that the remote feature ref equals local `HEAD` and that branch upstream tracking points to the same `origin` ref.
+
+## Follow-ups
+
+- Integration or merge into `main` remains a separate, explicitly authorized task because the remote histories have diverged.
+
+## Promotion Candidates
+
+- None; this task publishes an existing feature branch and introduces no canonical project fact or behavior change.