Files
LWLT-AIBOT/.project-docs/30-worklog/tasks/20260830-server-diagnostics-c4d8a1f2.md
T

6.6 KiB
Raw Blame History

Task: Add privacy-safe server diagnostics logging

Identity

  • Task ID: 20260830-server-diagnostics-c4d8a1f2
  • Mode: Feature
  • Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
  • Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
  • Base commit: 2360506607
  • Owner: codex
  • Status: Ready for integration

Scope

  • Add one privacy-safe structured diagnostic schema for control-plane service lifecycle, HTTP requests, task/audit state, parser workers, AgentBus, roster attachment ingress, database/runtime failures, and artifact cleanup.
  • Preserve correlation across request_id, task_id, AgentBus frame/channel/conversation identifiers, diagnostic event/stage, bounded error code/fingerprint, outcome, and duration without logging request bodies, attachment URLs/bytes, roster values, credentials, cookies, or tokens.
  • Make production logs directly usable from the server through bounded Docker JSON-log retention and a read-only diagnostic command that reports container state, readiness, and filtered recent logs.
  • Add regression coverage for redaction/error fingerprints, stable request IDs, production payload-log blocking, structured event fields, and deployment log rotation.
  • Update active control-plane, environment-example, and deployment documentation.

Intent And Constraints

  • Build on attachment-correlation commit 23605066076f6c7c463564281379cc28bb27ec06; retain all of its strict attachment and SSRF controls.
  • "Complete" means complete lifecycle and correlation metadata, not raw sensitive payload capture. Production must reject AGENTBUS_LOG_PAYLOADS=true.
  • Keep business audit rows authoritative while mirroring only event type, identifiers, state, and metadata keys into operational logs; never duplicate encrypted business data into logs.
  • Logging failures must never change task processing, attachment download, or ERP behavior.
  • Use stdout/stderr as the application log sink and Docker's existing log collection boundary; add bounded rotation rather than introducing a second mutable log database or repository log files.
  • Do not change parser contracts, Schema, mapping, ERP, Chrome extension, or release artifacts. Do not read secrets, deploy, restart services, mutate live tasks, access ERP, or send external messages.

Plan

  1. Add reusable diagnostic helpers for safe error codes, fingerprints, stack frames, request IDs, paths, durations, and emergency JSON stderr records.
  2. Wire structured service/HTTP/process/task/audit/parser events into the control plane and replace raw exception logging with safe descriptors.
  3. Add privacy-safe attachment metadata, DNS, redirect, response, completion, and failure milestones to the existing AgentBus event stream.
  4. Add Docker log rotation plus a read-only server diagnostic script and document filtering by request, task, frame, conversation, channel, or error code.
  5. Add focused regression tests, run all repository gates, record the outcome, and leave the branch ready for integration without deployment.

Outcome

  • Added a shared privacy-safe diagnostic layer with bounded error codes, fingerprints, sanitized stack frames, request IDs, paths, durations, metadata-key summaries, and emergency JSON stderr records.
  • Replaced raw exception logging across service startup, HTTP handling, parser workers, AgentBus, channel management, database runtime failures, CLI jobs, and OSS cleanup. Operational logging remains non-authoritative and cannot change business state.
  • Added correlated lifecycle events for service, HTTP, task state, audit staging, parser queue/worker/persistence, readiness, AgentBus channel/frame delivery, roster attachment ingress, artifact cleanup, and process shutdown.
  • Added detailed attachment milestones for metadata, DNS, IP family/count, redirects, HTTPS response, byte/hash verification, completion, failure, and duration without logging URL, hostname, IP, file name, bytes, roster values, or message bodies.
  • Production now rejects raw AgentBus payload logging. Pino redaction covers common credential fields, and HTTP automatic request serialization is disabled in favor of the bounded diagnostic schema.
  • Added bounded Docker json-file retention (default 20 MB × 10 files per container) and infra/diagnose-server.sh, a read-only command for container state, readiness, and literal correlation filtering.
  • Updated deployment examples and operator documentation. No live service, database, ERP, external channel, release artifact, or deployment state was changed.

Verification

  • node --run check:repo — passed, 9/9.
  • node --run check — passed.
  • node --run test:control-plane — passed, 135/135.
  • node --run test:legacy — passed, 248/248.
  • node --run build — passed.
  • sh -n infra/diagnose-server.sh infra/predeploy-check.sh — passed.
  • sh infra/diagnose-server.sh --help — passed; invalid unbounded --since was rejected with exit 2.
  • git diff --check — passed.
  • Ruby/Psych parse of docker-compose.yml with aliases enabled — passed.
  • docker compose --env-file .env.production.example config --quiet — not available on this workstation because the Docker CLI is not installed; Compose structure and rotation bindings are covered by the repository hygiene test.

Follow-ups

  • At integration/deployment time, set DEPLOYMENT_REVISION to the deployed commit or release identifier and keep AGENTBUS_LOG_PAYLOADS=false.
  • After separately authorized deployment/restart, run sh infra/diagnose-server.sh --since 10m and verify service.listening, readiness, current deployment revision, and the live AgentBus attachment path.
  • Existing pre-deployment container logs do not gain the new schema retroactively.

Promotion Candidates

  • Candidate: promote the privacy-safe diagnostic field contract and stdout/Docker retention boundary into canonical architecture/data-flow memory after integration acceptance. Target: .project-docs/10-architecture/system-architecture.md and the relevant canonical data-flow record. Evidence: this task record and the passing gates above. Human decision: not required unless canonical maintainers want a different log-retention policy.
  • Candidate: promote the AgentBus attachment diagnostic privacy rule (stage metadata only; never URL, hostname, IP, file name, bytes, payload, or roster values) into canonical business constraints after integration acceptance. Target: relevant AgentBus/business rules memory. Evidence: control-plane/src/input-attachment.ts, control-plane/src/agentbus.ts, and their regression tests. Human decision: not required.