6.6 KiB
6.6 KiB
Task: Add privacy-safe server diagnostics logging
Identity
- Task ID: 20260830-server-diagnostics-c4d8a1f2
- Mode: Feature
- Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
- Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
- Base commit:
2360506607 - Owner: codex
- Status: Ready for integration
Scope
- Add one privacy-safe structured diagnostic schema for control-plane service lifecycle, HTTP requests, task/audit state, parser workers, AgentBus, roster attachment ingress, database/runtime failures, and artifact cleanup.
- Preserve correlation across
request_id,task_id, AgentBus frame/channel/conversation identifiers, diagnostic event/stage, bounded error code/fingerprint, outcome, and duration without logging request bodies, attachment URLs/bytes, roster values, credentials, cookies, or tokens. - Make production logs directly usable from the server through bounded Docker JSON-log retention and a read-only diagnostic command that reports container state, readiness, and filtered recent logs.
- Add regression coverage for redaction/error fingerprints, stable request IDs, production payload-log blocking, structured event fields, and deployment log rotation.
- Update active control-plane, environment-example, and deployment documentation.
Intent And Constraints
- Build on attachment-correlation commit
23605066076f6c7c463564281379cc28bb27ec06; retain all of its strict attachment and SSRF controls. - "Complete" means complete lifecycle and correlation metadata, not raw sensitive payload capture. Production must reject
AGENTBUS_LOG_PAYLOADS=true. - Keep business audit rows authoritative while mirroring only event type, identifiers, state, and metadata keys into operational logs; never duplicate encrypted business data into logs.
- Logging failures must never change task processing, attachment download, or ERP behavior.
- Use stdout/stderr as the application log sink and Docker's existing log collection boundary; add bounded rotation rather than introducing a second mutable log database or repository log files.
- Do not change parser contracts, Schema, mapping, ERP, Chrome extension, or release artifacts. Do not read secrets, deploy, restart services, mutate live tasks, access ERP, or send external messages.
Plan
- Add reusable diagnostic helpers for safe error codes, fingerprints, stack frames, request IDs, paths, durations, and emergency JSON stderr records.
- Wire structured service/HTTP/process/task/audit/parser events into the control plane and replace raw exception logging with safe descriptors.
- Add privacy-safe attachment metadata, DNS, redirect, response, completion, and failure milestones to the existing AgentBus event stream.
- Add Docker log rotation plus a read-only server diagnostic script and document filtering by request, task, frame, conversation, channel, or error code.
- Add focused regression tests, run all repository gates, record the outcome, and leave the branch ready for integration without deployment.
Outcome
- Added a shared privacy-safe diagnostic layer with bounded error codes, fingerprints, sanitized stack frames, request IDs, paths, durations, metadata-key summaries, and emergency JSON stderr records.
- Replaced raw exception logging across service startup, HTTP handling, parser workers, AgentBus, channel management, database runtime failures, CLI jobs, and OSS cleanup. Operational logging remains non-authoritative and cannot change business state.
- Added correlated lifecycle events for service, HTTP, task state, audit staging, parser queue/worker/persistence, readiness, AgentBus channel/frame delivery, roster attachment ingress, artifact cleanup, and process shutdown.
- Added detailed attachment milestones for metadata, DNS, IP family/count, redirects, HTTPS response, byte/hash verification, completion, failure, and duration without logging URL, hostname, IP, file name, bytes, roster values, or message bodies.
- Production now rejects raw AgentBus payload logging. Pino redaction covers common credential fields, and HTTP automatic request serialization is disabled in favor of the bounded diagnostic schema.
- Added bounded Docker
json-fileretention (default 20 MB × 10 files per container) andinfra/diagnose-server.sh, a read-only command for container state, readiness, and literal correlation filtering. - Updated deployment examples and operator documentation. No live service, database, ERP, external channel, release artifact, or deployment state was changed.
Verification
node --run check:repo— passed, 9/9.node --run check— passed.node --run test:control-plane— passed, 135/135.node --run test:legacy— passed, 248/248.node --run build— passed.sh -n infra/diagnose-server.sh infra/predeploy-check.sh— passed.sh infra/diagnose-server.sh --help— passed; invalid unbounded--sincewas rejected with exit 2.git diff --check— passed.- Ruby/Psych parse of
docker-compose.ymlwith aliases enabled — passed. docker compose --env-file .env.production.example config --quiet— not available on this workstation because the Docker CLI is not installed; Compose structure and rotation bindings are covered by the repository hygiene test.
Follow-ups
- At integration/deployment time, set
DEPLOYMENT_REVISIONto the deployed commit or release identifier and keepAGENTBUS_LOG_PAYLOADS=false. - After separately authorized deployment/restart, run
sh infra/diagnose-server.sh --since 10mand verifyservice.listening, readiness, current deployment revision, and the live AgentBus attachment path. - Existing pre-deployment container logs do not gain the new schema retroactively.
Promotion Candidates
- Candidate: promote the privacy-safe diagnostic field contract and stdout/Docker retention boundary into canonical architecture/data-flow memory after integration acceptance. Target:
.project-docs/10-architecture/system-architecture.mdand the relevant canonical data-flow record. Evidence: this task record and the passing gates above. Human decision: not required unless canonical maintainers want a different log-retention policy. - Candidate: promote the AgentBus attachment diagnostic privacy rule (stage metadata only; never URL, hostname, IP, file name, bytes, payload, or roster values) into canonical business constraints after integration acceptance. Target: relevant AgentBus/business rules memory. Evidence:
control-plane/src/input-attachment.ts,control-plane/src/agentbus.ts, and their regression tests. Human decision: not required.