71 lines
6.6 KiB
Markdown
71 lines
6.6 KiB
Markdown
# Task: Add privacy-safe server diagnostics logging
|
||
|
||
## Identity
|
||
|
||
- Task ID: 20260830-server-diagnostics-c4d8a1f2
|
||
- Mode: Feature
|
||
- Branch: codex/20260830-wechat-attachment-correlation-9f3a2c-wechat-attachment-correlation
|
||
- Worktree: /Users/inmanx/Documents/lwltAPI-worktrees/20260830-wechat-attachment-correlation-9f3a2c
|
||
- Base commit: 23605066076f6c7c463564281379cc28bb27ec06
|
||
- Owner: codex
|
||
- Status: Ready for integration
|
||
|
||
## Scope
|
||
|
||
- Add one privacy-safe structured diagnostic schema for control-plane service lifecycle, HTTP requests, task/audit state, parser workers, AgentBus, roster attachment ingress, database/runtime failures, and artifact cleanup.
|
||
- Preserve correlation across `request_id`, `task_id`, AgentBus frame/channel/conversation identifiers, diagnostic event/stage, bounded error code/fingerprint, outcome, and duration without logging request bodies, attachment URLs/bytes, roster values, credentials, cookies, or tokens.
|
||
- Make production logs directly usable from the server through bounded Docker JSON-log retention and a read-only diagnostic command that reports container state, readiness, and filtered recent logs.
|
||
- Add regression coverage for redaction/error fingerprints, stable request IDs, production payload-log blocking, structured event fields, and deployment log rotation.
|
||
- Update active control-plane, environment-example, and deployment documentation.
|
||
|
||
## Intent And Constraints
|
||
|
||
- Build on attachment-correlation commit `23605066076f6c7c463564281379cc28bb27ec06`; retain all of its strict attachment and SSRF controls.
|
||
- "Complete" means complete lifecycle and correlation metadata, not raw sensitive payload capture. Production must reject `AGENTBUS_LOG_PAYLOADS=true`.
|
||
- Keep business audit rows authoritative while mirroring only event type, identifiers, state, and metadata keys into operational logs; never duplicate encrypted business data into logs.
|
||
- Logging failures must never change task processing, attachment download, or ERP behavior.
|
||
- Use stdout/stderr as the application log sink and Docker's existing log collection boundary; add bounded rotation rather than introducing a second mutable log database or repository log files.
|
||
- Do not change parser contracts, Schema, mapping, ERP, Chrome extension, or release artifacts. Do not read secrets, deploy, restart services, mutate live tasks, access ERP, or send external messages.
|
||
|
||
## Plan
|
||
|
||
1. Add reusable diagnostic helpers for safe error codes, fingerprints, stack frames, request IDs, paths, durations, and emergency JSON stderr records.
|
||
2. Wire structured service/HTTP/process/task/audit/parser events into the control plane and replace raw exception logging with safe descriptors.
|
||
3. Add privacy-safe attachment metadata, DNS, redirect, response, completion, and failure milestones to the existing AgentBus event stream.
|
||
4. Add Docker log rotation plus a read-only server diagnostic script and document filtering by request, task, frame, conversation, channel, or error code.
|
||
5. Add focused regression tests, run all repository gates, record the outcome, and leave the branch ready for integration without deployment.
|
||
|
||
## Outcome
|
||
|
||
- Added a shared privacy-safe diagnostic layer with bounded error codes, fingerprints, sanitized stack frames, request IDs, paths, durations, metadata-key summaries, and emergency JSON stderr records.
|
||
- Replaced raw exception logging across service startup, HTTP handling, parser workers, AgentBus, channel management, database runtime failures, CLI jobs, and OSS cleanup. Operational logging remains non-authoritative and cannot change business state.
|
||
- Added correlated lifecycle events for service, HTTP, task state, audit staging, parser queue/worker/persistence, readiness, AgentBus channel/frame delivery, roster attachment ingress, artifact cleanup, and process shutdown.
|
||
- Added detailed attachment milestones for metadata, DNS, IP family/count, redirects, HTTPS response, byte/hash verification, completion, failure, and duration without logging URL, hostname, IP, file name, bytes, roster values, or message bodies.
|
||
- Production now rejects raw AgentBus payload logging. Pino redaction covers common credential fields, and HTTP automatic request serialization is disabled in favor of the bounded diagnostic schema.
|
||
- Added bounded Docker `json-file` retention (default 20 MB × 10 files per container) and `infra/diagnose-server.sh`, a read-only command for container state, readiness, and literal correlation filtering.
|
||
- Updated deployment examples and operator documentation. No live service, database, ERP, external channel, release artifact, or deployment state was changed.
|
||
|
||
## Verification
|
||
|
||
- `node --run check:repo` — passed, 9/9.
|
||
- `node --run check` — passed.
|
||
- `node --run test:control-plane` — passed, 135/135.
|
||
- `node --run test:legacy` — passed, 248/248.
|
||
- `node --run build` — passed.
|
||
- `sh -n infra/diagnose-server.sh infra/predeploy-check.sh` — passed.
|
||
- `sh infra/diagnose-server.sh --help` — passed; invalid unbounded `--since` was rejected with exit 2.
|
||
- `git diff --check` — passed.
|
||
- Ruby/Psych parse of `docker-compose.yml` with aliases enabled — passed.
|
||
- `docker compose --env-file .env.production.example config --quiet` — not available on this workstation because the Docker CLI is not installed; Compose structure and rotation bindings are covered by the repository hygiene test.
|
||
|
||
## Follow-ups
|
||
|
||
- At integration/deployment time, set `DEPLOYMENT_REVISION` to the deployed commit or release identifier and keep `AGENTBUS_LOG_PAYLOADS=false`.
|
||
- After separately authorized deployment/restart, run `sh infra/diagnose-server.sh --since 10m` and verify `service.listening`, readiness, current deployment revision, and the live AgentBus attachment path.
|
||
- Existing pre-deployment container logs do not gain the new schema retroactively.
|
||
|
||
## Promotion Candidates
|
||
|
||
- Candidate: promote the privacy-safe diagnostic field contract and stdout/Docker retention boundary into canonical architecture/data-flow memory after integration acceptance. Target: `.project-docs/10-architecture/system-architecture.md` and the relevant canonical data-flow record. Evidence: this task record and the passing gates above. Human decision: not required unless canonical maintainers want a different log-retention policy.
|
||
- Candidate: promote the AgentBus attachment diagnostic privacy rule (stage metadata only; never URL, hostname, IP, file name, bytes, payload, or roster values) into canonical business constraints after integration acceptance. Target: relevant AgentBus/business rules memory. Evidence: `control-plane/src/input-attachment.ts`, `control-plane/src/agentbus.ts`, and their regression tests. Human decision: not required.
|