fix: keep dashboard candidate queries lightweight

This commit is contained in:
inman
2026-09-03 11:10:05 +08:00
parent 69ea6d2517
commit 5cbd8f1e94
4 changed files with 181 additions and 5 deletions

View File

@@ -0,0 +1,62 @@
# Task: Diagnose local dashboard refresh with no data
## Identity
- Task ID: 20260903-dashboard-refresh-no-data-b4e82f1a
- Mode: Feature
- Branch: codex/20260903-dashboard-refresh-no-data-b4e82f1a-dashboard-refresh-no-data
- Worktree: /Users/inmanx/Documents/lwltAPI-dashboard-refresh-no-data-b4e82f1a
- Base commit: 69ea6d25178d75abf8d7fd728bd31c61764caa7d
- Owner: codex
- Status: Ready for Integration
## Scope
- Reproduce the signed-in local `127.0.0.1:8786` leadership-dashboard refresh failure without changing accounts, tasks, database state, ERP state, or the running service.
- Trace the frontend request, HTTP response, structured service diagnostics, and dashboard candidate-query projection.
- Keep the accepted administrator/team-lead read-only visibility and 30-day default range unchanged while removing the pre-pagination full-operation payload read that exhausts the dashboard statement timeout.
- Add focused regression coverage and run repository verification from this isolated worktree; do not restart or deploy the service without separate authorization.
## Intent And Constraints
- Preserve the concurrently inspected team-lead dashboard scope and the integrated account-scoped execution/deletion decisions; this task changes dashboard read performance only.
- Keep complete instruction/result projection limited to the current page and keep legacy business classification deterministic from the minimum required operation fields.
- Do not read `.env`, credentials, cookies, browser storage, plaintext database business inputs, or customer payloads. Runtime observations must remain aggregate- or error-code-only.
- Keep the occupied main worktree and its running 8786 service untouched. All repository edits remain rooted in this task worktree and its recorded base; current `origin/main` `f52d9d7` was compared read-only and has the same dashboard candidate query.
## Outcome
- Reproduced the default 30-day refresh as HTTP 503 `operations_dashboard_query_timeout`; the matching service diagnostic reports PostgreSQL cancellation code `57014` before the `candidates` stage completes.
- Confirmed the same authenticated dashboard succeeds for seven days in about 0.7 seconds and returns 13 aggregate task records, proving the service, database readiness, session, permissions, and underlying data are present.
- Root cause: the candidate query selects the complete `t.operation` JSON for every matching task before applying the 20-row page, so the default range exceeds the 5-second SQL statement timeout as payload volume grows.
- Evidence record: [Local dashboard candidate-query timeout](../../50-evidence/topics/20260903-dashboard-refresh-no-data-b4e82f1a__dashboard-query-timeout.md).
- Confirmed current `origin/main` `f52d9d7` has the same dashboard candidate query, then kept the final feature commit on the task's recorded `69ea6d2` base so project-document drift is measured only against this task's changes.
- Removed full `t.operation` from the normal pre-pagination candidate projection. The candidate pass now reads lightweight metadata plus encrypted input only for legacy unclassified rows.
- Preserved legacy classification by hydrating only still-unresolved rows with a compact operation object containing the exact `action`, passenger `kind`, and arrangement `mode` fields used by the existing classifier.
- Kept full operation and result fields in the current-page detail and keyword-search projections, preserving visible input/output and search behavior.
- Added focused regression checks proving the base/candidate projections cannot reintroduce a full operation read and that legacy classification remains targeted.
## Verification
- Runtime read-only reproduction: 30-day request returned HTTP 503 after about 8.9 seconds; seven-day request returned HTTP 200 and 13 records.
- `node --test LianSyn-platform/app-operations-dashboard.test.mjs` passed: 5/5.
- `node --test --import tsx control-plane/test/account-authorization.test.ts` passed: 11/11.
- `node --run check:repo` passed: 10/10.
- `node --run check` passed.
- `node --run test:control-plane` passed: 159/159.
- `node --run test:legacy` passed: 268/268.
- `node --run build` passed.
- Post-change 30-day runtime timing was not claimed because applying the code requires integrating it and separately authorizing a restart of the occupied 8786 service.
## Follow-ups
- Integrate this feature branch, then restart the local 8786 service only with explicit authorization and repeat the authenticated 30-day dashboard request. Compare aggregate totals with the working seven-day baseline and confirm the `candidates`, optional `legacy_classification`, and `page_details` stage timings.
## Promotion Candidates
- Target canonical documents: `.project-docs/30-worklog/current-state.md` and `.project-docs/50-evidence/evidence-index.md` during Integration Gate review.
- Proposal: record that dashboard range queries must keep full operation/result payloads out of the pre-pagination candidate projection; only unresolved legacy classification may load its minimal action/kind/mode shape, while full detail remains page-scoped.
- Evidence: the authenticated 30-day/7-day comparison, PostgreSQL `57014` diagnostic, source diff, focused regression, and full repository verification above.
- Future impact: prevents normal dashboard growth from turning a valid non-empty result into a misleading timeout/empty board while preserving role scope, filters, business classification, and business-facing detail.
- Semantic conflicts: none; the change preserves `AUTH-001`, `AUTH-002`, the 30-day default, and the concurrent team-lead visibility inspection.
- Human confirmation required: not for the repository performance correction; still required before service restart or rollout.

View File

@@ -0,0 +1,37 @@
# Evidence: Local dashboard candidate-query timeout
## Scope
- Local signed-in `http://127.0.0.1:8786/operations-dashboard` reads only.
- Current 8786 process, health endpoints, HTTP response metadata, and privacy-safe structured diagnostics.
- Active dashboard query source at task base `69ea6d2`; read-only comparison confirmed current `origin/main` `f52d9d7` is unchanged in this query area.
## Findings
- `/health/live` and `/health/ready` passed; PostgreSQL and required migration `018_agentbus_account_workers` were ready.
- The default 30-day dashboard GET returned HTTP 503 after about 8.9 seconds with `operations_dashboard_query_timeout`.
- The matching service diagnostic reported PostgreSQL cancellation code `57014`; no `operations_dashboard.query.completed` event was emitted for the `candidates` stage.
- The same authenticated page completed a seven-day GET in about 0.7 seconds. The candidate, actor, and page-detail stages completed, and the aggregate response contained 13 tasks.
- Source inspection showed that the candidate projection selected complete `t.operation` JSON for every matching row before slicing the 20-row page. Full detail was therefore read for the entire range rather than only the current page.
## Conclusion
The visible empty board was a timeout presentation, not an empty database or authorization failure. Candidate payload amplification exhausted the five-second SQL statement timeout as the date range grew.
## Evidence Safety
- No `.env`, credentials, cookies, browser storage, plaintext instructions, customer data, task IDs, or operation payloads were read or recorded.
- Runtime evidence is limited to health state, HTTP/error codes, durations, query-stage names, and aggregate counts.
## Confidence
- High for the reproduced failure and failing query stage.
- Repository verification can prove the candidate projection no longer selects full operations, but post-change 30-day runtime timing requires a separately authorized service rollout/restart.
## Last Verified
2026-09-03
## Stale Trigger
Re-run the authenticated 30-day request after this feature is integrated and the local service is restarted from that revision, or whenever the dashboard query plan/projection changes again.

View File

@@ -87,6 +87,38 @@ test('table filters use a bounded single-connection read and default to 20 rows'
assert.match(dashboardQuery, /SET LOCAL statement_timeout/);
assert.match(dashboardQuery, /SELECT \$\{search \? taskSearchSelect : taskCandidateSelect\}/);
assert.match(dashboardQuery, /CASE WHEN t\.business_route_id IS NULL THEN t\.original_text_ciphertext ELSE NULL END/);
const baseSelect = dashboardQuery.slice(
dashboardQuery.indexOf('const taskBaseSelect'),
dashboardQuery.indexOf('const taskLegacyOperationSelect')
);
const legacyOperationSelect = dashboardQuery.slice(
dashboardQuery.indexOf('const taskLegacyOperationSelect'),
dashboardQuery.indexOf('const taskDetailSelect')
);
const detailSelect = dashboardQuery.slice(
dashboardQuery.indexOf('const taskDetailSelect'),
dashboardQuery.indexOf('const taskSearchSelect')
);
const searchSelect = dashboardQuery.slice(
dashboardQuery.indexOf('const taskSearchSelect'),
dashboardQuery.indexOf('const taskCandidateSelect')
);
const candidateSelect = dashboardQuery.slice(
dashboardQuery.indexOf('const taskCandidateSelect'),
dashboardQuery.indexOf('try {', dashboardQuery.indexOf('const taskCandidateSelect'))
);
assert.doesNotMatch(baseSelect, /t\.operation/);
assert.match(legacyOperationSelect, /'action', t\.operation -> 'action'/);
assert.match(legacyOperationSelect, /'kind', t\.operation #> '\{data,existing_refs,kind\}'/);
assert.match(legacyOperationSelect, /'mode', t\.operation #> '\{data,arrangement,mode\}'/);
assert.match(detailSelect, /t\.operation, t\.original_text_ciphertext/);
assert.match(searchSelect, /t\.operation, t\.original_text_ciphertext/);
assert.doesNotMatch(candidateSelect, /t\.operation/);
assert.doesNotMatch(candidateSelect, /taskLegacyOperationSelect/);
assert.match(dashboardQuery, /filter\(\(\{ projection \}\) => !projection\.task\.business_route_id\)/);
assert.match(dashboardQuery, /'legacy_classification'/);
assert.match(dashboardQuery, /SELECT t\.id, \$\{taskLegacyOperationSelect\}/);
assert.match(dashboardQuery, /legacyOperationById/);
assert.match(dashboardQuery, /'search_messages'/);
assert.match(dashboardQuery, /filter\(\(\{ directSearchMatch \}\) => !directSearchMatch\)/);
assert.doesNotMatch(dashboardQuery, /const pool = getPool\(this\.config\)/);

View File

@@ -4747,12 +4747,25 @@ export class TaskService {
{ key: 'unclassified', label: '其他任务' }
];
const taskBaseSelect = `t.id, t.task_id, t.status, t.business_route_id, t.archived_at,
t.created_at, t.updated_at, t.operation,
t.created_at, t.updated_at,
creator.id AS actor_id, creator.username AS actor_username,
creator.role AS actor_role, creator.is_active AS actor_is_active,
to_char(timezone('Asia/Shanghai', t.created_at), 'YYYY-MM-DD') AS business_day`;
const taskLegacyOperationSelect = `jsonb_build_object(
'action', t.operation -> 'action',
'data', jsonb_build_object(
'existing_refs', jsonb_build_object(
'kind', t.operation #> '{data,existing_refs,kind}'
),
'target_kind', t.operation #> '{data,target_kind}',
'arrangement', jsonb_build_object(
'mode', t.operation #> '{data,arrangement,mode}'
),
'mode', t.operation #> '{data,mode}'
)
) AS operation`;
const taskDetailSelect = `${taskBaseSelect},
t.original_text_ciphertext, t.message, t.success_receipt, t.success_receipt_at,
t.operation, t.original_text_ciphertext, t.message, t.success_receipt, t.success_receipt_at,
t.error_summary, t.error_summary_at, t.execution_result, t.parse_response,
(SELECT COUNT(*)::int
FROM agent_session_messages input_message
@@ -4761,7 +4774,7 @@ export class TaskService {
AND input_message.role = 'user'
AND input_message.input_source IS DISTINCT FROM 'reparse') AS input_turn_count`;
const taskSearchSelect = `${taskBaseSelect},
t.original_text_ciphertext, t.message, t.success_receipt, t.success_receipt_at,
t.operation, t.original_text_ciphertext, t.message, t.success_receipt, t.success_receipt_at,
t.error_summary, t.error_summary_at,
CASE
WHEN t.status = 'completed'
@@ -4839,8 +4852,40 @@ export class TaskService {
[context.organizationId]
);
const candidateRows = candidateResult.rows as Record<string, unknown>[];
const projectedCandidates = candidateRows.flatMap((row) => {
const projection = operationsDashboardTaskProjection(this.config, row);
let projectedRows = candidateRows.map((row) => ({
row,
projection: operationsDashboardTaskProjection(this.config, row)
}));
if (!search) {
const unresolvedLegacyIds = projectedRows
.filter(({ projection }) => !projection.task.business_route_id)
.map(({ row }) => text(row.id))
.filter(Boolean);
if (unresolvedLegacyIds.length) {
const legacyOperationResult = await dashboardQuery(
'legacy_classification',
`SELECT t.id, ${taskLegacyOperationSelect}
FROM tasks t
WHERE t.organization_id = $1
AND t.id = ANY($2::uuid[])`,
[context.organizationId, unresolvedLegacyIds]
);
const legacyOperationById = new Map(
(legacyOperationResult.rows as Record<string, unknown>[])
.map((row) => [text(row.id), row.operation])
);
projectedRows = projectedRows.map(({ row, projection }) => {
const operation = legacyOperationById.get(text(row.id));
if (operation === undefined) return { row, projection };
const enrichedRow = { ...row, operation };
return {
row: enrichedRow,
projection: operationsDashboardTaskProjection(this.config, enrichedRow)
};
});
}
}
const projectedCandidates = projectedRows.flatMap(({ row, projection }) => {
const routeMatches = !businessRouteId
|| (businessRouteId === 'unclassified'
? !projection.task.business_route_id