From 4321c77613ab9a75b1cb7474838fdc7bd94a2cb5 Mon Sep 17 00:00:00 2001 From: brother7 <7brother7@gmail.com> Date: Wed, 16 Sep 2026 14:34:49 +0800 Subject: [PATCH] test(design): expose video preparation and playback flow gaps --- .../20260916-design-media-client-a7d3e9c1.md | 14 +- .../diagnostics/design-video-flow.test.tsx | 159 ++++++++++++++++++ .../diagnostics/vitest.design-flow.config.ts | 11 ++ 3 files changed, 182 insertions(+), 2 deletions(-) create mode 100644 scripts/diagnostics/design-video-flow.test.tsx create mode 100644 scripts/diagnostics/vitest.design-flow.config.ts diff --git a/.project-docs/30-worklog/tasks/20260916-design-media-client-a7d3e9c1.md b/.project-docs/30-worklog/tasks/20260916-design-media-client-a7d3e9c1.md index 374521c..31eff68 100644 --- a/.project-docs/30-worklog/tasks/20260916-design-media-client-a7d3e9c1.md +++ b/.project-docs/30-worklog/tasks/20260916-design-media-client-a7d3e9c1.md @@ -12,7 +12,7 @@ ## Scope -- Diagnosis only: YouthCreationCard media and reference controls, localized blockers, image workspace direct-edit command path; own this task record only. +- Diagnosis only: YouthCreationCard media and reference controls, localized blockers, image workspace direct-edit command path. Resumed for user-requested flow verification; own this record and scripts/diagnostics/design-video-flow.test.tsx plus its isolated Vitest config. Use actual React components/store with a process-local bridge to the real server compiler, mock only transport/uploads; no product edits. ## Intent And Constraints @@ -27,18 +27,28 @@ - The active conversation mounts YouthCreationCard. FineTuneDrawer and DesignFormPane still contain professional frame/Shot controls but are not imported by any page in this checkout. The visible form cannot resolve the two explicit video blockers itself. - Generic unsupported messages hide reference_role_unsupported and text_to_video_unsupported behind identical wording. The prepare button disables on any blocker, so the user has no actionable route from those messages. 方案已更新 reports save state, not production readiness. - Related reachable defects: video duration control accepts 1-30 while current server schema accepts 2-15; quantity control accepts 1-4 while production capabilities cap each medium at 1. Screenshot values 6 and 1 are valid, so neither explains this incident. +- Resumed user-flow verification now exercises actual React controls and Zustand command dispatch. A local stdin bridge invokes the server's actual kernel, production capability constructor and compiler. Normal user steps produce direct_edit -> direct_edit -> request_quote and the four screenshot blockers, without any preparation reasoning step. The positive control supplies a complete semantic draft, then the same UI/store reaches the confirm button and emits confirm_generation. +- Additional confirmed end-of-flow defect: active DesignHistoryRail mounts DesignPlanHistory, which renders videos through DesignAssetThumbnail. The video elements lack controls and there is no play/view-video button or click handler; only download is offered. A successful video presentation fixture fails the in-app playback-entry assertion. DesignProductionPanel has a controlled player but that panel is not mounted by the active page. +- Additional source-only availability finding: ImageCanvas/index.tsx uses bootstrap.capabilities.generation, not the video flag, and YouthCreationCard always offers video. Server can legitimately expose image with video disabled, so this UI entry does not establish that video is available. +- History currently provides image preview/download but no explicit use-this-image-to-make-video action. Together with generic blocker wording and the unmounted first-frame/Shot controls, users cannot reliably recover via the visible plan controls. ## Verification -- Read YouthCreationCard media/upload/button handlers, image-workspace store command construction, youth-issue-copy, production capability constructor and compiler in the paired server task. No browser/client unit suite run; no deployed version verified. +- Initial pass read YouthCreationCard media/upload/button handlers, image-workspace store command construction, youth-issue-copy, production capability constructor and compiler in the paired server task. Follow-up below adds component execution; deployed version remains unverified. - Paired server command python -m scripts.diagnostics.design_media_blockers reproduces the screenshot-shaped four blockers, exit 1 intentionally. --compare exits 0 with eight scenario assertions: same reference as first_frame leaves only missing Shot; one Shot makes the plan ready. Returning to image alone leaves a hidden video-duration blocker and unsupported-reference blocker. - check_project_docs and task-aware check_doc_drift both PASS. Only this task record is changed since the recorded base; no product changes. Staged whitespace check is performed before the diagnostic handoff commit. +- Follow-up command: with DESIGN_DIAGNOSTIC_PYTHON pointing to the server project Python and DESIGN_DIAGNOSTIC_SERVER to the companion worktree, run node_modules/.bin/vitest run --config scripts/diagnostics/vitest.design-flow.config.ts --reporter=verbose. Final run: 1 passed, 2 failed, 17.57 seconds. Failures intentionally assert missing product acceptance; this explicitly selected diagnostic file is outside the normal unit-suite include paths. +- Initial bridge setup failures (Windows stdin decoding and a loose pre-existing client fixture missing server Field Decisions) were corrected in the diagnostic only: Python runs with UTF-8 and the baseline Specification is created by the actual server kernel. Final failures occur at the targeted user-flow assertions, not fixture setup. +- Existing root node_modules reused through an ignored junction in this owned worktree; no packages installed or manifests changed. Tests use jsdom, simulated upload/HTTP/pricing, local server processes and a synthetic completed-video presentation fixture. No desktop app, real LLM, gateway transport, video playback bytes, production API or paid generation used. +- Server companion verified the complete-draft quote -> one confirmed Task -> real Worker -> running/succeeded -> ready video Asset/public event path with simulated provider, output materializer and ledger; its targeted flow/adapter/dependency suite passed 6 tests. +- ESLint passed for both diagnostic TypeScript files; task-aware documentation drift passed. Cleanup of the temporary ignored dependency junction was rejected by the execution policy before running; no alternate deletion attempted. The link remains at this owned worktree/node_modules, pointing to the existing root node_modules. No product source file changed. ## Follow-ups - Await authorization for implementation. Keep the simple youth-facing UI, but make selecting image/video produce a coherent current plan; route semantic motion arrangement through the Agent and handle deterministic binding/cleanup in application code. Do not expose more professional form fields as the default fix. - Reference selection must distinguish actual supported uses from a mere text mention. For video, explicitly identify which asset starts the clip; for image, align selectable uses with real backend capabilities. Do not silently drop references. - Improve actionable capability-aware messages and controls. Separately verify installed client/server versions if deployed behavior differs from this local reproduction. +- Flow acceptance is NOT passed: fix both prepare-entry and video-playback dead ends, then validate available-video gating, starting-image reuse, Agent motion preparation, quote/explicit confirm, visible progress, result recovery and actual playback together. Real-provider validation must be separately approved; no claim of production video availability from mock-backed passes. ## Promotion Candidates diff --git a/scripts/diagnostics/design-video-flow.test.tsx b/scripts/diagnostics/design-video-flow.test.tsx new file mode 100644 index 0000000..b2d65ab --- /dev/null +++ b/scripts/diagnostics/design-video-flow.test.tsx @@ -0,0 +1,159 @@ +/** Offline connectivity diagnostic, intentionally red; not in the normal unit suite. + * Actual React controls + Zustand commands -> stdin bridge -> actual Python + * Specification kernel / production capability builder / compiler. Upload and + * transport are simulated. No live gateway, reasoner, paid task or provider. + */ +import { spawnSync } from 'node:child_process'; +import { act, cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { afterEach, beforeEach, expect, it, vi } from 'vitest'; +import { YouthCreationCard } from '@/pages/ImageCanvas/YouthCreationCard'; +import { DesignPlanHistory } from '@/pages/ImageCanvas/DesignPlanHistory'; +import { useImageWorkspaceStore } from '@/stores/image-workspace'; +import { + ImageWorkspaceApiError, submitImageWorkspaceCommand, uploadImageWorkspaceAsset, +} from '@/lib/image-workspace'; +import { + designFormFixture, designQuoteFixture, designWorkspaceFixture, +} from '../../tests/fixtures/design-workspace-v2'; +import type { + DesignAsset, DesignCompilationIssue, DesignSpecification, DesignUserFieldOperation, +} from '../../shared/image-workspace'; + +vi.mock('@/lib/image-workspace', async (original) => ({ + ...await original(), + submitImageWorkspaceCommand: vi.fn(), + uploadImageWorkspaceAsset: vi.fn(), + resolveImageWorkspaceAssetUrl: vi.fn().mockResolvedValue('blob:offline-preview'), +})); +vi.mock('@/lib/host-api', async (original) => ({ + ...await original(), setDesktopBackgroundLease: vi.fn(), +})); +vi.mock('@/stores/auth', () => ({ + useAuthStore: { getState: () => ({ isAuthenticated: () => true }) }, +})); + +function serverStep(specification: DesignSpecification | null, operations: DesignUserFieldOperation[]) { + const result = spawnSync(process.env.DESIGN_DIAGNOSTIC_PYTHON!, [ + '-X', 'utf8', '-m', 'scripts.diagnostics.design_media_blockers', '--json-step', + ], { + cwd: process.env.DESIGN_DIAGNOSTIC_SERVER, + input: JSON.stringify({ specification, operations }), encoding: 'utf8', + }); + if (result.status !== 0) throw new Error(result.stderr || String(result.error)); + return JSON.parse(result.stdout) as { + specification: DesignSpecification; ready: boolean; blockers: DesignCompilationIssue[]; + }; +} + +const asset: DesignAsset = { + assetId: 'image-1', workspaceId: 'workspace-1', role: 'uploaded', mediaType: 'image', + mimeType: 'image/png', width: 720, height: 1280, durationMilliseconds: null, + generationTaskId: null, createdAt: '2026-09-16T00:00:00Z', + contentPath: '/api/design/assets/image-1/content', +}; + +function LiveCard() { + const workspace = useImageWorkspaceStore((state) => state.workspace)!; + const blockers = useImageWorkspaceStore((state) => state.quoteBlockers); + return ; +} + +beforeEach(() => { + vi.clearAllMocks(); + useImageWorkspaceStore.getState().reset(); +}); +afterEach(cleanup); + +it.each([ + { label: 'current UI payload', complete: false }, + { label: 'control with a complete semantic video draft supplied', complete: true }, +])('reaches video confirmation: $label', async ({ complete }) => { + const initial = serverStep(null, [ + { kind: 'set', path: 'intent.media', value: 'image' }, + { kind: 'set', path: 'content.concept', value: '沙漠里的小动物围圈玩游戏，天上的云在飘。' }, + { kind: 'set', path: 'output.aspect_ratio', value: '9:16' }, + ]); + let workspace = designWorkspaceFixture({ form: designFormFixture({ + specification: initial.specification, activeQuotes: [], + }) }); + const commands: string[] = []; + expect(initial.ready).toBe(true); + useImageWorkspaceStore.setState({ + workspace, activeWorkspaceId: workspace.workspace.workspaceId, status: 'ready', + }); + vi.mocked(uploadImageWorkspaceAsset).mockResolvedValue(asset); + vi.mocked(submitImageWorkspaceCommand).mockImplementation(async (command) => { + commands.push(command.kind === 'apply_input' ? command.input.kind : command.kind); + const operations = command.kind === 'apply_input' && command.input.kind === 'direct_edit' + ? command.input.operations : []; + const step = serverStep(workspace.form.specification, operations); + if (command.kind === 'request_quote' && !step.ready) { + useImageWorkspaceStore.setState({ quoteBlockers: step.blockers }); + throw new ImageWorkspaceApiError(409, 'design_quote_blocked', 'offline blocked'); + } + workspace = { + ...workspace, assets: operations.some((operation) => operation.path === 'references') + ? [asset] : workspace.assets, + workspace: { ...workspace.workspace, workspaceViewRevision: workspace.workspace.workspaceViewRevision + 1 }, + form: { ...workspace.form, specification: step.specification, + specificationRevision: workspace.form.specificationRevision + 1, + directionRevision: workspace.form.directionRevision + 1 }, + }; + if (command.kind === 'request_quote') { + const quote = designQuoteFixture(); + workspace.form.activeQuotes = [{ ...quote, medium: 'video', + specificationRevision: workspace.form.specificationRevision, + outputSummary: { ...quote.outputSummary, medium: 'video', aspectRatio: '9:16', + durationSeconds: 6, deliveryFormat: 'mp4' } }]; + } + return { workspace, runId: 'offline-run', clientOperationId: command.clientOperationId }; + }); + render(); + fireEvent.change(screen.getByRole('combobox', { name: '类型' }), { target: { value: 'video' } }); + await waitFor(() => expect(useImageWorkspaceStore.getState().workspace?.form.specification.values.intent.media).toBe('video')); + fireEvent.change(screen.getByLabelText('选择参考图片'), { + target: { files: [new File(['offline-upload'], 'start.png', { type: 'image/png' })] }, + }); + await screen.findByText('已引用'); + if (complete) { + // Control input, not a claim that today's Agent/UI performs this transition. + const reference = workspace.form.specification.values.references[0]; + const step = serverStep(workspace.form.specification, [ + { kind: 'set', path: 'references', value: [{ ...reference, role: 'first_frame' }] }, + { kind: 'set', path: 'video.shots', value: [{ id: 'one', story_beat: '小动物围圈游戏，云缓慢飘过。' }] }, + ]); + workspace = { ...workspace, form: { ...workspace.form, specification: step.specification } }; + act(() => useImageWorkspaceStore.setState({ workspace })); + } + await waitFor(() => expect(screen.getByRole('button', { name: '准备制作方案' })).toBeEnabled()); + fireEvent.click(screen.getByRole('button', { name: '准备制作方案' })); + await waitFor(() => expect(commands).toContain('request_quote')); + console.log('FLOW_COMMANDS', commands); + console.log('FLOW_BLOCKERS', useImageWorkspaceStore.getState().quoteBlockers.map((item) => item.code)); + expect(useImageWorkspaceStore.getState().quoteBlockers).toEqual([]); + expect(screen.getByRole('button', { name: '确认并开始制作' })).toBeEnabled(); + fireEvent.click(screen.getByRole('button', { name: '确认并开始制作' })); + await waitFor(() => expect(commands).toContain('confirm_generation')); +}); + +it('lets a user play a completed video directly in the active history rail', async () => { + const video: DesignAsset = { ...asset, assetId: 'video-1', role: 'generated', + mediaType: 'video', mimeType: 'video/mp4', durationMilliseconds: 6000, + generationTaskId: 'task-video', contentPath: '/api/design/assets/video-1/content' }; + const workspace = designWorkspaceFixture(); + workspace.assets = [video]; + // Presentation fixture only; no claim that the provider generated these bytes. + workspace.tasks = [{ + taskId: 'task-video', status: 'succeeded', medium: 'video', createdAt: video.createdAt, + maximumCustomerChargeAtoms: 500, resultAssetIds: ['video-1'], issues: [], + outputSummary: { medium: 'video', aspectRatio: '9:16', outputCount: 1, durationSeconds: 6 }, + progress: { stage: 'completed', completedRequiredSteps: 2, totalRequiredSteps: 2 }, + } as typeof workspace.tasks[number]]; + const { container } = render(); + await screen.findAllByLabelText('视频作品预览'); + fireEvent.click(screen.getByText('制作记录')); + const players = [...container.querySelectorAll('video')]; + expect(players.length).toBeGreaterThan(0); + expect(players.some((player) => player.controls) + || screen.queryByRole('button', { name: /播放|查看视频/ }) !== null).toBe(true); +}); diff --git a/scripts/diagnostics/vitest.design-flow.config.ts b/scripts/diagnostics/vitest.design-flow.config.ts new file mode 100644 index 0000000..03b1a71 --- /dev/null +++ b/scripts/diagnostics/vitest.design-flow.config.ts @@ -0,0 +1,11 @@ +import { defineConfig } from 'vitest/config'; +import base from '../../vitest.config'; + +export default defineConfig({ + ...base, + test: { + ...base.test, + include: ['scripts/diagnostics/design-video-flow.test.tsx'], + testTimeout: 20000, + }, +});