merge: enable voice input for fresh agent conversations

This commit is contained in:
2026-08-17 14:42:25 +08:00
4 changed files with 146 additions and 1 deletions

View File

@@ -4,6 +4,7 @@ This file is the integrated default-branch snapshot. Feature tasks record progre
## Integrated Through
- `b6148a5`: AI Programming voice capture is available after an Agent is selected but before the lazy first OpenCode session exists; transcription fills the composer draft without creating an empty session, while runtime, loading, transcribing, busy, and recording guards remain unchanged.
- `4013edc` / `3b799af`: integrated per-user Code/Canvas/Learning/Robot entry policy from Works Square, projected by Electron Main as four booleans and enforced before disabled module routes initialize.
- `01bee31`: enabled AI Learning course catalog/generation/download/playback, Main-owned cloud/runtime bridges, verified external OpenMAIC player-artifact packaging, account profile reuse, removal of the transient `game-engine` Skill, and project-root `planning-with-files` output from the authoritative remote main. The merge hardens this with strict DTO/error projection, account-isolated local state, bounded same-origin downloads/packages, a nonce-protected account-bound player HTTP session, and an exact-source/origin single-document iframe bridge.
- `26b52d7`: Canvas Prompt Museum, editable server-priced generation Quotes, project deletion/task-detail workflow, cloud-default Canvas development entry, and Chinese-only UI consolidation from the authoritative remote main. Its transient bundled `game-engine` Skill is superseded by `01bee31`.

View File

@@ -12,6 +12,7 @@
## Scope
- On 2026-08-17, resume the existing Integration owner at the user's request to merge reviewed AI Programming fresh-Agent voice-input source `b6148a5` into local `main`, preserve lazy first-session creation and existing voice guards, run merged-tree verification and independent review, and keep remote push outside this resumption.
- On 2026-08-17, resume the existing Integration owner after the user's explicit takeover authorization to preserve the task's existing uncommitted record, merge reviewed Makelore module-access source tip `3b799af` into local `main`, promote its accepted per-user entry-policy facts, run merged-tree verification and independent review, and keep remote push outside this resumption unless separately authorized.
- On 2026-08-16, resume the existing Integration owner after the user restored remote credentials: fetch authoritative `origin/main`, complete the already-started merge of remote tip `26b52d7`, preserve both remote Canvas/Prompt Museum work and local Robot hotspot work, run merged-tree verification and independent review, then perform a normal non-forced push and verify remote-tip equality.
- On 2026-08-16, resume the existing Integration owner to merge reviewed cross-platform in-app Robot hotspot connection source `c1326a2` into local `main`, supersede only ADR-002's manual operating-system hotspot-selection step, preserve firmware/credential/Binding boundaries, and keep remote push outside this resumption.
@@ -54,6 +55,7 @@
- The user has now explicitly authorized page-owned selection and connection of nearby open `Xiaozhi-*` provisioning hotspots on both Windows and macOS. This supersedes only manual operating-system hotspot selection; discovery remains unauthenticated convenience, Main remains the sole native-network owner, and signed macOS plus physical-Robot smoke remain release gates.
- The latest instruction explicitly authorizes fetching, semantically resolving conflicts, and pushing `main`. Use the existing merge topology; do not rebase, reset, stash, discard remote/local commits, or force-push. Preserve remote Prompt Museum/Canvas/Chinese-only/Skill behavior alongside the already reviewed local Robot native hotspot path and `koffi` packaging.
- The 2026-08-17 takeover confirmation authorizes continuing this existing Integration owner while preserving its uncommitted task history. For module access, source-task records remain read-only; the policy is a client entry/initialization gate rather than API authorization, and release acceptance requires the matching Works migration/API, a newly packaged client, and real-account four-module smoke.
- The fresh-Agent voice-input request authorizes a local `main` merge only. It does not authorize remote push, Canvas voice enablement, eager OpenCode runtime/session creation, or changes to the existing Renderer -> Host API -> Electron Main -> Works Square speech boundary.
## Project Context Loaded
@@ -101,6 +103,14 @@ Relevant understanding:
Gate result:
- Passed.
### 2026-08-17 Fresh-Agent Voice Input Integration Resume
- Reused the existing Integration owner because it still exclusively owns clean local `main` at `9fd9a7761d3872f25e853ac1b059e55251c6ae8f`; `task_context.py touch` refreshed the reservation and registry doctor passed.
- Formed reviewed source commit `b6148a5cb45e70a75c055a57f8b269f24c619a6a` as the exact direct child of local `main`. Its feature task is `ready_for_integration`, its worktree is clean, and final read-only Sol review returned `PASS` with no blocking Standards or Spec findings.
- Read the source outcome, verification, follow-ups, and promotion candidates against the integrated AI Programming session lifecycle. The source restores the existing lazy-session contract and proposes no architecture, domain, or product-direction change.
- Started a normal `--no-ff --no-commit` merge of `b6148a5`; Git reported no textual conflicts. The source task record remains reachable on its source commit/branch and is excluded from the integrated `main` tree.
- Gate result: Passed for local no-ff merge, minimal current-state reconciliation, proportionate merged-tree verification, and independent final review. Remote push remains outside this resumption.
### 2026-08-17 Per-User Module Access Integration Resume
- Reused the existing Integration owner after the user explicitly authorized takeover while preserving its uncommitted record. Registry status still identifies task `20260813-sync-push-main-9c2f71`, mode `integration`, branch `main`, and worktree `D:\Datas\OthersProjects\makelore`; no new task context was created.
@@ -186,6 +196,13 @@ Gate result:
## Plan
### 2026-08-17 Fresh-Agent Voice Input Integration Plan
1. Merge reviewed source `b6148a5` into local `main` with a normal no-ff merge, preserving source history while excluding the source-owned task record from the final tree.
2. Record the integrated behavior in current state without changing architecture, domain rules, README, Canvas, Host API, Main, or Preload contracts.
3. Run the focused fresh-Agent/lazy-session/voice regressions, the complete ChatPanel test file and full unit suite, typecheck, scoped lint, production build, document gates, whitespace/unmerged checks, and exact topology checks.
4. Obtain an independent read-only Sol `PASS`/`FAIL` review, create the verified local merge commit, record its exact topology, and do not push remotely.
### 2026-08-16 Remote `26b52d7` Synchronization Plan
1. Complete the existing merge against freshly fetched `origin/main=26b52d7`, resolving README to preserve both remote Canvas Prompt Museum and local Robot hotspot behavior, then confirm there are no unmerged entries or dependency regressions.
@@ -265,6 +282,9 @@ Gate result:
## Outcome
- Independent final staged-merge review returned `PASS` with no P0-P3 findings. It confirmed the exact no-ff topology, four-file scope, source-task-record exclusion, lazy-session behavior, preserved guards, regression strength, and document ownership boundaries.
- The staged voice-input merge preserves the exact first-parent/second-parent topology `9fd9a77` + `b6148a5`, contains no unmerged entries or unstaged changes, and has passed merged-tree product, build, and project-document gates. It is ready for independent final review before the local merge commit is created.
- On 2026-08-17, formed reviewed voice-input source commit `b6148a5` and staged a conflict-free normal no-ff merge onto local `main=9fd9a77`. The product diff is limited to the ChatPanel voice predicate and its fresh-Agent component regression; the source-owned task record is excluded from the `main` tree.
- Created normal no-ff module-access merge commit `4013edcbbc982fcbef0a0ff6287c5307cd87bf8a` with preserved integration-history commit `eb16f73e4dece86a193dee0f797a58d77869470f` as first parent and reviewed source tip `3b799af17370dfa6c4c995c3fcbab8f2d84c3dda` as second parent. Canonical reconciliation records the Main-owned four-boolean projection, default-open compatibility, `design``painting`, disabled card/root/deep/alias guards before initialization, Code policy hydration, terminal `401` dual-session cleanup, global `/settings`, and the client-entry-only security boundary.
- A mandatory pre-push fetch found authoritative `origin/main` had advanced to `01bee3188be4b03b6b358c2da60f1f9ed22d707f`, so no stale push was attempted. Started a second normal `--no-ff --no-commit` merge with reviewed local merge `bb16c1d12a24b2957149db960edba6e6660691aa` as first parent and `01bee31` as second parent; README was the only textual conflict and was reconciled to preserve both the enabled Learning module and the detailed Robot hotspot flow.
- Preserved the remote Learning product scope, four-module navigation, profile reuse, removal of `game-engine`, and project-root `planning-with-files` output. The merged tree no longer depends on a sibling OpenMAIC checkout: CI/release packaging requires the fixed manifest artifact, while an explicit local source remains development-only.
@@ -389,6 +409,12 @@ Gate result:
## Verification
- 2026-08-17 independent final fresh-Agent voice integration review — `PASS`, no P0-P3 findings; exact staged topology/scope, source-record exclusion, lazy session creation, existing guards, test quality, and project-document boundaries were confirmed.
- 2026-08-17 fresh-Agent/lazy-session/voice focused selection — 5/5 passed; the complete `opencode-chat-panel.test.tsx` file passed 85/85 on the staged merged tree.
- 2026-08-17 staged merged-tree `pnpm test` — 175 files / 2048 tests passed.
- 2026-08-17 staged merged-tree `pnpm run typecheck` and scoped ESLint on `OpencodeChatPanel.tsx` plus its unit test — passed.
- 2026-08-17 staged merged-tree `pnpm run build:vite` — Renderer, Electron Main, and Preload passed (`3921`, `133`, and `1` modules transformed); only existing mixed-import and chunk-size warnings remain.
- `check_project_docs.py`, integration task-aware `check_doc_drift.py`, task registry doctor, staged whitespace, source-record exclusion, and unmerged-entry checks — passed before independent review.
- 2026-08-17 module-access merged-tree focused Vitest — 4 files / 69 tests passed.
- 2026-08-17 module-access merged-tree full Vitest — 175 files / 2047 tests passed.
- 2026-08-17 merged-tree TypeScript `tsc --noEmit` and scoped ESLint on all changed TypeScript/TSX files — passed.

View File

@@ -1040,7 +1040,7 @@ export function OpencodeChatPanel({ variant = 'main', navigationDraft, onOpenPro
);
const canStopSession = canUseSessions && Boolean(selectedSessionId) && sessionBusy && composerStopArmed;
const canUseVoiceInput = canUseSessions
&& Boolean(selectedSessionId)
&& Boolean(selectedSessionId || selectedAgentId)
&& !loading
&& voiceInputState !== 'transcribing'
&& (!sessionBusy || voiceInputState === 'recording');

View File

@@ -1010,6 +1010,124 @@ describe('OpencodeChatPanel', () => {
expect(screen.queryByText('课程槽位')).not.toBeInTheDocument();
});
it('records voice into a fresh Agent conversation without eagerly creating a session', async () => {
installMockVoiceWavTranscoder();
const activeProject = {
id: 'prj_fresh_voice',
path: 'D:/repo/fresh-voice',
name: 'fresh-voice',
createdAt: '2026-08-17T00:00:00.000Z',
updatedAt: '2026-08-17T00:00:00.000Z',
lastOpenedAt: '2026-08-17T00:00:00.000Z',
};
const config = createConfiguredProjectConfig();
const getUserMedia = vi.fn().mockResolvedValue({
getTracks: vi.fn(() => [{ stop: vi.fn() }]),
});
class MockMediaRecorder {
public ondataavailable: ((event: { data: Blob }) => void) | null = null;
public onstop: (() => void) | null = null;
public state: RecordingState = 'inactive';
static isTypeSupported(): boolean {
return true;
}
start(): void {
this.state = 'recording';
}
stop(): void {
this.state = 'inactive';
this.ondataavailable?.({ data: new Blob(['audio bytes'], { type: 'audio/webm' }) });
this.onstop?.();
}
}
Object.defineProperty(navigator, 'mediaDevices', {
configurable: true,
value: { getUserMedia },
});
Object.defineProperty(window, 'MediaRecorder', {
configurable: true,
value: MockMediaRecorder,
});
useAuthStore.setState({
initialized: true,
loading: false,
error: null,
accessToken: 'access-token',
refreshToken: null,
tokenType: 'Bearer',
expiresAt: Date.now() + 60_000,
user: {
username: 'student',
userId: '42',
tenantId: null,
deptId: null,
authorities: [],
},
});
useProjectConfigStore.setState({
configsByProjectId: { [activeProject.id]: config },
});
useOpencodeStore.setState({
status: { state: 'running', port: 4096, url: 'http://127.0.0.1:4096' },
projects: [activeProject],
activeProject,
sessions: [],
selectedSessionId: null,
sessionStatuses: {},
sessionMessages: [],
});
hostApiFetchMock.mockImplementation(async (path: string, init?: RequestInit) => {
if (path === '/api/works/speech/transcriptions' && init?.method === 'POST') {
return {
success: true,
transcription: {
text: 'voice before first message',
model: 'gpt-4o-mini-transcribe',
},
};
}
if (path === '/api/opencode/status') {
return { state: 'running', port: 4096, url: 'http://127.0.0.1:4096' };
}
if (path === '/api/opencode/projects') {
return { projects: [activeProject], activeProject };
}
if (path === '/api/opencode/config-summary') {
return { summary: { model: 'openai/gpt-5', smallModel: null, providerIds: ['openai'], providerCount: 1 } };
}
if (path === '/api/opencode/sessions') return { sessions: [] };
if (path === '/api/opencode/sessions/status') return { statuses: {} };
throw new Error(`Unexpected path ${path}`);
});
render(<OpencodeChatPanel variant="main" />);
fireEvent.click(await screen.findByTestId('project-agent-chat-game-art'));
await waitFor(() => {
expect(screen.getByTestId('project-agent-chat-game-art')).toHaveAttribute('aria-pressed', 'true');
});
expect(useOpencodeStore.getState().selectedSessionId).toBeNull();
const voiceButton = screen.getByTestId('opencode-voice-input-button');
expect(voiceButton).toBeEnabled();
fireEvent.click(voiceButton);
await waitFor(() => expect(getUserMedia).toHaveBeenCalledWith({ audio: true }));
fireEvent.click(voiceButton);
await waitFor(() => {
expect(screen.getByRole('textbox')).toHaveValue('voice before first message');
});
expect(hostApiFetchMock.mock.calls.some(([path, init]) => (
path === '/api/opencode/sessions'
&& (init as RequestInit | undefined)?.method === 'POST'
))).toBe(false);
});
it('keeps project-agent actions enabled when a pre-start status response is delayed', async () => {
const activeProject = { id: 'prj_delayed_status', path: 'D:/repo/delayed-status', name: 'delayed-status', createdAt: '2026-07-12T00:00:00.000Z', updatedAt: '2026-07-12T00:00:00.000Z', lastOpenedAt: '2026-07-12T00:00:00.000Z' };
const config = createConfiguredProjectConfig();