215 lines
11 KiB
Markdown
215 lines
11 KiB
Markdown
# MAIC Editor Agent — `regenerate_scene` (next-release capability widening)
|
|
|
|
- Date: 2026-06-21
|
|
- Base branch: `feat/maic-editor-agent-v0`
|
|
- Status: design approved, pending spec review → implementation plan
|
|
|
|
## 1. Background
|
|
|
|
The MAIC Editor Agent v0 (`feat/maic-editor-agent-v0`) stands up a server-side
|
|
`pi` Agent that streams `AgentEvent`s to the editor sidebar over SSE. Its
|
|
capability surface is deliberately one tool — `regenerate_scene_actions` — gated
|
|
by a tool allowlist (`V0_ALLOWLIST`). The design philosophy is **capability =
|
|
allowlist**: widening capability means adding tools to the allowlist, never
|
|
hardcoding a workflow.
|
|
|
|
Two limitations make v0 a demo rather than a usable editing assistant:
|
|
|
|
1. The agent can only regenerate **narration/actions**, not slide **content**.
|
|
The system prompt hard-refuses any content edit; the user must hand-edit the
|
|
canvas themselves.
|
|
2. The agent never **sees** the current scene. Scene context (outline / content)
|
|
is injected into the tool's deps (`getSceneContext`) but is invisible to the
|
|
model. So even `regenerate_scene_actions` is effectively a blind re-roll — the
|
|
model cannot translate "this slide is too dense" into a precise instruction
|
|
because it does not know what is on the slide.
|
|
|
|
## 2. Goal
|
|
|
|
Reach **basic user usability** in the next release by letting the agent
|
|
**read** a slide and then **regenerate the whole slide (content + narration)
|
|
under a natural-language instruction**, with a safe apply/restore story.
|
|
|
|
## 3. Scope decisions (locked)
|
|
|
|
| Decision | Choice | Rationale |
|
|
|---|---|---|
|
|
| Capability class | **Regeneration only** — no add / delete / reorder / insert | Structural ops are one click away on the canvas + NavRail; the agent's differentiated value is generation, not clicking structural buttons. Keeps allowlist + blast radius tight. |
|
|
| Regeneration granularity | **Whole page** (content + actions) in one tool | Simplest mental model — "redo this slide". Reuses the standard `generateSceneContent` → `generateSceneActions` steps. |
|
|
| Steerability | **Instruction-driven** (every regeneration accepts an optional NL instruction) | Without an instruction, regeneration is a slot machine, not a tool. This is the usability unlock. |
|
|
| Apply model | **Apply directly + explicit "还原到重生成前" button** on the tool card | Snapshots the pre-regenerate scene; does not rely on the user remembering Ctrl+Z; lighter than a full preview/approve surface. |
|
|
| Scene-type scope | **slide only** | Most pages are slides; lowest risk; matches the restrained v0 posture. Other types: the agent honestly says "not supported yet". |
|
|
| Agent ↔ content access | **`read_scene_content` tool** (read-then-act) | The agent reads the slide to reason and craft a precise instruction. Read-only is zero-risk and yields free Q&A ("what's on this slide?"). |
|
|
|
|
### Explicit non-goals (next release)
|
|
|
|
- No new-slide / delete / reorder / duplicate capability.
|
|
- No element-level or region-select local editing (a separate inline local-edit
|
|
track, out of scope here).
|
|
- No quiz / interactive / PBL regeneration.
|
|
- No multi-scene / deck-wide operations.
|
|
- No preview/diff surface beyond the restore button.
|
|
|
|
## 4. Architecture
|
|
|
|
The next-release allowlist grows from 1 tool to 3:
|
|
|
|
| Tool | Kind | Purpose |
|
|
|---|---|---|
|
|
| `read_scene_content` | read (new) | Return the current slide's content + outline for a `sceneId` so the model can reason, answer questions, and distil a precise regenerate instruction. |
|
|
| `regenerate_scene` | write (new) | Regenerate the whole slide (content + actions) for a `sceneId`, using **trusted injected content as baseline** + the agent's NL instruction → `generateSceneContent` + `generateSceneActions`. slide-only. |
|
|
| `regenerate_scene_actions` | write (existing) | Unchanged. Regenerate narration/actions only. |
|
|
|
|
### 4.1 Trust boundary (carries v0's "model is not a data source" rule)
|
|
|
|
- `read_scene_content` surfaces content **to the model's reasoning context**, on
|
|
demand, pulled from the same client-injected `sceneContextMap` the route
|
|
already receives. This replaces "pre-stuff everything" with "pull what you
|
|
need" — strictly more token-efficient.
|
|
- `regenerate_scene` **executes** against the trusted injected content resolved
|
|
by `sceneId` (never against content the model retyped). The model contributes
|
|
only `{ sceneId?, instruction? }`. The instruction is NL intent, not content.
|
|
|
|
So: the model reads content to **write a good instruction**; the generation
|
|
consumes content from the **trusted source**. Both paths are needed.
|
|
|
|
## 5. Components & changes
|
|
|
|
### 5.1 `read_scene_content` tool — `lib/agent/tools/read-scene-content.ts` (new)
|
|
|
|
- Factory `makeReadSceneContentTool(deps)` mirroring the
|
|
`makeRegenerateSceneActionsTool` shape.
|
|
- Args (typebox): `{ sceneId?: string }` — defaults to the active scene.
|
|
- `execute`: resolves `getSceneContext(sceneId)` and returns a compact,
|
|
model-readable projection of `{ title, type, outline, content }`. For
|
|
non-slide types it still returns a readable summary (read is safe for all
|
|
types even though regenerate is slide-only).
|
|
- Reuses the existing `SceneContext` deps; no new data plumbing.
|
|
|
|
### 5.2 `regenerate_scene` tool — `lib/agent/tools/regenerate-scene.ts` (new)
|
|
|
|
- Factory `makeRegenerateSceneTool({ aiCall, getSceneContext })`, sibling to
|
|
`regenerate-scene-actions.ts`.
|
|
- Args (typebox): `{ sceneId?: string, instruction?: string }`.
|
|
- `execute`:
|
|
1. Resolve trusted context (`outline`, `allOutlines`, current `content`,
|
|
`stageId`) via `getSceneContext`.
|
|
2. **Guard**: if `outline.type !== 'slide'`, return a typed refusal result
|
|
(the model relays "only slides are supported yet"). No generation.
|
|
3. Call the content pipeline (see 5.4) with the current content as baseline +
|
|
`instruction` as `editDirective`, then regenerate actions to match.
|
|
4. Return `{ content, actions }` in `details`, same `tool_execution_end`
|
|
contract as `regenerate_scene_actions`, so the client applies them.
|
|
|
|
### 5.3 Registry / allowlist — `lib/agent/tools/registry.ts`
|
|
|
|
- `ToolsetDeps` unchanged shape (same `SceneContext` deps reused by all three).
|
|
- `buildToolset` returns `[read_scene_content, regenerate_scene,
|
|
regenerate_scene_actions]`.
|
|
- `V0_ALLOWLIST` → `new Set(['read_scene_content', 'regenerate_scene',
|
|
'regenerate_scene_actions'])`.
|
|
|
|
### 5.4 Generation pipeline — `@openmaic/generation`
|
|
|
|
`generateSceneContent` already threads a `languageDirective` into its prompt;
|
|
this is the template for the new inputs. Add to `SceneContentOptions`:
|
|
|
|
- `editDirective?: string` — the agent's NL instruction, woven into the slide
|
|
content prompt the same way `languageDirective` is.
|
|
- `baselineContent?: GeneratedSlideContent` — the current slide content, fed as
|
|
the edit baseline so content-specific instructions ("drop the 2nd bullet",
|
|
"sharpen the title") operate on the real slide rather than re-rolling from
|
|
outline. When absent, behaviour is today's outline-based generation.
|
|
|
|
Only the **slide** branch of `generateSceneContent` consumes the baseline in the
|
|
next release; other branches ignore it (slide-only scope).
|
|
|
|
### 5.5 Server route — `app/api/agent/edit/route.ts`
|
|
|
|
- Wire the two new tools through `buildToolset` (no new request fields needed —
|
|
`sceneContextMap` already carries everything).
|
|
- The existing `aiCall` (resolved `maic-agent` model) is shared by all tools.
|
|
|
|
### 5.6 System prompt — `lib/agent/runtime/build-agent.ts`
|
|
|
|
Rewrite the capability-boundary paragraph from "you can ONLY regenerate
|
|
narration" to:
|
|
|
|
- You can **read** the current slide (`read_scene_content`) to understand it and
|
|
answer questions about it.
|
|
- You can **regenerate the whole slide** (content + narration) for **slide-type**
|
|
scenes, following the user's instruction (`regenerate_scene`); you can also
|
|
regenerate **only the narration** (`regenerate_scene_actions`).
|
|
- You still CANNOT: add / delete / reorder / duplicate slides; edit quiz,
|
|
interactive (GenUI) or whiteboard content; regenerate non-slide scenes — say
|
|
so honestly and suggest the user edits those on the canvas.
|
|
- Regeneration rebuilds the slide wholesale; it cannot guarantee specific
|
|
existing elements/cues survive unchanged.
|
|
|
|
### 5.7 Client — `components/edit/AgentPanel/*` + `lib/agent/client/*`
|
|
|
|
- New `regenerate-scene-tool-ui.tsx` tool card (sibling to the existing
|
|
`regenerate-tool-ui.tsx`): on `tool_execution_end`,
|
|
1. **snapshot** the scene's pre-regenerate `{ content, actions }` into the
|
|
card's local state,
|
|
2. apply the new `{ content, actions }` into `useStageStore`,
|
|
3. render a **"还原到重生成前"** button that re-applies the snapshot.
|
|
- `read_scene_content` needs no special card — its result feeds the model's next
|
|
turn; the assistant reply renders normally.
|
|
- `use-agent-runtime` apply logic generalised to dispatch by tool name
|
|
(regenerate_scene → content+actions; regenerate_scene_actions → actions only).
|
|
|
|
## 6. Data flow (happy path)
|
|
|
|
```
|
|
user: "这页太满了，精简成3个要点"
|
|
→ agent calls read_scene_content(activeScene)
|
|
→ tool returns {title, outline, content(7 bullets)}
|
|
→ agent reasons, calls regenerate_scene(sceneId, instruction:
|
|
"精简为3点，保留 X/Y/Z，删掉细节举例")
|
|
→ tool: guard slide ✓; generateSceneContent(outline, aiCall,
|
|
{baselineContent: current, editDirective: instruction})
|
|
→ generateSceneActions(...) ; returns {content, actions}
|
|
→ client tool card: snapshot old → apply new → show 还原 button
|
|
→ agent reply: "已精简为3个要点，旁白也对齐了。"
|
|
```
|
|
|
|
## 7. Guardrails
|
|
|
|
- Master gate unchanged: `isMaicEditorEnabled()` 404s the route when off.
|
|
- `beforeToolCall` allowlist gate now admits the 3 tools; everything else denied.
|
|
- `regenerate_scene` hard-refuses non-slide types inside `execute`.
|
|
- Quota: unchanged v0 unlimited stub (shape aligned to the planned quota hook);
|
|
regeneration is the expensive path, so this is where the quota hook will bite.
|
|
- Concurrency / superseded-run guards from v0 (stop-then-resend) apply unchanged.
|
|
|
|
## 8. Testing
|
|
|
|
Mirror the existing `tests/lib/agent/**` coverage:
|
|
|
|
- `read-scene-content.test.ts`: returns trusted projection; defaults to active
|
|
scene; safe on non-slide types.
|
|
- `regenerate-scene.test.ts`: slide → calls content+actions with baseline +
|
|
editDirective; non-slide → typed refusal, no generation; model-supplied
|
|
content is ignored (trust boundary).
|
|
- `scene-generator` content test: `editDirective` + `baselineContent` reach the
|
|
slide prompt; absent → today's behaviour unchanged (regression guard).
|
|
- `use-agent-runtime` client test: regenerate_scene applies content+actions and
|
|
the restore snapshot round-trips.
|
|
- allowlist test updated to expect the 3-tool set.
|
|
|
|
## 9. Risks & open points
|
|
|
|
- **Baseline-edit prompt quality**: feeding current content + instruction into
|
|
the slide prompt is the one non-trivial prompt change. Needs eval on a few
|
|
representative instructions (精简 / 加例子 / 改标题 / 删某点) before release.
|
|
- **Wholesale replace surprises**: even with the restore button, a user who
|
|
hand-edited then asked for an unrelated regeneration loses those edits until
|
|
they hit restore. Acceptable for "basic usability"; preview/approve is the
|
|
later upgrade.
|
|
- **Token growth**: `read_scene_content` puts slide content into the model
|
|
context. Single-slide content is small; do not let the agent read all siblings
|
|
by default.
|
|
- **Scene-type creep**: quiz/interactive are intentionally deferred; resist
|
|
widening the slide-only guard in this release.
|