fix(browser): preserve chat focus during agent browsing
This commit is contained in:
@@ -0,0 +1,52 @@
|
||||
# Task: Fix agent browser stealing preview tab and failing when hidden
|
||||
|
||||
## Identity
|
||||
|
||||
- Task ID: 20260927-browser-tab-focus-5c92d7a1
|
||||
- Mode: Feature
|
||||
- Branch: codex/20260927-browser-tab-focus-5c92d7a1-browser-tab-focus
|
||||
- Worktree: D:\Datas\OthersProjects\.codex-worktrees\makelore\20260927-browser-tab-focus-5c92d7a1
|
||||
- Base commit: 220e89992b4d847eadcc64bd8faf058d0e25d36d
|
||||
- Owner: codex
|
||||
- Status: Ready for Integration
|
||||
|
||||
## Scope
|
||||
|
||||
- Main-owned Agent Browser viewport/tool readiness and Pi open presentation requests; targeted module, tool and Electron regressions; README behavior contract.
|
||||
- Isolated from packaging task 20260927-package-170-a7c3 and completed consultation-tool changes. No Yuxi/server changes.
|
||||
|
||||
## Intent And Constraints
|
||||
|
||||
- Switching from Work to chat must preserve the browser and allow Agent debugging. Reopening an established page must not repeatedly select Work.
|
||||
- Preserve first-open real viewport initialization, explicit close, project/module/window teardown, sandbox and current-project/CDP restrictions.
|
||||
- Concurrent Task Gate and Planning Gate passed at base 220e899; relevant browser architecture/domain and peer task records read. User feedback authorizes distinguishing tab hiding from closing.
|
||||
- First reproduce hidden-page CDP failure and repeated-open presentation; apply the smallest owning-layer fix, then verify module/tool regressions, actual Electron behavior, typecheck and build.
|
||||
- No live-app restart, model traffic, deployment, push, merge or worktree cleanup in this task.
|
||||
|
||||
## Outcome
|
||||
|
||||
- Reproduced two primary causes: hidden views with valid bounds were rejected as VIEWPORT_NOT_READY; every Pi open emitted agent-browser:show and waited for presentation, even when reusing an established view.
|
||||
- Tool readiness now checks the established viewport rather than current visibility. Initial no-bounds rejection, generation/project checks, debugger state and explicit teardown remain intact.
|
||||
- Pi open only requests foreground presentation for a new viewport. Reuse/navigation publishes state and preserves the user's selected tab.
|
||||
- Actual Electron testing uncovered an additional reachable failure: hidden Page.captureScreenshot timed out despite evaluate/navigation succeeding. Disabling background throttling and using fromSurface=false did not fix it. Electron capturePage with stayHidden:true/stayAwake:false requests a frame successfully; adapter now does that before hidden CDP capture, retaining CDP parameters, tab selection and composer focus without a permanent rendering override.
|
||||
- Updated README. No Renderer workaround, new runtime dependency, server/Yuxi edit, cloud traffic, installation, merge or push.
|
||||
|
||||
## Verification
|
||||
|
||||
- Red: pnpm exec vitest run tests/unit/agent-browser-core.test.ts tests/unit/pi-product-tools.test.ts -t 'keeps a view hidden|preserves the selected tab' --maxWorkers=2. Three expected failures: hidden CDP VIEWPORT_NOT_READY and repeated presentation requests.
|
||||
- Green: the same three regressions passed after fixing module/tool ownership boundaries.
|
||||
- Related regression suite: agent-browser-core, pi-product-tools, agent-browser-panel, agent-browser-electron-adapter and agent-browser-routes: 134 tests passed.
|
||||
- Actual Electron E2E: 4 passed (native Pi tool + sandboxed WebContentsView while switching tabs; repeated Work preparation/draft retention; failed preparation recovery; native publication preflight isolation/cleanup). The native browser test verifies initial presentation, repeated hidden open including navigation, evaluate, reload, console events, screenshot, unchanged generation, restored Work view and explicit close. Project/model APIs are fixtures; native browser/tool/Renderer components are real. No paid model involved.
|
||||
- pnpm run typecheck and pnpm run build:vite passed. Build has existing Browserslist age / chunk warnings.
|
||||
- Additional Main-only tsc check reports 66 existing diagnostics. TypeScript compiler-host comparison against base 220e899 with the three changed product sources replaced in memory produced the same 66 diagnostics, zero added, none in touched files. This optional broader check is not green; standard project typecheck is green.
|
||||
- Official Electron webContents.capturePage documentation confirms stayHidden/stayAwake options: https://www.electronjs.org/docs/latest/api/web-contents#contentscapturepagerect-opts . Behavior also verified on the repository's Electron 43.4.0.
|
||||
- Diff review and whitespace check performed; no debug probes or permanent background-throttling override remain.
|
||||
|
||||
## Follow-ups
|
||||
|
||||
- Await user-requested integration and a subsequent rebuilt client; current running installation is unchanged.
|
||||
- Main compiler's pre-existing 66 diagnostics are outside this browser fix; no errors added.
|
||||
|
||||
## Promotion Candidates
|
||||
|
||||
- Targets: .project-docs/20-architecture/module-map.md, .project-docs/20-architecture/data-flow.md and .project-docs/40-domain/business-rules.md. Clarify that only first viewport initialization requires foreground presentation; hiding the Work tab retains its view/debugger and tool access, while explicit close/project/module/window teardown still releases it. Evidence: actual native E2E plus 134 focused tests. Future impact: prevent reinstating a visibility gate or unconditional show request. Semantic conflict: older text says hidden browsers cannot be debugged; the user's current request explicitly supersedes that behavior, so no additional product approval is needed for this distinction during integration.
|
||||
@@ -119,7 +119,8 @@ Pi 正式包必须继续运行 `pnpm run verify:artifact:pi`、`pnpm run smoke:p
|
||||
### 共享开发浏览器
|
||||
|
||||
- Electron Main 持有 sandboxed `WebContentsView`、项目级持久浏览器配置和按需 CDP 连接；被调试页面不获得 Makelore Preload、Node.js 能力或 Host API 凭证。Renderer 只提交当前项目 id 和可见区域，真实项目路径仍由 Main 解析和校验。
|
||||
- 用户和 Agent 操作同一个页面。Agent 发起 `open` 后，Main 通知 Coding 右侧面板展开，并最多等待 5 秒取得当前 generation 的可见 bounds；展示失败会销毁当前视图并返回原始 `AgentBrowserFault` 代码，不会把隐藏页面伪装成可调试状态。Agent 通过 Main 代理的页面级 CDP 工具导航、读取 Console/Network 和执行调试命令。
|
||||
- 用户和 Agent 操作同一个页面。Agent 首次打开新浏览器时，Main 通知作品页展开，并最多等待 5 秒取得当前 generation 的可见 bounds；首次展示失败会销毁当前视图并返回原始 `AgentBrowserFault` 代码。取得视口后，切回操作对话只隐藏页面，Agent 仍可通过 Main 代理的页面级 CDP 工具导航、读取 Console/Network 和执行调试命令；重复 `open` 或导航只更新浏览器状态，不强制切换用户当前标签。
|
||||
- 隐藏页面截图先通过 Electron 原生捕获请求一帧，再执行原始 CDP 截图，以保留格式、裁剪等参数；整个过程不展示页面、不转移输入焦点，也不持续关闭后台节流。
|
||||
- 非 Web 协议、文件注入、跨目标及宿主级命令会被阻止。关闭面板、离开模块或项目、隐藏窗口、Renderer 丢失及后台休眠都会销毁 `WebContentsView`、detach debugger 并释放页面；关闭态没有浏览器轮询或 DOM observer。诊断域按 owner 计数，只在 Agent 当前运行或用户展开 Console/Network 时启用，运行结算/抽屉收起后释放；事件读取使用 5 秒长轮询、最小 drain backoff 和 500 条界面上限。该能力独立于发布和部署。
|
||||
- 一键提交时，Makelore 会从待上传构建归档的同一组 Main-owned 内存字节启动临时回环站点，并在两个独立的临时 Chromium profile 中检查桌面和移动视口的主页面加载、运行错误、失败资源与白屏。临时页面不挂载到界面，不读取或写入用户浏览器的 Cookie、历史和登录态；检查结束后始终销毁并清理，也不要求用户预先打开开发预览。
|
||||
- 客户端复用 Electron 内置 Chromium，不安装 Playwright 或额外浏览器。预检只改善提交前反馈，可被非官方客户端绕过，也不会上传“已通过”凭据；平台仍把源码、构建归档和清单视为不可信输入，逐字节重算并在人工审核后发布。安装包携带固定 npm 运行时，项目依赖和 Vite 版本由 `package-lock.json` 锁定；依赖准备需要本地网络。
|
||||
|
||||
@@ -20,7 +20,7 @@ function isAllowedTopLevelUrl(value: string): boolean {
|
||||
}
|
||||
}
|
||||
|
||||
function wrapDebugger(contents: WebContents): AgentBrowserDebuggerPort {
|
||||
function wrapDebugger(contents: WebContents, isVisible: () => boolean): AgentBrowserDebuggerPort {
|
||||
const debuggerEvents = contents.debugger as unknown as {
|
||||
on(event: string, listener: PortListener): void;
|
||||
removeListener(event: string, listener: PortListener): void;
|
||||
@@ -29,8 +29,14 @@ function wrapDebugger(contents: WebContents): AgentBrowserDebuggerPort {
|
||||
attach: (version) => contents.debugger.attach(version),
|
||||
detach: () => contents.debugger.detach(),
|
||||
isAttached: () => contents.debugger.isAttached(),
|
||||
sendCommand: (method, params, sessionRef) =>
|
||||
contents.debugger.sendCommand(method, params, sessionRef),
|
||||
sendCommand: async (method, params, sessionRef) => {
|
||||
if (method === 'Page.captureScreenshot' && !isVisible()) {
|
||||
// CDP capture can wait forever for a hidden view's compositor. Electron's
|
||||
// native capture requests a frame without showing or focusing the view.
|
||||
await contents.capturePage(undefined, { stayHidden: true, stayAwake: false });
|
||||
}
|
||||
return contents.debugger.sendCommand(method, params, sessionRef);
|
||||
},
|
||||
on: (event, listener) => debuggerEvents.on(event, listener),
|
||||
removeListener: (event, listener) =>
|
||||
debuggerEvents.removeListener(event, listener),
|
||||
@@ -53,13 +59,13 @@ function wrapNavigation(contents: WebContents): AgentBrowserNavigationPort {
|
||||
};
|
||||
}
|
||||
|
||||
function wrapWebContents(contents: WebContents): AgentBrowserWebContentsPort {
|
||||
function wrapWebContents(contents: WebContents, isVisible: () => boolean): AgentBrowserWebContentsPort {
|
||||
const contentsEvents = contents as unknown as {
|
||||
on(event: string, listener: PortListener): void;
|
||||
removeListener(event: string, listener: PortListener): void;
|
||||
};
|
||||
return {
|
||||
debugger: wrapDebugger(contents),
|
||||
debugger: wrapDebugger(contents, isVisible),
|
||||
navigationHistory: wrapNavigation(contents),
|
||||
loadURL: (url) => contents.loadURL(url),
|
||||
getURL: () => contents.getURL(),
|
||||
@@ -111,7 +117,7 @@ export class ElectronAgentBrowserAdapter implements AgentBrowserAdapter {
|
||||
this.guardSession(contents.session);
|
||||
|
||||
const view: AgentBrowserViewPort = {
|
||||
webContents: wrapWebContents(contents),
|
||||
webContents: wrapWebContents(contents, () => nativeView.getVisible()),
|
||||
setBounds: (bounds: AgentBrowserBounds) =>
|
||||
nativeView.setBounds(this.toNativeBounds(bounds)),
|
||||
setVisible: (visible: boolean) => nativeView.setVisible(visible),
|
||||
|
||||
@@ -615,7 +615,7 @@ export class AgentBrowserModule {
|
||||
}
|
||||
|
||||
async readEvents(input: AgentBrowserReadEventsInput): Promise<AgentBrowserCdpEventPage> {
|
||||
let record = this.requirePresented(input.projectPath);
|
||||
let record = this.requireViewport(input.projectPath);
|
||||
const after = normalizeNonNegativeInteger(input.after, 0, 'after');
|
||||
const limit = normalizeIntegerRange(input.limit, 100, 1, 200, 'limit');
|
||||
const waitMs = normalizeIntegerRange(input.waitMs, 0, 0, MAX_WAIT_MS, 'waitMs');
|
||||
@@ -645,13 +645,13 @@ export class AgentBrowserModule {
|
||||
|| current !== record
|
||||
|| current.eventBuffer.cursor !== cursorBeforeWait;
|
||||
});
|
||||
record = this.requirePresented(input.projectPath);
|
||||
record = this.requireViewport(input.projectPath);
|
||||
page = record.eventBuffer.read(page.nextCursor, input.methods, limit);
|
||||
return page;
|
||||
}
|
||||
|
||||
async readPayload(input: AgentBrowserReadPayloadInput): Promise<AgentBrowserPayloadChunk> {
|
||||
this.requirePresented(input.projectPath);
|
||||
this.requireViewport(input.projectPath);
|
||||
return this.payloadStore.read(input.handle, input.offset, input.maxBytes);
|
||||
}
|
||||
|
||||
@@ -1589,7 +1589,7 @@ export class AgentBrowserModule {
|
||||
}
|
||||
|
||||
private requireAttached(projectPath: string): BrowserRecord {
|
||||
const record = this.requirePresented(projectPath);
|
||||
const record = this.requireViewport(projectPath);
|
||||
if (record.state === 'suspended_devtools') {
|
||||
throw new AgentBrowserFault(
|
||||
'DEVTOOLS_CONFLICT',
|
||||
@@ -1617,12 +1617,14 @@ export class AgentBrowserModule {
|
||||
return record;
|
||||
}
|
||||
|
||||
private requirePresented(projectPath: string): BrowserRecord {
|
||||
private requireViewport(projectPath: string): BrowserRecord {
|
||||
const record = this.requireRecord(projectPath);
|
||||
if (!record.visible || !record.bounds) {
|
||||
// Switching to chat hides the native view without ending its debugging session.
|
||||
// A new view still needs its first real viewport before tools can operate.
|
||||
if (!record.bounds) {
|
||||
throw new AgentBrowserFault(
|
||||
'VIEWPORT_NOT_READY',
|
||||
'开发浏览器已收起，智能体调试已暂停。',
|
||||
'开发浏览器显示区域尚未准备好。',
|
||||
true,
|
||||
record.generation,
|
||||
);
|
||||
|
||||
@@ -119,7 +119,9 @@ export class PiAgentBrowserTool {
|
||||
});
|
||||
const runKey = `${context.conversationId}:${context.runId}`;
|
||||
this.diagnosticRuns.set(runKey, { projectPath: context.projectPath, owner });
|
||||
if (this.requestPresentation) {
|
||||
// Only a new viewport needs foreground presentation. Reusing an initialized
|
||||
// page must respect the user's selected tab, including while navigating.
|
||||
if (this.requestPresentation && !snapshot.bounds) {
|
||||
try {
|
||||
this.requestPresentation(snapshot);
|
||||
snapshot = await this.browser.waitForPresentation({
|
||||
@@ -139,6 +141,8 @@ export class PiAgentBrowserTool {
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
} else {
|
||||
this.publishState?.(snapshot);
|
||||
}
|
||||
return result(action, publicSnapshot(snapshot));
|
||||
}
|
||||
|
||||
72
tests/e2e/fixtures/native-agent-browser.ts
Normal file
72
tests/e2e/fixtures/native-agent-browser.ts
Normal file
@@ -0,0 +1,72 @@
|
||||
import type { BrowserWindow } from 'electron';
|
||||
import path from 'node:path';
|
||||
import type { AgentBrowserBounds } from '../../../shared/agent-browser';
|
||||
import { AgentBrowserModule } from '../../../electron/agent-browser/module';
|
||||
import { ElectronAgentBrowserAdapter } from '../../../electron/agent-browser/electron-adapter';
|
||||
import { CodingAttachmentStore } from '../../../electron/coding-projects/attachment-store';
|
||||
import { PiAgentBrowserTool } from '../../../electron/coding-runtime/pi/extensions/agent-browser';
|
||||
|
||||
// Loaded in Electron Main by the E2E spec: use the production view, debugger and
|
||||
// Pi tool while the surrounding project/model APIs remain isolated fixtures.
|
||||
export class NativeAgentBrowserFixture {
|
||||
readonly browser: AgentBrowserModule;
|
||||
readonly tool: PiAgentBrowserTool;
|
||||
showRequests = 0;
|
||||
readonly context;
|
||||
|
||||
constructor(window: BrowserWindow, projectPath: string, readonly url: string) {
|
||||
this.context = {
|
||||
projectId: 'project-pi-first-chat', projectPath,
|
||||
conversationId: 'conversation-pi-first-chat', runId: 'browser-e2e',
|
||||
};
|
||||
this.browser = new AgentBrowserModule(new ElectronAgentBrowserAdapter(window));
|
||||
this.tool = new PiAgentBrowserTool(
|
||||
this.browser,
|
||||
new CodingAttachmentStore(path.join(projectPath, 'attachments')),
|
||||
(snapshot) => {
|
||||
this.showRequests += 1;
|
||||
window.webContents.send('agent-browser:show', snapshot);
|
||||
},
|
||||
(snapshot) => window.webContents.send('agent-browser:state', snapshot),
|
||||
);
|
||||
}
|
||||
|
||||
async execute(input: Record<string, unknown>) {
|
||||
const result = await this.tool.execute(this.context, input);
|
||||
return { value: JSON.parse(result.content[0].text), showRequests: this.showRequests };
|
||||
}
|
||||
|
||||
async route(route: string, body?: Record<string, unknown>) {
|
||||
const { projectId, projectPath } = this.context;
|
||||
if (route.startsWith('/api/agent-browser/state?')) {
|
||||
return { success: true, browser: await this.browser.getSnapshot(projectPath) };
|
||||
}
|
||||
if (route === '/api/agent-browser/ensure-work') {
|
||||
const snapshot = await this.browser.getSnapshot(projectPath);
|
||||
return {
|
||||
success: true, status: 'ready',
|
||||
browser: snapshot.state === 'closed'
|
||||
? await this.browser.open({ projectId, projectPath, url: this.url, visible: false })
|
||||
: snapshot,
|
||||
};
|
||||
}
|
||||
if (route === '/api/agent-browser/present') {
|
||||
return {
|
||||
success: true,
|
||||
browser: await this.browser.present({
|
||||
projectPath, visible: body?.visible === true,
|
||||
bounds: body?.bounds as AgentBrowserBounds | undefined,
|
||||
}),
|
||||
};
|
||||
}
|
||||
if (route === '/api/agent-browser/close') {
|
||||
return { success: true, browser: await this.browser.close(projectPath) };
|
||||
}
|
||||
throw new Error('Unexpected native browser fixture route: ' + route);
|
||||
}
|
||||
|
||||
async dispose() {
|
||||
await this.tool.releaseRun(this.context.conversationId, this.context.runId);
|
||||
await this.browser.close(this.context.projectPath);
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,9 @@
|
||||
import type { ElectronApplication, Page } from 'playwright-core';
|
||||
import { appendFile } from 'node:fs/promises';
|
||||
import { createServer } from 'node:http';
|
||||
import { createRequire } from 'node:module';
|
||||
import { resolve } from 'node:path';
|
||||
import type { NativeAgentBrowserFixture } from './fixtures/native-agent-browser';
|
||||
import { expect, getStableWindow, test } from './fixtures/electron';
|
||||
|
||||
type CapturedRequest = {
|
||||
@@ -102,6 +106,7 @@ async function installCodingFirstChatHost(
|
||||
};
|
||||
const mainGlobal = globalThis as typeof globalThis & {
|
||||
__makelorePiFirstChatE2E?: MainState;
|
||||
__makeloreNativeBrowserE2E?: NativeAgentBrowserFixture;
|
||||
};
|
||||
const state: MainState = {
|
||||
captured: [],
|
||||
@@ -593,6 +598,9 @@ async function installCodingFirstChatHost(
|
||||
: []).map((item) => ({ ...item, ...metadata.get(item.id) })),
|
||||
});
|
||||
}
|
||||
if (path.startsWith('/api/agent-browser/') && mainGlobal.__makeloreNativeBrowserE2E) {
|
||||
return respond(await mainGlobal.__makeloreNativeBrowserE2E.route(path, body));
|
||||
}
|
||||
if (path.startsWith('/api/agent-browser/state?')) {
|
||||
return respond({ success: true, browser: browserSnapshot() });
|
||||
}
|
||||
@@ -2045,6 +2053,93 @@ test('work tab prepares the page automatically and keeps the student draft throu
|
||||
expect(requests.filter((request) => request.path.endsWith('/prompt'))).toHaveLength(0);
|
||||
});
|
||||
|
||||
test('agent browser continues in the chat tab without repeatedly selecting Work', async ({ launchElectronApp }) => {
|
||||
const require = createRequire(resolve('package.json'));
|
||||
const { build } = createRequire(require.resolve('vite/package.json'))('esbuild') as typeof import('esbuild');
|
||||
const bundlePath = test.info().outputPath('native-agent-browser.cjs');
|
||||
await build({
|
||||
entryPoints: [resolve('tests/e2e/fixtures/native-agent-browser.ts')],
|
||||
outfile: bundlePath, bundle: true, platform: 'node', format: 'cjs', external: ['electron'],
|
||||
});
|
||||
let pageLoads = 0;
|
||||
const server = createServer((_request, response) => {
|
||||
pageLoads += 1;
|
||||
response.setHeader('Content-Type', 'text/html; charset=utf-8');
|
||||
response.end('<html><body><h1>Browser background test</h1><script>console.log("page-ready")</script></body></html>');
|
||||
});
|
||||
await new Promise<void>((done) => server.listen(0, '127.0.0.1', done));
|
||||
const address = server.address();
|
||||
if (!address || typeof address === 'string') throw new Error('Missing loopback port');
|
||||
const url = 'http://127.0.0.1:' + address.port + '/';
|
||||
const electronApp = await launchElectronApp({ skipSetup: true });
|
||||
try {
|
||||
let page = await getStableWindow(electronApp);
|
||||
const connection = await page.evaluate(async () => ({ token: await window.electron.ipcRenderer.invoke('hostapi:token') as string, baseUrl: await window.electron.ipcRenderer.invoke('hostapi:base-url') as string }));
|
||||
await installCodingFirstChatHost(electronApp, connection, true);
|
||||
await settleSnapshot(electronApp);
|
||||
await disableCodingEventSource(page);
|
||||
await page.reload(); page = await getStableWindow(electronApp);
|
||||
await page.getByTestId('ai-module-option-programming').click();
|
||||
await page.evaluate(() => { window.location.hash = '/chat'; });
|
||||
const composer = page.getByTestId('coding-message-composer').getByRole('textbox');
|
||||
await composer.fill('继续对话，不要抢走标签');
|
||||
await electronApp.evaluate(({ app, BrowserWindow }, input) => {
|
||||
const { NativeAgentBrowserFixture } = process.mainModule!.require(input.bundlePath) as typeof import('./fixtures/native-agent-browser');
|
||||
(globalThis as typeof globalThis & { __makeloreNativeBrowserE2E?: NativeAgentBrowserFixture }).__makeloreNativeBrowserE2E =
|
||||
new NativeAgentBrowserFixture(BrowserWindow.getAllWindows()[0], app.getPath('userData'), input.url);
|
||||
}, { bundlePath, url });
|
||||
const execute = async (input: Record<string, unknown>) => await electronApp.evaluate(async (_electron, command) => {
|
||||
return await (globalThis as typeof globalThis & { __makeloreNativeBrowserE2E: NativeAgentBrowserFixture }).__makeloreNativeBrowserE2E.execute(command);
|
||||
}, input);
|
||||
const opened = await execute({ action: 'open', url });
|
||||
expect(opened.value.visible).toBe(true);
|
||||
expect(opened.showRequests).toBe(1);
|
||||
await expect(page.getByRole('tab', { name: '作品', exact: true })).toHaveAttribute('aria-selected', 'true');
|
||||
await page.getByRole('tab', { name: '操作对话', exact: true }).click();
|
||||
await expect.poll(async () => (await execute({ action: 'status' })).value.visible).toBe(false);
|
||||
await composer.focus();
|
||||
const generation = opened.value.generation;
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
// Include a new URL: open may navigate, but must not reveal the native view.
|
||||
const reused = await execute({ action: 'open', url: url + '?attempt=' + attempt });
|
||||
expect(reused.value).toMatchObject({ browserId: opened.value.browserId, generation, visible: false });
|
||||
expect(reused.showRequests).toBe(1);
|
||||
const evaluated = await execute({
|
||||
action: 'send_cdp', method: 'Runtime.evaluate',
|
||||
params: { expression: 'document.querySelector("h1").textContent', returnByValue: true },
|
||||
});
|
||||
expect(evaluated.value).toMatchObject({ kind: 'inline', value: { result: { value: 'Browser background test' } } });
|
||||
await expect(page.getByRole('tab', { name: '操作对话', exact: true })).toHaveAttribute('aria-selected', 'true');
|
||||
await expect(composer).toHaveValue('继续对话，不要抢走标签');
|
||||
}
|
||||
const loadsBeforeReload = pageLoads;
|
||||
await execute({ action: 'navigate', navigation: 'reload' });
|
||||
await expect.poll(() => pageLoads).toBeGreaterThan(loadsBeforeReload);
|
||||
const events = await execute({ action: 'read_events', after: 0 });
|
||||
expect(events.value.events.some((event: { method: string }) => event.method === 'Runtime.consoleAPICalled')).toBe(true);
|
||||
const screenshot = await execute({ action: 'send_cdp', method: 'Page.captureScreenshot' });
|
||||
expect(screenshot.value).toMatchObject({ attachmentId: expect.any(String), mime: 'image/png' });
|
||||
expect(screenshot.showRequests).toBe(1);
|
||||
expect((await execute({ action: 'status' })).value.visible).toBe(false);
|
||||
await expect(composer).toHaveValue('继续对话，不要抢走标签');
|
||||
await expect(composer).toBeFocused();
|
||||
await page.getByRole('tab', { name: '作品', exact: true }).click();
|
||||
await expect.poll(async () => (await execute({ action: 'status' })).value.visible).toBe(true);
|
||||
expect((await execute({ action: 'status' })).value).toMatchObject({ browserId: opened.value.browserId, generation });
|
||||
await page.getByRole('tab', { name: '操作对话', exact: true }).click();
|
||||
await expect.poll(async () => (await execute({ action: 'status' })).value.visible).toBe(false);
|
||||
await execute({ action: 'close' });
|
||||
expect((await execute({ action: 'status' })).value.state).toBe('closed');
|
||||
} finally {
|
||||
await electronApp.evaluate(async () => {
|
||||
const mainGlobal = globalThis as typeof globalThis & { __makeloreNativeBrowserE2E?: NativeAgentBrowserFixture };
|
||||
await mainGlobal.__makeloreNativeBrowserE2E?.dispose();
|
||||
delete mainGlobal.__makeloreNativeBrowserE2E;
|
||||
});
|
||||
await new Promise<void>((done, reject) => server.close((error) => error ? reject(error) : done()));
|
||||
}
|
||||
});
|
||||
|
||||
test('work failure offers a useful recovery and retains the student draft after opening', async ({ launchElectronApp }) => {
|
||||
const electronApp = await launchElectronApp({ skipSetup: true });
|
||||
let page = await getStableWindow(electronApp);
|
||||
|
||||
@@ -1280,7 +1280,9 @@ describe('AgentBrowserModule', () => {
|
||||
await expect(module.sendCdp({
|
||||
projectPath,
|
||||
method: 'Runtime.evaluate',
|
||||
})).rejects.toMatchObject({ code: 'VIEWPORT_NOT_READY' });
|
||||
})).resolves.toMatchObject({ kind: 'inline' });
|
||||
await expect(module.readEvents({ projectPath, after: 0 })).resolves.toBeDefined();
|
||||
await module.navigate({ projectPath, action: 'reload' });
|
||||
const reopened = await module.open({
|
||||
projectId: 'clock',
|
||||
projectPath,
|
||||
@@ -1288,10 +1290,18 @@ describe('AgentBrowserModule', () => {
|
||||
visible: true,
|
||||
});
|
||||
expect(reopened).toMatchObject({
|
||||
browserId: first.browserId,
|
||||
generation: first.generation,
|
||||
visible: false,
|
||||
bounds: { x: 1, y: 2, width: 640, height: 480 },
|
||||
});
|
||||
expect(adapter.views[0].visible).toBe(false);
|
||||
expect(adapter.views).toHaveLength(1);
|
||||
await module.close(projectPath);
|
||||
await expect(module.sendCdp({
|
||||
projectPath,
|
||||
method: 'Runtime.evaluate',
|
||||
})).rejects.toMatchObject({ code: 'BROWSER_NOT_OPEN' });
|
||||
});
|
||||
|
||||
it('waits for the matching browser generation to be presented without polling', async () => {
|
||||
|
||||
@@ -32,6 +32,7 @@ const electronMocks = vi.hoisted(() => {
|
||||
webContents: unknown;
|
||||
setBounds: ReturnType<typeof vi.fn>;
|
||||
setVisible: ReturnType<typeof vi.fn>;
|
||||
getVisible: ReturnType<typeof vi.fn>;
|
||||
}> = [];
|
||||
const permissionHandler = vi.fn();
|
||||
const permissionCheckHandler = vi.fn();
|
||||
@@ -66,11 +67,13 @@ const electronMocks = vi.hoisted(() => {
|
||||
isDevToolsOpened: vi.fn().mockReturnValue(false),
|
||||
reload: vi.fn(),
|
||||
executeJavaScript: vi.fn().mockResolvedValue({ ok: true }),
|
||||
capturePage: vi.fn().mockResolvedValue({}),
|
||||
close: vi.fn(),
|
||||
setWindowOpenHandler: vi.fn(),
|
||||
});
|
||||
readonly setBounds = vi.fn();
|
||||
readonly setVisible = vi.fn();
|
||||
readonly getVisible = vi.fn().mockReturnValue(true);
|
||||
|
||||
constructor(readonly options: unknown) {
|
||||
nativeViews.push(this);
|
||||
@@ -102,6 +105,29 @@ describe('ElectronAgentBrowserAdapter', () => {
|
||||
electronMocks.browserSession.removeAllListeners();
|
||||
});
|
||||
|
||||
it.each([false, true])('captures a page without presenting a hidden native view (visible=%s)', async (visible) => {
|
||||
const adapter = new ElectronAgentBrowserAdapter({} as never);
|
||||
const view = adapter.createView('browser-screenshot');
|
||||
const native = electronMocks.nativeViews[0];
|
||||
native.getVisible.mockReturnValue(visible);
|
||||
const contents = native.webContents as InstanceType<typeof electronMocks.MockWebContentsView>['webContents'];
|
||||
const params = { format: 'jpeg', quality: 80, clip: { x: 0, y: 0, width: 320, height: 200, scale: 1 } };
|
||||
contents.debugger.sendCommand.mockResolvedValue({ data: 'screenshot' });
|
||||
await expect(view.webContents.debugger.sendCommand('Page.captureScreenshot', params, 'child-session'))
|
||||
.resolves.toEqual({ data: 'screenshot' });
|
||||
if (visible) {
|
||||
expect(contents.capturePage).not.toHaveBeenCalled();
|
||||
} else {
|
||||
expect(contents.capturePage).toHaveBeenCalledExactlyOnceWith(undefined, { stayHidden: true, stayAwake: false });
|
||||
expect(contents.capturePage.mock.invocationCallOrder[0]).toBeLessThan(contents.debugger.sendCommand.mock.invocationCallOrder[0]);
|
||||
}
|
||||
expect(contents.debugger.sendCommand).toHaveBeenCalledWith('Page.captureScreenshot', params, 'child-session');
|
||||
expect(native.setVisible).not.toHaveBeenCalled();
|
||||
contents.capturePage.mockClear();
|
||||
await view.webContents.debugger.sendCommand('Runtime.evaluate', { expression: '1' });
|
||||
expect(contents.capturePage).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('creates an isolated sandboxed WebContentsView and mounts it in Main', async () => {
|
||||
const addChildView = vi.fn();
|
||||
const removeChildView = vi.fn();
|
||||
|
||||
@@ -458,6 +458,45 @@ describe('PI-090 product tools', () => {
|
||||
});
|
||||
});
|
||||
|
||||
it.each([false, true])('preserves the selected tab when reopening an initialized browser (visible=%s)', async (visible) => {
|
||||
const root = await temporaryRoot('makelore-pi-browser-reuse-');
|
||||
const initialized = {
|
||||
browserId: 'browser-a', projectId: 'project-a', projectPath: root,
|
||||
state: 'attached' as const, generation: 3, url: 'http://127.0.0.1:4173/', title: 'App',
|
||||
visible, bounds: { x: 10, y: 20, width: 800, height: 600 },
|
||||
canGoBack: false, canGoForward: false, eventCursor: 0,
|
||||
};
|
||||
const browser = {
|
||||
open: vi.fn(async () => initialized),
|
||||
waitForPresentation: vi.fn(async () => ({ ...initialized, visible: true })),
|
||||
setDiagnostics: vi.fn(async () => initialized),
|
||||
} as unknown as AgentBrowserModule;
|
||||
const requestAgentBrowserPresentation = vi.fn();
|
||||
const publishAgentBrowserState = vi.fn();
|
||||
const tools = new PiProductTools({
|
||||
browser,
|
||||
attachments: new CodingAttachmentStore(path.join(root, 'attachments')),
|
||||
bundledSkillsDir: path.resolve('resources/coding-skills'),
|
||||
requestAgentBrowserPresentation,
|
||||
publishAgentBrowserState,
|
||||
});
|
||||
const context = {
|
||||
conversationId: 'conversation-a', runId: 'run-a', resourceId: 'browser-a',
|
||||
projectId: 'project-a', projectPath: root, skillIds: [],
|
||||
};
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
const result = await tools.execute('agent_browser', context, { action: 'open', url: initialized.url });
|
||||
expect(JSON.parse(result.content[0].text)).toMatchObject({ visible, generation: 3 });
|
||||
}
|
||||
expect(requestAgentBrowserPresentation).not.toHaveBeenCalled();
|
||||
expect(browser.waitForPresentation).not.toHaveBeenCalled();
|
||||
expect(publishAgentBrowserState).toHaveBeenLastCalledWith(initialized);
|
||||
await tools.settleRun('conversation-a', 'run-a');
|
||||
expect(browser.setDiagnostics).toHaveBeenCalledWith({
|
||||
projectPath: root, enabled: false, owner: 'agent:conversation-a:run-a',
|
||||
});
|
||||
});
|
||||
|
||||
it('closes an agent-opened browser when presentation times out', async () => {
|
||||
const root = await temporaryRoot('makelore-pi-browser-timeout-');
|
||||
const hidden = {
|
||||
|
||||
Reference in New Issue
Block a user