makelore/.project-docs/50-evidence/topics/20260822-pi-runtime-qualification-c52e8a14__pi-runtime-qualification.md

PI-000 Pi Runtime Qualification Evidence

Scope And Decision State

• Task: 20260822-pi-runtime-qualification-c52e8a14
• Candidate: @earendil-works/pi-coding-agent@0.84.2
• Current decision: PI-000 Done with explicit waivers. This is not an all-gates Pass.
• Ready frontier consequence: PI-010, PI-020, and PI-030 are unlocked by the planner's audited 18-node/31-edge DAG.
• macOS Phase-0 decision: Explicitly Waived / Deferred by user on 2026-08-22. This is not a Pass. macOS x64/arm64 qualification moves to PI-150 final cross-platform release validation.
• Real Provider decision: QG-004 and QG-005 are Explicitly Waived / Accepted Risk by user on 2026-08-22, not Pass. They no longer block PI-000 or PI-150.
• Accepted unverified risk includes real authentication, endpoint/proxy/rate-limit/provider-specific response variation; base URL, header, environment credential, model and image compatibility; real provider concurrency/serialization; and cross-worker abort/event/session/model/credential isolation.
• This record summarizes reproducible evidence. Raw generated JSON reports, temporary staged runtimes, and controlled directory artifacts are intentionally not committed.

Confirmed Upstream Identity

• npm package version: 0.84.2
• npm CLI entry: dist/cli.js
• declared Node engine: >=22.19.0
• upstream tag: v0.84.2
• upstream commit and npm gitHead: 914cf1472e715297caa30db4b9535d534a9eb718
• package pin in Makelore: exact 0.84.2, without caret or tilde
• project package manager used for the lockfile: the repository-pinned pnpm@10.33.4

The selected Electron 43.4.0 executable exposes Node 24.18.1 when launched with ELECTRON_RUN_AS_NODE=1, which satisfies Pi's declared engine floor.

Windows x64 Qualification Sample

Environment:

• OS: Windows x64, release 10.0.26200
• Runtime seam: Electron executable plus ELECTRON_RUN_AS_NODE=1
• Artifact: a temporary production-only staging directory installed from Pi's published npm-shrinkwrap.json
• Startup samples: 5 cold and 5 warm
• Cold definition: fresh Pi config, session, and project directories for each process
• Warm definition: shared primed Pi config directory with fresh session and project directories for each process

Measured result:

Measurement	Samples	p50	p95	max	Budget
Cold get_state ready	5	654 ms	681 ms	681 ms	3000 ms
Warm get_state ready	5	661 ms	674 ms	674 ms	1500 ms
Worker RSS	10	122780 KiB	125780 KiB	125780 KiB	report-only
RPC process exit	10	14 ms	16 ms	16 ms	deterministic

All measured processes exited by closing stdin. No SIGTERM, force kill, or retained child was needed in the successful sample.

The deterministic provider-failure sample used a closed loopback endpoint, disabled automatic retry, and reached prompt accepted followed by agent_settled with an assistant error stop reason in 42 ms. This is below the 10-second deterministic-failure limit and does not claim provider success.

Controlled Electron Builder Artifact Sample

PI-000 also built a Windows x64 Electron Builder dir artifact from the same temporary Pi production closure. This is a controlled packaged-boundary probe, not Makelore's final product artifact and not the permanent PI-030/PI-150 bundler wiring. It deliberately does not inherit the product's OpenCode, Python, UV, signing, installer, or other electron-builder.yml hooks.

The packaged boundary was:

• executable: MakelorePiProbe.exe
• archive: resources/app.asar
• native extraction: resources/app.asar.unpacked
• Pi CLI: resources/app.asar/dist/cli.js
• process seam: packaged executable plus ELECTRON_RUN_AS_NODE=1

Artifact resolution confirmed Pi 0.84.2, its CLI, all 130 Windows-applicable production packages, all six runtime WASM/native assets, and native .node extraction. The nine non-Windows lock entries were skipped by their declared platform constraints. Electron Builder flattened @types/retry@0.12.0 from p-retry/node_modules/@types/retry to the artifact root node_modules/@types/retry; the verifier checks the same version at the resolved artifact location instead of requiring npm's original physical layout.

Measured result from 5 cold and 5 warm packaged samples:

Measurement	Samples	p50	p95	max	Budget
Cold get_state ready	5	668 ms	682 ms	682 ms	3000 ms
Warm get_state ready	5	660 ms	663 ms	663 ms	1500 ms
Worker RSS	10	138404 KiB	141320 KiB	141320 KiB	report-only
RPC process exit	10	16 ms	18 ms	18 ms	deterministic

The packaged session, deterministic failure settlement, reopen, local shell overlap, one-worker shell abort isolation, and stdin-close exit checks also passed. The deterministic failure settled in 36 ms; the overlapping local shell sample completed in 2098 ms. A post-run process query found zero controlled artifact processes and zero Pi RPC-looking processes.

Linux x64 Qualification Sample

The Linux run used an isolated repository snapshot under WSL2 Ubuntu 24.04 x64. The snapshot installed the frozen lockfile with Node 22.22.2 and the exact repository-pinned pnpm@10.33.4; Pi then ran through Electron 43.4.0 and its embedded Node 24.18.1. This exercises the Linux executable, Linux native dependency selection, asar layout, and process behavior. It is not evidence for a native compositor, AppImage/deb installer, or a non-WSL Linux kernel.

Workspace 5+5 result:

Measurement	Samples	p50	p95	max	Budget
Cold get_state ready	5	426 ms	853 ms	853 ms	3000 ms
Warm get_state ready	5	432 ms	437 ms	437 ms	1500 ms
Worker RSS	10	168228 KiB	170388 KiB	170388 KiB	report-only
RPC process exit	10	7 ms	8 ms	8 ms	deterministic

Temporary production-closure 5+5 result:

Measurement	Samples	p50	p95	max	Budget
Cold get_state ready	5	426 ms	428 ms	428 ms	3000 ms
Warm get_state ready	5	423 ms	440 ms	440 ms	1500 ms
Worker RSS	10	170380 KiB	175956 KiB	175956 KiB	report-only
RPC process exit	10	7 ms	8 ms	8 ms	deterministic

Controlled Electron Builder dir/app.asar 5+5 result:

Measurement	Samples	p50	p95	max	Budget
Cold get_state ready	5	526 ms	545 ms	545 ms	3000 ms
Warm get_state ready	5	545 ms	569 ms	569 ms	1500 ms
Worker RSS	10	181412 KiB	181928 KiB	181928 KiB	report-only
RPC process exit	10	7 ms	7 ms	7 ms	deterministic

The Linux shrinkwrap has 131 applicable packages and 8 OS/CPU-skipped entries. The controlled artifact resolved all applicable packages, the Pi CLI, all six tracked runtime assets, and every native asset that must live under app.asar.unpacked; no package or asset was missing. Electron Builder applied the same-version @types/retry@0.12.0 flattening seen on Windows. Session persistence/reopen, deterministic failure settlement, local overlap, shell abort isolation, and stdin-close cleanup passed in all three boundaries. A final process listing contained only WSL init and the listing command itself—no packaged Pi or RPC worker remained.

Runtime And Session Behaviors Confirmed On Windows

• Strict LF-only JSONL framing, including payloads containing U+2028 and U+2029.
• get_state response and request id correlation.
• Persistent session file creation after a settled assistant turn.
• Reopen by session file with the same stable session id.
• Session history remained readable after reopen; the sample contained six entries.
• Direct RPC shell execution returned the expected output.
• Idle abort was accepted.
• Two workers had distinct session ids.
• Two direct shell operations overlapped; total elapsed time was 2112 ms for a 4-second left command and 2-second right command.
• abort_bash cancelled the left worker without cancelling or corrupting the right worker.
• Resource discovery was explicitly disabled for the qualification sample with --no-extensions, --no-skills, --no-prompt-templates, --no-themes, --no-context-files, and --no-approve; startup ran with --offline and PI_TELEMETRY=0.

The two-worker result proves local process/session/shell isolation only. It is not evidence of two overlapping real-provider turns.

Production Closure Evidence

Pi's published production shrinkwrap contains 139 dependency package entries:

• 130 entries are applicable and present on Windows x64.
• 9 entries are correctly skipped by OS/CPU constraints.
• Required image-processing asset is present: @silvia-odwyer/photon-node/photon_rs_bg.wasm.
• Applicable clipboard native binary is present: @mariozechner/clipboard-win32-x64-msvc.
• Pi TUI Windows x64/arm64 prebuilds and its packaged Darwin prebuilds are present in the staged closure.

Published-package caveat: the shrinkwrap is production-only, but the published package still lists ten development dependencies. npm 11 validates those development entries before applying --omit=dev, so an unmodified npm ci --omit=dev rejects the package as out of sync. The qualification staging copy removes devDependencies from only the temporary copied package root and then installs the unchanged published shrinkwrap. This must be considered when PI-030 defines the permanent staging mechanism.

Provider Contract Matrix

Static Pi contract coverage and a packaged-runtime loopback HTTP/SSE contract sample are confirmed for all current Makelore protocol identifiers:

Makelore protocol	Pi configuration	Loopback endpoint	Windows / Linux overlap	Real turn
openai-completions	api: openai-completions	/v1/chat/completions	304 / 292 ms	not run — waived/unverified
openai-responses	api: openai-responses	/v1/responses	309 / 290 ms	not run — waived/unverified
anthropic-messages	api: anthropic-messages	/v1/messages	290 / 277 ms	not run — waived/unverified
openrouter	api: openai-completions plus thinkingFormat/sessionAffinityFormat: openrouter	/v1/chat/completions	297 / 288 ms	not run — waived/unverified

For each protocol, the packaged runtime sent four POST requests through 127.0.0.1, used the selected model, included the custom header, injected the environment-only credential into the protocol-appropriate header (Authorization or x-api-key), and carried the PNG image in request history. Two independent workers settled successfully with stop, their provider-shaped turns overlapped, aborting the next turn in one worker produced aborted, and the unaffected worker still settled with stop. All workers exited by closing stdin.

The Anthropic contract has a meaningful base URL difference: its SDK appends /v1/messages, so the configured base URL is the provider root. The OpenAI-style clients receive a base URL ending in /v1 and append chat/completions or responses.

This loopback result proves Pi's request serialization, SSE parsing, image/header/model/credential plumbing, worker overlap, and abort isolation without external cost. It does not qualify any real Provider Account, provider authentication, proxy behavior, rate limits, or provider-specific response variation.

Pi's custom provider contract supports base URL, environment-resolved API key, custom headers, model id, and text/image input declaration. The probe accepts a credential-free fixture file shaped as follows; the secret remains only in the named environment variable:

{
  "id": "qualification-provider",
  "apiProtocol": "openai-responses",
  "baseUrl": "https://provider.example/v1",
  "apiKeyEnv": "QUALIFICATION_PROVIDER_KEY",
  "headers": { "X-Provider-Header": "literal-or-$ENV_REFERENCE" },
  "model": {
    "id": "provider-model-id",
    "input": ["text", "image"],
    "contextWindow": 128000,
    "maxTokens": 8192
  }
}

Run a real sample with:

pnpm run probe:pi-runtime -- --provider-fixture <fixture.json> [--image <image-path>]

The real-provider path is implemented to require two successful overlapping turns with distinct sessions, then abort a new turn in one worker while the other worker settles successfully. It reports only provider/model ids, environment-variable name, and header names; it does not print credential values.

No recognized provider credential environment variable was available in the implementation environment, so no real-provider request was sent and no provider cost was incurred.

Reproduce the local contract sample against an existing controlled artifact with:

pnpm run probe:pi-provider-contracts -- --electron-executable <artifact-executable> --cli-path <artifact-app.asar-cli>

Deferred macOS Execution Handoff For PI-150

A pre-execution portability review found and fixed one supported-path defect: the artifact-side closure inspector derived resources from dirname(process.execPath), which is correct on Windows/Linux but points to Contents/MacOS/resources on macOS. The probe now computes the platform resource directory once and passes it into the packaged process, preserving the actual macOS location Contents/Resources/app.asar. A focused path test and a real Windows controlled packaged regression pass verify the shared logic. The user explicitly deferred actual macOS execution from PI-000 to PI-150 on 2026-08-22, so no macOS Pass is claimed and the late-discovery risk remains.

Use a separate checkout and frozen install for each target architecture. Before installing, node -p "process.platform + ' ' + process.arch + ' ' + process.version" must report darwin x64 or darwin arm64 for the sample being collected. Do not reuse one architecture's node_modules or Electron download for the other architecture.

Run this sequence in each architecture-specific checkout:

pnpm install --frozen-lockfile
node scripts/probe-pi-runtime.mjs --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-workspace.json
pnpm run probe:pi-runtime -- --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-staged.json
pnpm run probe:pi-runtime:packaged -- --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-packaged.json

Read artifact.executable and artifact.appAsar from the packaged JSON report. The Pi CLI path is <artifact.appAsar>/dist/cli.js. Run the credential-free packaged four-protocol contract sample before replacing the controlled artifact:

pnpm run probe:pi-provider-contracts -- --electron-executable <artifact.executable> --cli-path <artifact.appAsar>/dist/cli.js --artifact-label controlled-electron-builder-dir-app-asar --report release/pi-provider-contract-macos.json

For an optional, non-gating real Provider diagnostic, create one credential-free fixture per protocol being sampled and keep each secret only in the environment variable named by that fixture's apiKeyEnv. Reuse the existing packaged artifact through the runtime override so the run exercises the packaged Electron seam without rebuilding it for every protocol:

node scripts/probe-pi-runtime.mjs --samples 5 --timeout-ms 10000 --electron-executable <artifact.executable> --cli-path <artifact.appAsar>/dist/cli.js --artifact-label controlled-electron-builder-dir-app-asar --provider-fixture <protocol-fixture.json> --image <qualification-image> --report release/pi-provider-<protocol>-macos.json

The user waived requiring these commands for openai-completions, openai-responses, anthropic-messages, and openrouter. If an optional diagnostic is run, preserve only redacted metrics and protocol/provider/model identifiers; do not commit fixtures containing secrets, generated reports, controlled artifacts, caches, or user data. A final process query must show no packaged probe or Pi RPC worker remains.

Gate Status

Gate	Status	Evidence or blocker
QG-001 exact identity and pin	Pass	exact npm/tag/commit/engine/CLI and lockfile pin
QG-002 packaged Electron Node RPC lifecycle	Pass	Phase-0 scope is Windows x64 + Linux x64 by explicit user decision; both controlled artifacts spawn, become ready, open session, accept prompt, abort, settle, reopen, and exit cleanly. macOS is deferred to PI-150, not passed.
QG-003 production dependency closure	Pass	Phase-0 controlled Windows/Linux artifacts resolve all 130/131 applicable packages and six runtime assets. Final product artifacts and deferred macOS closure belong to PI-150.
QG-004 Provider Account matrix	Explicitly Waived / Accepted Risk	four-protocol loopback request/SSE/image contracts pass; real accounts, credentials, endpoints, proxies, authentication, rate-limit and provider-specific behavior remain unverified by user decision on 2026-08-22
QG-005 real-provider worker concurrency	Explicitly Waived / Accepted Risk	local shell and loopback protocol turns overlap with isolated abort/model/credential state; real provider overlap/serialization and abort/event/session/model/credential isolation remain unverified by user decision on 2026-08-22
QG-006 performance and cleanup metrics	Pass	Phase-0 Windows/Linux cold/warm/RSS/exit sample counts and p50/p95/max are recorded and within budgets. macOS metrics are explicitly deferred to PI-150.
QG-007 stop-and-amend condition	Pending / not triggered	no confirmed packaging, Provider, or budget failure yet; any mandatory-gate failure stops migration and requires the specified utility-process SDK amendment

Remaining And Optional Evidence

1. Real Provider fixtures are optional diagnostics after the explicit waiver. Their absence does not block PI-000 or PI-150, but the current record must continue to report every real turn as unverified; a later confirmed failure is new architecture evidence, not something the waiver converts into a Pass.
2. At PI-150, run the deferred macOS x64/arm64 workspace, staged, final packaged, loopback, closure/native/resource, and metrics matrix. Missing either architecture blocks a cross-platform release-ready claim even though it no longer blocks PI-000.
3. If the accepted Linux definition requires a native desktop distribution or release package, repeat the Linux sample outside WSL2 at PI-150; the current Phase-0 evidence is deliberately labeled as Ubuntu 24.04 userspace on WSL2 with a controlled dir artifact.