feat: qualify Pi runtime cutover foundation

.project-docs/50-evidence/topics/20260822-pi-runtime-qualification-c52e8a14__pi-runtime-qualification.md

@@ -0,0 +1,232 @@
+# PI-000 Pi Runtime Qualification Evidence
+
+## Scope And Decision State
+
+- Task: `20260822-pi-runtime-qualification-c52e8a14`
+- Candidate: `@earendil-works/pi-coding-agent@0.84.2`
+- Current decision: **PI-000 Done with explicit waivers**. This is not an all-gates Pass.
+- Ready frontier consequence: `PI-010`, `PI-020`, and `PI-030` are unlocked by the planner's audited 18-node/31-edge DAG.
+- macOS Phase-0 decision: **Explicitly Waived / Deferred by user on 2026-08-22**. This is not a Pass. macOS x64/arm64 qualification moves to `PI-150` final cross-platform release validation.
+- Real Provider decision: `QG-004` and `QG-005` are **Explicitly Waived / Accepted Risk by user on 2026-08-22**, not Pass. They no longer block PI-000 or PI-150.
+- Accepted unverified risk includes real authentication, endpoint/proxy/rate-limit/provider-specific response variation; base URL, header, environment credential, model and image compatibility; real provider concurrency/serialization; and cross-worker abort/event/session/model/credential isolation.
+- This record summarizes reproducible evidence. Raw generated JSON reports, temporary staged runtimes, and controlled directory artifacts are intentionally not committed.
+
+## Confirmed Upstream Identity
+
+- npm package version: `0.84.2`
+- npm CLI entry: `dist/cli.js`
+- declared Node engine: `>=22.19.0`
+- upstream tag: `v0.84.2`
+- upstream commit and npm `gitHead`: `914cf1472e715297caa30db4b9535d534a9eb718`
+- package pin in Makelore: exact `0.84.2`, without caret or tilde
+- project package manager used for the lockfile: the repository-pinned `pnpm@10.33.4`
+
+The selected Electron `43.4.0` executable exposes Node `24.18.1` when launched with `ELECTRON_RUN_AS_NODE=1`, which satisfies Pi's declared engine floor.
+
+## Windows x64 Qualification Sample
+
+Environment:
+
+- OS: Windows x64, release `10.0.26200`
+- Runtime seam: Electron executable plus `ELECTRON_RUN_AS_NODE=1`
+- Artifact: a temporary production-only staging directory installed from Pi's published `npm-shrinkwrap.json`
+- Startup samples: 5 cold and 5 warm
+- Cold definition: fresh Pi config, session, and project directories for each process
+- Warm definition: shared primed Pi config directory with fresh session and project directories for each process
+
+Measured result:
+
+| Measurement | Samples | p50 | p95 | max | Budget |
+|---|---:|---:|---:|---:|---:|
+| Cold `get_state` ready | 5 | 654 ms | 681 ms | 681 ms | 3000 ms |
+| Warm `get_state` ready | 5 | 661 ms | 674 ms | 674 ms | 1500 ms |
+| Worker RSS | 10 | 122780 KiB | 125780 KiB | 125780 KiB | report-only |
+| RPC process exit | 10 | 14 ms | 16 ms | 16 ms | deterministic |
+
+All measured processes exited by closing stdin. No SIGTERM, force kill, or retained child was needed in the successful sample.
+
+The deterministic provider-failure sample used a closed loopback endpoint, disabled automatic retry, and reached `prompt` accepted followed by `agent_settled` with an assistant `error` stop reason in 42 ms. This is below the 10-second deterministic-failure limit and does not claim provider success.
+
+## Controlled Electron Builder Artifact Sample
+
+PI-000 also built a Windows x64 Electron Builder `dir` artifact from the same temporary Pi production closure. This is a controlled packaged-boundary probe, not Makelore's final product artifact and not the permanent `PI-030`/`PI-150` bundler wiring. It deliberately does not inherit the product's OpenCode, Python, UV, signing, installer, or other `electron-builder.yml` hooks.
+
+The packaged boundary was:
+
+- executable: `MakelorePiProbe.exe`
+- archive: `resources/app.asar`
+- native extraction: `resources/app.asar.unpacked`
+- Pi CLI: `resources/app.asar/dist/cli.js`
+- process seam: packaged executable plus `ELECTRON_RUN_AS_NODE=1`
+
+Artifact resolution confirmed Pi `0.84.2`, its CLI, all 130 Windows-applicable production packages, all six runtime WASM/native assets, and native `.node` extraction. The nine non-Windows lock entries were skipped by their declared platform constraints. Electron Builder flattened `@types/retry@0.12.0` from `p-retry/node_modules/@types/retry` to the artifact root `node_modules/@types/retry`; the verifier checks the same version at the resolved artifact location instead of requiring npm's original physical layout.
+
+Measured result from 5 cold and 5 warm packaged samples:
+
+| Measurement | Samples | p50 | p95 | max | Budget |
+|---|---:|---:|---:|---:|---:|
+| Cold `get_state` ready | 5 | 668 ms | 682 ms | 682 ms | 3000 ms |
+| Warm `get_state` ready | 5 | 660 ms | 663 ms | 663 ms | 1500 ms |
+| Worker RSS | 10 | 138404 KiB | 141320 KiB | 141320 KiB | report-only |
+| RPC process exit | 10 | 16 ms | 18 ms | 18 ms | deterministic |
+
+The packaged session, deterministic failure settlement, reopen, local shell overlap, one-worker shell abort isolation, and stdin-close exit checks also passed. The deterministic failure settled in 36 ms; the overlapping local shell sample completed in 2098 ms. A post-run process query found zero controlled artifact processes and zero Pi RPC-looking processes.
+
+## Linux x64 Qualification Sample
+
+The Linux run used an isolated repository snapshot under WSL2 Ubuntu 24.04 x64. The snapshot installed the frozen lockfile with Node `22.22.2` and the exact repository-pinned `pnpm@10.33.4`; Pi then ran through Electron `43.4.0` and its embedded Node `24.18.1`. This exercises the Linux executable, Linux native dependency selection, asar layout, and process behavior. It is not evidence for a native compositor, AppImage/deb installer, or a non-WSL Linux kernel.
+
+Workspace 5+5 result:
+
+| Measurement | Samples | p50 | p95 | max | Budget |
+|---|---:|---:|---:|---:|---:|
+| Cold `get_state` ready | 5 | 426 ms | 853 ms | 853 ms | 3000 ms |
+| Warm `get_state` ready | 5 | 432 ms | 437 ms | 437 ms | 1500 ms |
+| Worker RSS | 10 | 168228 KiB | 170388 KiB | 170388 KiB | report-only |
+| RPC process exit | 10 | 7 ms | 8 ms | 8 ms | deterministic |
+
+Temporary production-closure 5+5 result:
+
+| Measurement | Samples | p50 | p95 | max | Budget |
+|---|---:|---:|---:|---:|---:|
+| Cold `get_state` ready | 5 | 426 ms | 428 ms | 428 ms | 3000 ms |
+| Warm `get_state` ready | 5 | 423 ms | 440 ms | 440 ms | 1500 ms |
+| Worker RSS | 10 | 170380 KiB | 175956 KiB | 175956 KiB | report-only |
+| RPC process exit | 10 | 7 ms | 8 ms | 8 ms | deterministic |
+
+Controlled Electron Builder `dir`/`app.asar` 5+5 result:
+
+| Measurement | Samples | p50 | p95 | max | Budget |
+|---|---:|---:|---:|---:|---:|
+| Cold `get_state` ready | 5 | 526 ms | 545 ms | 545 ms | 3000 ms |
+| Warm `get_state` ready | 5 | 545 ms | 569 ms | 569 ms | 1500 ms |
+| Worker RSS | 10 | 181412 KiB | 181928 KiB | 181928 KiB | report-only |
+| RPC process exit | 10 | 7 ms | 7 ms | 7 ms | deterministic |
+
+The Linux shrinkwrap has 131 applicable packages and 8 OS/CPU-skipped entries. The controlled artifact resolved all applicable packages, the Pi CLI, all six tracked runtime assets, and every native asset that must live under `app.asar.unpacked`; no package or asset was missing. Electron Builder applied the same-version `@types/retry@0.12.0` flattening seen on Windows. Session persistence/reopen, deterministic failure settlement, local overlap, shell abort isolation, and stdin-close cleanup passed in all three boundaries. A final process listing contained only WSL init and the listing command itself—no packaged Pi or RPC worker remained.
+
+## Runtime And Session Behaviors Confirmed On Windows
+
+- Strict LF-only JSONL framing, including payloads containing U+2028 and U+2029.
+- `get_state` response and request id correlation.
+- Persistent session file creation after a settled assistant turn.
+- Reopen by session file with the same stable session id.
+- Session history remained readable after reopen; the sample contained six entries.
+- Direct RPC shell execution returned the expected output.
+- Idle `abort` was accepted.
+- Two workers had distinct session ids.
+- Two direct shell operations overlapped; total elapsed time was 2112 ms for a 4-second left command and 2-second right command.
+- `abort_bash` cancelled the left worker without cancelling or corrupting the right worker.
+- Resource discovery was explicitly disabled for the qualification sample with `--no-extensions`, `--no-skills`, `--no-prompt-templates`, `--no-themes`, `--no-context-files`, and `--no-approve`; startup ran with `--offline` and `PI_TELEMETRY=0`.
+
+The two-worker result proves local process/session/shell isolation only. It is not evidence of two overlapping real-provider turns.
+
+## Production Closure Evidence
+
+Pi's published production shrinkwrap contains 139 dependency package entries:
+
+- 130 entries are applicable and present on Windows x64.
+- 9 entries are correctly skipped by OS/CPU constraints.
+- Required image-processing asset is present: `@silvia-odwyer/photon-node/photon_rs_bg.wasm`.
+- Applicable clipboard native binary is present: `@mariozechner/clipboard-win32-x64-msvc`.
+- Pi TUI Windows x64/arm64 prebuilds and its packaged Darwin prebuilds are present in the staged closure.
+
+Published-package caveat: the shrinkwrap is production-only, but the published package still lists ten development dependencies. npm 11 validates those development entries before applying `--omit=dev`, so an unmodified `npm ci --omit=dev` rejects the package as out of sync. The qualification staging copy removes `devDependencies` from only the temporary copied package root and then installs the unchanged published shrinkwrap. This must be considered when `PI-030` defines the permanent staging mechanism.
+
+## Provider Contract Matrix
+
+Static Pi contract coverage and a packaged-runtime loopback HTTP/SSE contract sample are confirmed for all current Makelore protocol identifiers:
+
+| Makelore protocol | Pi configuration | Loopback endpoint | Windows / Linux overlap | Real turn |
+|---|---|---|---:|---|
+| `openai-completions` | `api: openai-completions` | `/v1/chat/completions` | 304 / 292 ms | not run — waived/unverified |
+| `openai-responses` | `api: openai-responses` | `/v1/responses` | 309 / 290 ms | not run — waived/unverified |
+| `anthropic-messages` | `api: anthropic-messages` | `/v1/messages` | 290 / 277 ms | not run — waived/unverified |
+| `openrouter` | `api: openai-completions` plus `thinkingFormat/sessionAffinityFormat: openrouter` | `/v1/chat/completions` | 297 / 288 ms | not run — waived/unverified |
+
+For each protocol, the packaged runtime sent four POST requests through `127.0.0.1`, used the selected model, included the custom header, injected the environment-only credential into the protocol-appropriate header (`Authorization` or `x-api-key`), and carried the PNG image in request history. Two independent workers settled successfully with `stop`, their provider-shaped turns overlapped, aborting the next turn in one worker produced `aborted`, and the unaffected worker still settled with `stop`. All workers exited by closing stdin.
+
+The Anthropic contract has a meaningful base URL difference: its SDK appends `/v1/messages`, so the configured base URL is the provider root. The OpenAI-style clients receive a base URL ending in `/v1` and append `chat/completions` or `responses`.
+
+This loopback result proves Pi's request serialization, SSE parsing, image/header/model/credential plumbing, worker overlap, and abort isolation without external cost. It does **not** qualify any real Provider Account, provider authentication, proxy behavior, rate limits, or provider-specific response variation.
+
+Pi's custom provider contract supports base URL, environment-resolved API key, custom headers, model id, and text/image input declaration. The probe accepts a credential-free fixture file shaped as follows; the secret remains only in the named environment variable:
+
+```json
+{
+  "id": "qualification-provider",
+  "apiProtocol": "openai-responses",
+  "baseUrl": "https://provider.example/v1",
+  "apiKeyEnv": "QUALIFICATION_PROVIDER_KEY",
+  "headers": { "X-Provider-Header": "literal-or-$ENV_REFERENCE" },
+  "model": {
+    "id": "provider-model-id",
+    "input": ["text", "image"],
+    "contextWindow": 128000,
+    "maxTokens": 8192
+  }
+}
+```
+
+Run a real sample with:
+
+```text
+pnpm run probe:pi-runtime -- --provider-fixture <fixture.json> [--image <image-path>]
+```
+
+The real-provider path is implemented to require two successful overlapping turns with distinct sessions, then abort a new turn in one worker while the other worker settles successfully. It reports only provider/model ids, environment-variable name, and header names; it does not print credential values.
+
+No recognized provider credential environment variable was available in the implementation environment, so no real-provider request was sent and no provider cost was incurred.
+
+Reproduce the local contract sample against an existing controlled artifact with:
+
+```text
+pnpm run probe:pi-provider-contracts -- --electron-executable <artifact-executable> --cli-path <artifact-app.asar-cli>
+```
+
+## Deferred macOS Execution Handoff For PI-150
+
+A pre-execution portability review found and fixed one supported-path defect: the artifact-side closure inspector derived `resources` from `dirname(process.execPath)`, which is correct on Windows/Linux but points to `Contents/MacOS/resources` on macOS. The probe now computes the platform resource directory once and passes it into the packaged process, preserving the actual macOS location `Contents/Resources/app.asar`. A focused path test and a real Windows controlled packaged regression pass verify the shared logic. The user explicitly deferred actual macOS execution from PI-000 to PI-150 on 2026-08-22, so no macOS Pass is claimed and the late-discovery risk remains.
+
+Use a separate checkout and frozen install for each target architecture. Before installing, `node -p "process.platform + ' ' + process.arch + ' ' + process.version"` must report `darwin x64` or `darwin arm64` for the sample being collected. Do not reuse one architecture's `node_modules` or Electron download for the other architecture.
+
+Run this sequence in each architecture-specific checkout:
+
+```text
+pnpm install --frozen-lockfile
+node scripts/probe-pi-runtime.mjs --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-workspace.json
+pnpm run probe:pi-runtime -- --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-staged.json
+pnpm run probe:pi-runtime:packaged -- --samples 5 --timeout-ms 10000 --report release/pi-runtime-macos-packaged.json
+```
+
+Read `artifact.executable` and `artifact.appAsar` from the packaged JSON report. The Pi CLI path is `<artifact.appAsar>/dist/cli.js`. Run the credential-free packaged four-protocol contract sample before replacing the controlled artifact:
+
+```text
+pnpm run probe:pi-provider-contracts -- --electron-executable <artifact.executable> --cli-path <artifact.appAsar>/dist/cli.js --artifact-label controlled-electron-builder-dir-app-asar --report release/pi-provider-contract-macos.json
+```
+
+For an optional, non-gating real Provider diagnostic, create one credential-free fixture per protocol being sampled and keep each secret only in the environment variable named by that fixture's `apiKeyEnv`. Reuse the existing packaged artifact through the runtime override so the run exercises the packaged Electron seam without rebuilding it for every protocol:
+
+```text
+node scripts/probe-pi-runtime.mjs --samples 5 --timeout-ms 10000 --electron-executable <artifact.executable> --cli-path <artifact.appAsar>/dist/cli.js --artifact-label controlled-electron-builder-dir-app-asar --provider-fixture <protocol-fixture.json> --image <qualification-image> --report release/pi-provider-<protocol>-macos.json
+```
+
+The user waived requiring these commands for `openai-completions`, `openai-responses`, `anthropic-messages`, and `openrouter`. If an optional diagnostic is run, preserve only redacted metrics and protocol/provider/model identifiers; do not commit fixtures containing secrets, generated reports, controlled artifacts, caches, or user data. A final process query must show no packaged probe or Pi RPC worker remains.
+
+## Gate Status
+
+| Gate | Status | Evidence or blocker |
+|---|---|---|
+| QG-001 exact identity and pin | Pass | exact npm/tag/commit/engine/CLI and lockfile pin |
+| QG-002 packaged Electron Node RPC lifecycle | Pass | Phase-0 scope is Windows x64 + Linux x64 by explicit user decision; both controlled artifacts spawn, become ready, open session, accept prompt, abort, settle, reopen, and exit cleanly. macOS is deferred to PI-150, not passed. |
+| QG-003 production dependency closure | Pass | Phase-0 controlled Windows/Linux artifacts resolve all 130/131 applicable packages and six runtime assets. Final product artifacts and deferred macOS closure belong to PI-150. |
+| QG-004 Provider Account matrix | Explicitly Waived / Accepted Risk | four-protocol loopback request/SSE/image contracts pass; real accounts, credentials, endpoints, proxies, authentication, rate-limit and provider-specific behavior remain unverified by user decision on 2026-08-22 |
+| QG-005 real-provider worker concurrency | Explicitly Waived / Accepted Risk | local shell and loopback protocol turns overlap with isolated abort/model/credential state; real provider overlap/serialization and abort/event/session/model/credential isolation remain unverified by user decision on 2026-08-22 |
+| QG-006 performance and cleanup metrics | Pass | Phase-0 Windows/Linux cold/warm/RSS/exit sample counts and p50/p95/max are recorded and within budgets. macOS metrics are explicitly deferred to PI-150. |
+| QG-007 stop-and-amend condition | Pending / not triggered | no confirmed packaging, Provider, or budget failure yet; any mandatory-gate failure stops migration and requires the specified utility-process SDK amendment |
+
+## Remaining And Optional Evidence
+
+1. Real Provider fixtures are optional diagnostics after the explicit waiver. Their absence does not block PI-000 or PI-150, but the current record must continue to report every real turn as unverified; a later confirmed failure is new architecture evidence, not something the waiver converts into a Pass.
+2. At PI-150, run the deferred macOS x64/arm64 workspace, staged, final packaged, loopback, closure/native/resource, and metrics matrix. Missing either architecture blocks a cross-platform release-ready claim even though it no longer blocks PI-000.
+3. If the accepted Linux definition requires a native desktop distribution or release package, repeat the Linux sample outside WSL2 at PI-150; the current Phase-0 evidence is deliberately labeled as Ubuntu 24.04 userspace on WSL2 with a controlled `dir` artifact.