Compare commits

...
3 Commits
28 changed files with 583 additions and 220 deletions

No files matched your search

+2 -2
View File
@@ -43,8 +43,8 @@ ZHINIAN_DATA_BACKEND=local
DATABASE_URL=
# Migration runner only: PostgreSQL role used by the Go API DATABASE_URL.
DATABASE_APP_ROLE=
# Optional URL query for an explicit RDS CA, for example:
# postgresql://user:password@rds-host:5432/app?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem
# PostgreSQL transport is intentionally plaintext; clients force sslmode=disable.
# Example: postgresql://user:password@rds-host:5432/app?sslmode=disable
DATABASE_POOL_MAX=10
DATABASE_IDLE_TIMEOUT_MS=30000
DATABASE_CONNECTION_TIMEOUT_MS=5000
@@ -28,6 +28,9 @@ The project exists to give organizations an Alibaba Cloud ACK-deployable AI crea
- Same-origin browser authentication through signed, chunked `zhinian_session` cookies with per-request account/organization/sessionVersion revalidation.
- Production persistence fails closed: explicit RDS PostgreSQL through the Go
PostgreSQL Adapter; local JSON is development/test only.
- PostgreSQL clients enforce plaintext (`sslmode=disable`) for the selected RDS
endpoint. Production database traffic must stay on the internal network and
be restricted by VPC, security-group, and RDS allowlist controls.
- Production Web is a static export on Nginx. Browser runtime requests stay
same-origin and all API/file/auth behavior belongs to Go.
- Cross-instance concurrency stays in PostgreSQL: `claim_generation_jobs` for job claims and `billing_post_wallet_entry` for wallet idempotency; no process-local lock replacements.
+1 -1
View File
@@ -4,7 +4,7 @@
| ID | Decision | Status | Date | Applies To | Detail |
|---|---|---|---|---|---|
| RDS-001 | Production persistence uses explicit direct PostgreSQL through one server-only adapter; local JSON is explicit development/test mode. | Accepted | 2026-08-12 | Server stores and scripts | `ZHINIAN_DATA_BACKEND=postgres` fails closed and never silently falls back. |
| RDS-001 | Production persistence uses explicit direct PostgreSQL through one server-only adapter; local JSON is explicit development/test mode. | Accepted; transport amended 2026-08-16 | 2026-08-12, amended 2026-08-16 | Server stores and scripts | `ZHINIAN_DATA_BACKEND=postgres` fails closed and never silently falls back. PostgreSQL clients enforce plaintext (`sslmode=disable`) because the selected RDS endpoint refuses TLS; production must use the internal endpoint with VPC, security-group, and allowlist isolation. |
| RDS-002 | Database changes use versioned, checksummed, advisory-locked migrations. | Accepted; execution amended 2026-08-14 | 2026-08-12 | Database schema and rollout | Initial production schema is executed manually from `database/migrations/*.sql` plus application-role grants; no migration Job manifest is shipped with the static Web image. |
| ADR-003 | Production is a same-origin static Next.js export on Nginx plus a Go modular-monolith backend with an embedded WorkerLoop. | Accepted; repository implementation complete in `b14b4fc`; production rollout pending | 2026-08-12, amended 2026-08-16 | Application and ACK architecture | Browser routes are static; all runtime API/file/auth responsibility belongs to Go. See `adr-003-next-go-target.md`. |
| DEP-001 | Production starts fresh with the ADR-003 static Web + Go topology, no legacy cutover or Node Worker, and manual SQL schema initialization. | Accepted; manifests updated 2026-08-16 | 2026-08-14 | Deployment model and schema initialization | No migration Job pod; Go bootstraps the super administrator, owns the session Secret, and receives all runtime/backend configuration. |
+8 -4
View File
@@ -15,7 +15,8 @@ and are not a supported production path.
## Approved Target Flows
The implementation and desired ACK routing are present in `b14b4fc`.
The static implementation and desired ACK routing are present in `b14b4fc`;
`ed97814` adds the plaintext PostgreSQL transport correction.
Production is already online, but the static revision and exact live Service
ownership have not been confirmed from cluster configuration or logs:
@@ -24,7 +25,7 @@ ownership have not been confirmed from cluster configuration or logs:
| Browser UI | Browser | Same-origin Ingress -> Nginx static Web | Preserve current page URLs; Nginx performs no application logic. |
| Browser identity | Browser auth Module | Same-origin Ingress -> Go `GET /api/auth/me` | Browser automatically sends HttpOnly Cookie; validate anonymous/authenticated response shapes, keep no token in JavaScript, and use client guards only for UX. |
| Browser business/file requests | Browser components | Same-origin Ingress -> Go `/api`, `/uploads`, `/generated-results` | Go revalidates session/account/organization/sessionVersion and enforces every protected action. |
| Backend persistence | Go Modules | PostgreSQL Adapter -> RDS | Parameterized queries and transactions; fail closed in production. |
| Backend persistence | Go Modules | PostgreSQL Adapter -> internal RDS endpoint | Parameterized queries and transactions; fail closed in production; code-enforced plaintext (`sslmode=disable`) within VPC/security-group/allowlist isolation. |
| Task execution | Embedded Go WorkerLoop | RDS claim -> provider -> OSS -> RDS -> Webhook | Bounded concurrency, recoverable leases, one owner for external side effects. |
| Asset lifecycle | Go Assets | OSS plus RDS metadata | Shared storage required before horizontal scaling. |
| Schema rollout | Manual operator or dedicated CI | RDS | Execute immutable versioned SQL plus grants outside long-lived workloads; no Web-image migration Job. |
@@ -39,7 +40,9 @@ ownership have not been confirmed from cluster configuration or logs:
## External Interfaces
- Alibaba Cloud RDS PostgreSQL via its internal endpoint and verified TLS CA.
- Alibaba Cloud RDS PostgreSQL via its internal endpoint using code-enforced
plaintext; VPC, security-group, and RDS allowlist controls are the transport
isolation boundary.
- Alibaba Cloud ACK resources under `deploy/ack/`.
- Live production at `https://nianxxaigc.nianxx.cn`; public `/api/ready` has returned HTTP 200 with PostgreSQL configured, without proving the owning Service.
- The legacy internal Worker prefix is denied by Ingress; production uses the
@@ -47,7 +50,8 @@ ownership have not been confirmed from cluster configuration or logs:
The accepted production topology uses the embedded Go WorkerLoop. The current
live revision produces an RSC error for authenticated `/create`; the rollout
must deploy `b14b4fc` under immutable image references and smoke login,
must deploy `ed97814` under immutable image references and smoke PostgreSQL
bootstrap/readiness, login,
authenticated routes, logout, roles, `/healthz`, `/api/health`, and
`/api/ready`.
+3 -2
View File
@@ -11,7 +11,7 @@
| `database/migrations/` | Immutable versioned PostgreSQL schema changes | Executed manually/through dedicated operator CI; no migration Job reuses Web. |
| `deploy/ack/` | Seven ACK manifests plus Secret template | Static Web + Go topology; Web has no runtime config/Secret and Go owns the session Secret. |
| `backend/cmd/zhinian-api` | Go application entrypoint, configuration, HTTP server composition, health/readiness | Sole runtime API owner targeted by checked-in Ingress. |
| `backend/internal/*` | ADR-003 deep modules and adapters, 18 packages: `identity`, `administration`, `assets`, `billing`, `usage`, `jobs`, `providers`, `webhook`, `httpapi`, `publicapi`, `application`, `orchestration`, `postgres`, `localstore`, `logging`, `settings`, `templates`, `prompt` | Implemented in `b14b4fc`; publication of immutable images and rollout of the static Web + Go revision remain pending. |
| `backend/internal/*` | ADR-003 deep modules and adapters, 18 packages: `identity`, `administration`, `assets`, `billing`, `usage`, `jobs`, `providers`, `webhook`, `httpapi`, `publicapi`, `application`, `orchestration`, `postgres`, `localstore`, `logging`, `settings`, `templates`, `prompt` | Implemented in `b14b4fc`; `ed97814` enforces plaintext PostgreSQL. Publication of immutable images and live rollout remain pending. |
| `contracts/**/*.json` | Language-neutral HTTP/Cookie/auth/jobs/billing/storage/webhook contract fixtures | Shared acceptance source for TypeScript and Go consumers. |
## Dependency Direction
@@ -46,7 +46,8 @@ Real internal seams are PostgreSQL transport, object storage, generation provide
- `database/migrations/` and the two concurrency-sensitive PostgreSQL functions.
- Account authentication/password transactions and billing wallet idempotency.
- ACK Secrets, RDS CA mounting, Ingress protection for internal Worker routes, and pool connection budgeting.
- ACK Secrets, private-network enforcement for unencrypted RDS traffic,
Ingress protection for internal Worker routes, and pool connection budgeting.
- `backend/internal/{postgres,jobs,billing}`: claim and wallet correctness across Go replica scaling until WorkerLoop concurrency is deliberate.
- Static Web image construction/container startup still needs CI smoke evidence.
- Go `emptyDir` file state is lost on Pod replacement when OSS is absent.
@@ -2,14 +2,19 @@
## Current Architecture
The first production deployment is online at `https://nianxxaigc.nianxx.cn`. The public `/api/ready` endpoint has been observed returning HTTP 200 with PostgreSQL configured. The exact live Service owner for each path has not been confirmed through cluster configuration or logs, so the deployed routing shape is not inferred here.
The first production deployment is online at `https://nianxxaigc.nianxx.cn`.
An earlier public `/api/ready` response returned HTTP 200 with PostgreSQL
configured, but the latest observed Go API startup fails during bootstrap
because the RDS endpoint refuses TLS. The exact live Service owner and deployed
image revision for each path have not been confirmed through cluster
configuration or logs.
The live revision predates `b14b4fc` and authenticated `/create` currently
triggers a production RSC error. The repository now implements the stricter
ADR-003 boundary: Next.js statically exports pages, unprivileged Nginx serves
them, the browser reads identity from Go `/api/auth/me`, and Go owns every
runtime API/file route plus database-backed authorization and the embedded
WorkerLoop. The new images and ACK configuration have not yet been deployed.
The repository implements the strict ADR-003 boundary: Next.js statically
exports pages, unprivileged Nginx serves them, the browser reads identity from
Go `/api/auth/me`, and Go owns every runtime API/file route plus database-backed
authorization and the embedded WorkerLoop. Revision `ed97814` additionally
forces PostgreSQL plaintext for the TLS-refusing RDS endpoint. Deployment and
live validation of that exact revision remain pending.
## Approved Target Architecture
@@ -17,9 +22,9 @@ The accepted target in ADR-003 is a same-origin static Next.js export on Nginx p
| Target component | Responsibility | Constraint | Implementation state |
|---|---|---|---|
| Static Web | Next.js build output (`out/`) and browser UI served by Nginx | No SSR, Middleware, Route Handlers, runtime configuration, application Secret, or internal Go URL. | Implemented and statically verified in `b14b4fc`; deployment pending. |
| Static Web | Next.js build output (`out/`) and browser UI served by Nginx | No SSR, Middleware, Route Handlers, runtime configuration, application Secret, or internal Go URL. | Implemented and statically verified in `b14b4fc`; publish the image from `ed97814` with the matching Go release. |
| Go backend | Existing HTTP/file contracts, identity, administration, assets, jobs, billing, usage, providers, storage, Webhooks, readiness | Owns relational access and embeds the WorkerLoop in the approved topology. | Implemented in `backend/`; production is online, but exact live path ownership is not asserted without cluster evidence. |
| RDS PostgreSQL | Relational state and cross-instance concurrency | Retains versioned migrations and both concurrency-sensitive database functions. | Repository procedure requires manual SQL (migrations 0001/0002) plus role grants; live execution evidence remains to be confirmed. |
| RDS PostgreSQL | Relational state and cross-instance concurrency | Retains versioned migrations and both concurrency-sensitive database functions. Clients enforce plaintext (`sslmode=disable`); use only the internal endpoint protected by VPC, security groups, and an RDS allowlist. | Repository procedure requires manual SQL (migrations 0001/0002) plus role grants; live plaintext bootstrap/readiness and network-isolation evidence remain to be confirmed. |
| Schema operator/CI | Schema and application-role grants | Runs versioned SQL outside long-lived workloads; never reuses the static Web image. | First-deployment procedure is manual; no migration Job manifest is shipped. |
| Alibaba Cloud OSS | Shared generated/uploaded assets | Must be production-ready before horizontal workload scaling. | Still behind a storage Adapter; not validated against real OSS. |
@@ -42,6 +47,9 @@ replica until file storage is shared.
## Important Boundaries
- Production backend selection is explicit and fail-closed; never turn a PostgreSQL configuration failure into local JSON fallback.
- PostgreSQL transport is code-enforced plaintext because the selected RDS
endpoint refuses TLS. Database traffic must stay on the Alibaba Cloud private
network and be restricted with VPC, security-group, and allowlist controls.
- Store callers depend on stable store interfaces, not `pg` or SQL details.
- Multi-statement consistency uses one transaction client; atomic job claim and wallet posting remain database functions.
- Database credentials and the session-signing Secret belong to Go and the
@@ -56,8 +64,9 @@ replica until file storage is shared.
## Related Decisions
- Current implementation: `RDS-001` and `RDS-002` (schema execution now manual SQL per `DEP-001`).
- Accepted implementation: `ADR-003` as amended by `b14b4fc`; production is
online, but the new static revision and exact live routing remain unverified.
- Accepted implementation: `ADR-003` as amended by `b14b4fc`, plus the
`RDS-001` transport amendment implemented in `ed97814`; production is online,
but the exact revision and live routing remain unverified.
- First-deployment model: `DEP-001`.
## Last Updated
+36 -18
View File
@@ -17,23 +17,28 @@ This file is the integrated default-branch snapshot. Feature tasks record progre
- `b14b4fc` (pure static Next.js export, browser-to-Go auth, Go-only runtime API
ownership, unprivileged Nginx Web, and first-deployment ACK cleanup; task
`20260816-static-frontend-go-api-4f8c2a7d`)
- `acf368b` (diagnosis of the production Go bootstrap failure against an RDS
endpoint that refuses PostgreSQL TLS)
- `ed97814` (code-enforced plaintext PostgreSQL for Go and retained Node
tooling, removal of the obsolete RDS CA deployment dependency, and protocol
regression coverage; task `20260816-disable-postgres-tls-d4a89c12`)
## Current Focus
The first production deployment is live at `https://nianxxaigc.nianxx.cn`; the
currently observed revision still fails authenticated `/create` with an RSC
error. Repository revision `b14b4fc` removes that request-time frontend seam:
Next.js now emits static `out/` files served by unprivileged Nginx, the browser
loads identity from same-origin Go `/api/auth/me`, and Ingress routes all
`/api`, `/uploads`, and `/generated-results` traffic directly to Go. Web has no
runtime ConfigMap, Secret, database credential, or internal Go URL. The new
images/manifests have not yet been deployed, and exact live Service ownership
still requires cluster evidence.
The first production deployment is live at `https://nianxxaigc.nianxx.cn`.
After a redeployment, the observed Go API process fails during super-admin
bootstrap because its RDS endpoint refuses a TLS negotiation. Repository
revision `ed97814` retains the static Web + Go-only runtime boundary from
`b14b4fc` and forces every maintained PostgreSQL client to plaintext
(`sslmode=disable`), without requiring an Alibaba Cloud RDS configuration
change. The fixed Go image and updated ACK manifest have not yet been verified
in the live cluster, and the exact deployed image revisions still require
cluster evidence.
## Recently Completed
- 2026-08-12: Replaced the Supabase/PostgREST runtime path with a server-only `pg` adapter across data, account, and billing stores.
- 2026-08-12: Added versioned PostgreSQL migrations, strict backend selection, verified-CA TLS, database readiness, and ACK Web/Worker/migration manifests.
- 2026-08-12: Added versioned PostgreSQL migrations, strict backend selection, database readiness, and ACK Web/Worker/migration manifests; the original verified-CA transport decision was amended on 2026-08-16.
- 2026-08-12: Accepted and documented the Next.js frontend plus Go backend target, migration contracts, and acceptance criteria.
- 2026-08-14: Implemented and merged the Go backend (foundation, identity, administration, assets, billing, usage, jobs/providers/webhooks/worker loop, public and compatibility HTTP surfaces) with language-neutral contract fixtures and migration 0002.
- 2026-08-14: Reconciled canonical architecture, decision, history, commitment, and positioning memory with the merged Go implementation (task `20260814-go-memory-reconcile-7f2a9c41`).
@@ -47,23 +52,30 @@ still requires cluster evidence.
Worker/migration manifests, and added static/deployment regressions (task
`20260816-static-frontend-go-api-4f8c2a7d`, commit `b14b4fc`; not yet
deployed).
- 2026-08-16: Diagnosed the Go bootstrap failure as a TLS negotiation against
an endpoint that refuses TLS, then changed Go and retained Node PostgreSQL
clients to enforce plaintext, removed the obsolete CA deployment dependency,
and added real wire-protocol regression coverage (task
`20260816-disable-postgres-tls-d4a89c12`, commit `ed97814`; live rollout not
yet verified).
## In Progress
- Build, publish, and deploy immutable Web and Go images for `b14b4fc`, then
verify the static Web + Go-only runtime boundary in the live ACK cluster.
- Build, publish, and deploy immutable Web and Go images containing `ed97814`,
then verify PostgreSQL readiness/bootstrap and the static Web + Go-only
runtime boundary in the live ACK cluster.
## Next Recommended Steps
1. Build and smoke the pinned unprivileged Nginx Web image in CI or another
host with Docker, then publish Web and Go images under new immutable tags or
digests.
1. Build and smoke Web and Go images from `ed97814` in CI or another host with
Docker, then publish them under new immutable tags or digests.
2. Create/verify the `zhinian` Namespace, run target-cluster server-side dry
runs, apply the production Go-owned Secret and six checked-in resource
manifests (not `secrets.example.yaml`), and confirm live Ingress/Service
ownership from cluster state.
3. Smoke anonymous login, authenticated `/create?mode=video`, logout, and each
admin role; verify Web `/healthz`, Go `/api/health`, and Go `/api/ready`.
3. Confirm Go bootstrap completes without an SSLRequest, then smoke anonymous
login, authenticated `/create?mode=video`, logout, and each admin role;
verify Web `/healthz`, Go `/api/health`, and Go `/api/ready`.
4. Configure OSS or another shared/persistent store before any Go Pod
replacement that must preserve current local uploads/generated results.
5. Continue real RDS/provider/Webhook validation and confirm the public
@@ -71,7 +83,10 @@ still requires cluster evidence.
## Open Questions / Blockers
- Canonical memory does not yet record the live RDS PostgreSQL version, connection budget, endpoint, TLS/CA details, database roles, ACK network policy, or confirmed request-path Service ownership.
- Canonical memory does not yet record the live RDS PostgreSQL version,
connection budget, exact internal endpoint, database roles, effective VPC /
security-group / allowlist controls, or confirmed request-path Service
ownership.
- Real OSS bucket/credential configuration is still needed for shared asset storage.
- Public `/api/v1` support promises for external consumers need explicit confirmation.
- The static Web Docker image has not been built or container-smoked in this
@@ -79,6 +94,9 @@ still requires cluster evidence.
## Risky Areas
- PostgreSQL transport is intentionally unencrypted. The RDS connection must
remain on the private network and be constrained by VPC, security-group, and
allowlist controls; those live controls still require recorded validation.
- Database grants and least-privilege roles still require documented validation against the live RDS instance.
- Go currently stores local uploads/results on `emptyDir` when OSS is absent;
Pod replacement or rolling update loses them, not only horizontal scaling.
+2
View File
@@ -20,6 +20,8 @@
| 2026-08-16 | `20260816-integrate-auth-ssr-9d7e4c2a` | Serialized integration of source commit `498c2fa` and canonical reconciliation for the authenticated SSR-to-Go identity bridge. No live deployment was performed. | Current state, task history, system overview, module map, data flow, commitments |
| 2026-08-16 | `20260816-static-frontend-go-api-4f8c2a7d` | Revision `b14b4fc` replaces production SSR/Middleware/Next APIs with a static export on unprivileged Nginx; browser runtime traffic goes directly to the Go-owned same-origin API/file surface. | Task record, proposal, application/deployment docs |
| 2026-08-16 | `20260816-integrate-static-frontend-2c7e91b4` | Serialized canonical promotion of the user-approved static Web + Go-only runtime architecture. | Positioning, success criteria, ADR-003, decision index, current state, architecture, domain rules, commitments, task history |
| 2026-08-16 | `20260816-disable-postgres-tls-d4a89c12` | Revision `ed97814` forces plaintext PostgreSQL in Go and retained Node clients, removes the obsolete RDS CA deployment dependency, and adds wire-level regression coverage for an endpoint that refuses TLS. No live deployment was performed. | Task record, application/deployment docs |
| 2026-08-16 | `20260816-integrate-plaintext-postgres-6e3b1a90` | Serialized canonical promotion of the user-approved plaintext PostgreSQL transport decision and its private-network security boundary. | Positioning, decision index, current state, architecture, commitments, task history |
## Notes
@@ -0,0 +1,97 @@
# Task: Diagnose Go RDS TLS startup failure
## Identity
- Task ID: 20260816-diagnose-go-rds-tls-9c4a7e21
- Mode: Feature
- Branch: main
- Worktree: D:\Datas\OthersProjects\NianAIGC
- Base commit: bb50d06d1da67556571eb5e0fdb7312339842e71
- Owner: codex
- Status: Ready for Integration
## Scope
- Diagnose the Go API startup failure whose root error is PostgreSQL `server
refused TLS connection` during first-super-administrator account listing.
- Verify the repository TLS defaults and ACK Secret contract without reading
or exposing production credentials.
- Keep the task diagnostic-only; do not change application behavior or mutate
the live ACK/RDS environment without cluster access and explicit operational
execution.
## Intent And Constraints
- Build a deterministic local PostgreSQL wire-protocol feedback loop that
exercises the real Go administration list path.
- Separate confirmed protocol behavior from the unverified live RDS control
plane setting.
- Prefer verified TLS for production; document `sslmode=disable` only as a
deliberate plaintext fallback when the operator accepts that tradeoff.
- Redact credentials and delete all temporary diagnostic code before finish.
## Outcome
- Confirmed the Go PostgreSQL configuration defaults to `verify-full` when
`DATABASE_URL` omits `sslmode`; the checked-in Secret example also specifies
`verify-full` and an RDS CA path.
- Reproduced the exact production error chain against a local PostgreSQL
protocol endpoint that returns `N` to `SSLRequest`: `list accounts` -> `list
administration accounts` -> `tls error (server refused TLS connection)`.
- Re-ran the identical administration query path against the same endpoint
with `sslmode=disable`; it passed. This isolates the failure to a mismatch
between client TLS mode and endpoint TLS capability, before credentials,
grants, schema, or bootstrap creation are evaluated.
- The most likely live cause is that the RDS instance/selected connection
address does not have SSL enabled while the deployed `DATABASE_URL` requests
verified TLS. Wrong endpoint/port or a TLS-refusing intermediary remain
lower-probability alternatives until checked from the Go Pod/RDS console.
- No product source, deployment manifest, or canonical project-memory file was
changed. The temporary diagnostic test was deleted.
## Verification
- RED: `go test ./internal/application -run
TestDiagnosticListAccountsAgainstTLSRefusingEndpoint -count=1 -v` reproduced
the exact `server refused TLS connection` error on the production query path.
- GREEN: with diagnostic mode set to `disable`, the same command and fake
endpoint passed.
- A credential-free SSLRequest probe to the real RDS host timed out from this
workstation; this is consistent with a private/VPC endpoint but does not
establish the RDS SSL setting.
- `kubectl config current-context` reported no configured context, so no live
Secret, Pod DNS result, or RDS endpoint was inspected or mutated.
- Alibaba Cloud's current RDS PostgreSQL documentation confirms that SSL must
be enabled for SSL-mode connections, `disable` is the non-SSL client mode,
and enabling/changing the protected address restarts the instance with a
minute-level interruption risk.
## Follow-ups
- In the RDS console, verify whether SSL is enabled and whether the protected
connection address exactly matches the host used by the Go Pod.
- Recommended production repair: enable RDS SSL for that address, mount the
downloaded CA as `zhinian-rds-ca`, retain `sslmode=verify-full`, update the
Go database Secret, and restart/smoke the Go Deployment.
- Temporary recovery alternative: explicitly use `sslmode=disable` in the Go
`DATABASE_URL`, accepting plaintext database transport inside the network,
then restart and verify the Deployment.
- After TLS negotiation succeeds, separately validate credentials, grants,
migrations 0001/0002, and super-administrator bootstrap.
## Promotion Candidates
- Target: production RDS validation evidence and deployment troubleshooting
guidance.
Proposal: record the live RDS SSL state/protected address and add a
credential-free SSLRequest check before deploying a `verify-full` database
URL.
Evidence: the Go startup failure and deterministic RED/GREEN protocol loop
recorded above.
Future impact: prevents fail-closed Go startup from being mistaken for an
account/bootstrap defect when the endpoint refuses TLS.
Semantic conflicts: none; this strengthens existing RDS-001/RDS hardening
commitments.
Human confirmation required: no for evidence/runbook promotion; enabling or
disabling production RDS SSL remains an operator decision because it can
restart the instance or weaken transport security.
@@ -0,0 +1,94 @@
# Task: Disable PostgreSQL TLS for Go production
## Identity
- Task ID: 20260816-disable-postgres-tls-d4a89c12
- Mode: Feature
- Branch: main
- Worktree: D:\Datas\OthersProjects\NianAIGC
- Base commit: acf368b6fe290f44157ada79448673ab11808f7c
- Owner: codex
- Status: Ready for Integration
## Scope
- Change the Go PostgreSQL adapter so production never negotiates TLS, even
when the existing `DATABASE_URL` contains `sslmode=verify-full` and
`sslrootcert` from the previous deployment template.
- Align the Node migration client, ACK Secret/template checks, Go Deployment,
environment example, and deployment guide with plaintext PostgreSQL.
- Preserve explicit PostgreSQL selection, credentials, authorization,
migrations, pooling, and fail-closed startup behavior.
## Intent And Constraints
- The user explicitly chose code-side plaintext transport instead of changing
Alibaba Cloud RDS SSL configuration and accepted the resulting lack of
database link encryption.
- A newly built Go image must start with the existing TLS-bearing Secret; no
live Secret or RDS control-plane mutation is part of this task.
- Work test-first at the configuration/connection seam that reproduced the
production `server refused TLS connection` failure.
- Keep the change surgical and do not alter API, authentication, billing,
storage, or database schema behavior.
## Outcome
- Go `ParseConfig` removes case-insensitive `sslmode`/`sslrootcert` values,
writes `sslmode=disable`, never reads a CA, and records plaintext mode.
- Go `Open` normalizes the URL again and explicitly clears pgx `TLSConfig` and
fallbacks, so a direct or legacy `Config` cannot negotiate TLS.
- The migration Node client and retained server-only TypeScript adapter apply
the same normalization and pass `ssl: false` to `pg`.
- ACK no longer defines or mounts `zhinian-rds-ca`; migration and Go Secret
examples use `sslmode=disable`.
- Environment examples, both READMEs, deployment guidance, and manifest
assertions now disclose plaintext PostgreSQL transport and require private
network isolation.
- No live Alibaba Cloud or ACK configuration was changed.
## Verification
- RED: new Go tests failed because the old parser read `sslrootcert`, defaulted
to `verify-full`, and `Open` required TLS configuration.
- RED: new Node tests failed because the old clients tried to read a missing CA
and the ACK checker found the retained CA mount.
- GREEN: `go test ./...` passed for every backend package.
- GREEN: `npm test` passed 53 files / 160 tests.
- GREEN: `npx tsc --noEmit --incremental false` passed.
- GREEN: `npm run deploy:check` passed all seven checked-in ACK manifests.
- GREEN: `node --check` passed for both modified Node scripts.
- GREEN: `npm run build` exported all 14 static pages successfully.
- `git diff --check` passed with line-ending warnings only.
- First read-only `sol_reviewer` verdict: FAIL because `backend/README.md`
retained TLS claims and tests did not observe a real PostgreSQL startup
packet; both findings were fixed.
- Final read-only `sol_reviewer` verdict: PASS on both Standards and Spec after
the Go wire test observed plaintext StartupMessage `196608` (not SSLRequest
`80877103`) and the executable TypeScript configuration test passed.
## Follow-ups
- Build and deploy an immutable Go image, then verify startup bootstrap and
`/api/ready` against the live RDS instance.
- Reapplying the database Secret is not required for TLS removal because the
new clients override old TLS parameters, but the checked-in plaintext Secret
template should be used for future rotations.
- Reassess TLS if the network boundary or compliance requirements change.
## Promotion Candidates
- Target: RDS-001/current architecture/database deployment commitments.
Proposal: record that production PostgreSQL transport is intentionally
plaintext and code-enforced, superseding the prior verified-CA TLS default.
Evidence: production RDS refused TLS, the deterministic diagnosis task
`20260816-diagnose-go-rds-tls-9c4a7e21`, and the user's explicit 2026-08-16
instruction not to modify Alibaba Cloud and to remove TLS in code.
Future impact: future deployment templates and database clients must not
silently reintroduce TLS without a new operator decision and compatible RDS
configuration.
Semantic conflicts: canonical current state and commitments still describe
verified-CA TLS as the prior target.
Human confirmation required: already received for plaintext production
transport; canonical promotion still requires the serialized Integration
Gate.
@@ -0,0 +1,66 @@
# Task: Integrate plaintext PostgreSQL decision
## Identity
- Task ID: 20260816-integrate-plaintext-postgres-6e3b1a90
- Mode: Integration
- Branch: main
- Worktree: D:\Datas\OthersProjects\NianAIGC
- Base commit: ed978142ebbea4ed8e4d3743f9ece42a334c6ea5
- Owner: codex
- Status: Ready for Integration
## Scope
- Promote the completed plaintext PostgreSQL implementation and decision into
canonical project memory.
- Reconcile stale verified-CA/TLS wording in current architecture, deployment
commitments, and current-state guidance.
- Do not change application code, Alibaba Cloud configuration, or historical
task/proposal records.
## Intent And Constraints
- The user explicitly requires a code-only correction and no Alibaba Cloud RDS
configuration change.
- PostgreSQL clients must enforce plaintext even when an existing Secret still
contains TLS query parameters.
- Canonical memory must disclose that transport confidentiality now depends on
the internal endpoint plus VPC, security-group, and allowlist isolation.
- Source feature task `20260816-disable-postgres-tls-d4a89c12` and commit
`ed97814` are read-only inputs to this integration task.
## Outcome
- Canonical `RDS-001` now records the plaintext transport amendment and its
required private-network isolation boundary.
- Current state, architecture, positioning, commitments, and history now point
rollout at `ed97814` and no longer describe verified-CA TLS as the target.
- No application code, cloud configuration, or deployment state was changed by
this integration task.
## Verification
- Source implementation final `sol_reviewer`: PASS for Standards and Spec after
both identified gaps were fixed.
- `check_project_docs.py --target .`: PASS.
- `check_doc_drift.py --target . --task-id
20260816-integrate-plaintext-postgres-6e3b1a90`: PASS; only this integration
task record and authorized canonical documents changed.
- `git diff --check`: PASS (line-ending conversion warnings only).
- First read-only integration review: FAIL on one stale TLS/CA maintenance item
and this record's pending verification state; both findings were remediated.
- Final read-only integration re-review: PASS; both initial documentation
findings are closed and no residual Standards or Spec blocker remains.
## Follow-ups
- Build, publish, and deploy immutable Web and Go images from `ed97814`.
- Apply the checked-in Go manifest without the obsolete CA mount and verify
bootstrap/readiness against the real internal RDS endpoint.
- Record effective VPC, security-group, allowlist, database-role, and live
request-path ownership evidence without exposing credentials.
## Promotion Candidates
- None; this integration task directly updates canonical memory.
+3 -3
View File
@@ -4,7 +4,7 @@ Track future-facing memory: promised follow-ups, unfinished loops, timed checks,
| Date | Commitment | Trigger / Due | Owner | Status | Next Action |
|---|---|---|---|---|---|
| 2026-08-12 | Validate migration, TLS, permissions, and readiness against the real Alibaba Cloud RDS instance. | Production hardening follow-up | Deployment owner | Open | Record the live RDS roles/CA/network configuration and validation evidence; public `/api/ready` currently returns HTTP 200 with PostgreSQL configured. |
| 2026-08-12 | Validate migrations, plaintext connectivity, permissions, and readiness against the real Alibaba Cloud RDS instance. | Production hardening follow-up | Deployment owner | Open | Deploy `ed97814`, then record live roles, internal endpoint, VPC/security-group/allowlist controls, bootstrap, and readiness evidence. Do not treat the earlier public `/api/ready` response as evidence for the new image. |
| 2026-08-12 | Keep Go at one replica until generated assets use OSS or another shared store; static Web may scale independently. | Before raising Go replicas or replacing a Pod whose local files must survive | Deployment owner | Open | Configure and validate external object storage; absent OSS, `emptyDir` files are lost on Pod replacement. |
| 2026-08-12 | Harden production runtimes to non-root with explicit writable paths. | Security hardening follow-up | Application owner | Completed | Go already uses UID 10001; `b14b4fc` moves Web to unprivileged Nginx UID/GID 101 with read-only root and a `/tmp` volume. |
| 2026-08-12 | Implement ADR-003 only after executable compatibility contracts exist. | Before starting the Go migration | Application owner | Completed | Contracts exist under `contracts/`; Go implementation merged 2026-08-14. |
@@ -13,8 +13,8 @@ Track future-facing memory: promised follow-ups, unfinished loops, timed checks,
| 2026-08-14 | Confirm the live status of `ZHINIAN_BOOTSTRAP_ADMIN_PHONE` / `ZHINIAN_BOOTSTRAP_ADMIN_PASSWORD`; the Go process creates the first super administrator exactly once and uses the default name when `ZHINIAN_BOOTSTRAP_ADMIN_NAME` is absent. | Production configuration audit | Deployment owner | Open | Verify deployed configuration and bootstrap outcome without exposing credentials. |
| 2026-08-14 | Validate the Go backend against non-production RDS, real OSS, provider credentials, and external Webhooks. | Production hardening follow-up | Deployment owner | Open | Run the contract and recovery suites against representative external dependencies and record any live parity gaps. |
| 2026-08-14 | Confirm the public `/api/v1` support promise for external consumers. | Production compatibility follow-up | Product owner | Open | Product decision on which v1 endpoints and behaviors are guaranteed for partners. |
| 2026-08-16 | Release and smoke-test authenticated `/create` SSR through the internal Go identity bridge. | Superseded by the static architecture decision | Deployment owner | Superseded | Do not deploy the SSR bridge as the target architecture; deploy `b14b4fc` static Web + Go instead. |
| 2026-08-16 | Publish and deploy `b14b4fc` static Web + Go under immutable image references, then smoke the full same-origin boundary. | Before closing the production login repair | Deployment owner | Open | Build/container-smoke Web, server-side dry-run ACK, verify live path owners, anonymous login, `/create?mode=video`, logout, role pages, Web `/healthz`, Go `/api/health`, and Go `/api/ready`. |
| 2026-08-16 | Release and smoke-test authenticated `/create` SSR through the internal Go identity bridge. | Superseded by the static architecture decision | Deployment owner | Superseded | Do not deploy the SSR bridge as the target architecture; deploy the static Web + Go release containing `ed97814` instead. |
| 2026-08-16 | Publish and deploy `ed97814` static Web + Go under immutable image references, then smoke PostgreSQL bootstrap and the full same-origin boundary. | Before closing the production login repair | Deployment owner | Open | Build/container-smoke both images, server-side dry-run ACK, verify no PostgreSQL SSLRequest, confirm live path owners, anonymous login, `/create?mode=video`, logout, role pages, Web `/healthz`, Go `/api/health`, and Go `/api/ready`. |
## Use
+4 -1
View File
@@ -9,7 +9,10 @@ This is the integrated registry of stale or conflicting canonical memory. Update
## Missing Context
- RDS deployment inputs (target version, connection budget, endpoint, TLS/CA bundle, database roles, ACK network policy) — needed before production validation.
- RDS deployment inputs (target version, connection budget, exact internal
endpoint, database roles, effective VPC/security-group/RDS allowlist policy)
— needed before production validation of the intentionally plaintext
connection.
- Real OSS bucket/credential configuration — needed before horizontal scaling or Go asset validation.
- The public `/api/v1` support promise for external consumers — needed before the first production deployment.
+1 -1
View File
@@ -225,7 +225,7 @@ cp .env.example .env.local
- `ALI_OSS_*`:用于上传素材和生成结果转存
- `ZHINIAN_DATA_BACKEND`:生产使用 `postgres`,开发可使用 `local`
- `DATABASE_URL`:仅服务端读取的 PostgreSQL 连接串
- `DATABASE_URL` 的 `sslmode` / `sslrootcert`:在同一个连接串中配置 RDS TLS;默认使用 `sslmode=verify-full`
- PostgreSQL 客户端强制使用 `sslmode=disable` 且不读取 CA。ACK 到 RDS 的数据库链路为明文,只应使用 RDS 内网地址,并通过 VPC、安全组和白名单限制访问。
当 `ZHINIAN_DATA_BACKEND=local` 时,应用使用 `.runtime/data/web-app-state.json` 作为单实例开发数据层。生产 `postgres` 模式缺少连接配置会直接失败,不会静默写入本地 JSON。如果 OSS 未配置,上传和 mock 结果会保存到 `.runtime/uploads` 和 `.runtime/generated-results`,并通过 Go 路由提供访问。
+1 -1
View File
@@ -277,7 +277,7 @@ cp .env.example .env.local
| `ALI_OSS_*` | 上传素材和生成结果转存配置 |
| `ZHINIAN_DATA_BACKEND` | `postgres` 或 `local` |
| `DATABASE_URL` | PostgreSQL 连接串(仅放 Secret) |
| `DATABASE_URL` 的 `sslmode` / `sslrootcert` | 在同一个连接串中配置 RDS TLS;默认使用 `sslmode=verify-full` |
| PostgreSQL 传输 | 客户端强制 `sslmode=disable` 且不读取 CA;ACK 到 RDS 的链路为明文,只应走内网并通过 VPC、安全组和白名单限制访问 |
`ZHINIAN_DATA_BACKEND=local` 时,应用使用 `.runtime/data/web-app-state.json` 作为单实例开发数据层;生产 `postgres` 模式配置错误会直接失败。未配置 OSS 时,上传和生成结果会写入 `.runtime/uploads` 与 `.runtime/generated-results`。
+8 -5
View File
@@ -20,9 +20,10 @@ Implemented Modules:
tenant-scoped reports.
- `templates`, `prompt`, `settings`, and `logging`: the remaining compatibility
modules used by the HTTP surface.
- `postgres`: fail-closed configuration, verified-CA TLS, readiness, atomic
account mutations, and calls to the existing claim and wallet PostgreSQL
functions. PostgreSQL is the production relational source of truth.
- `postgres`: fail-closed configuration, code-enforced plaintext
`sslmode=disable`, readiness, atomic account mutations, and calls to the
existing claim and wallet PostgreSQL functions. PostgreSQL is the production
relational source of truth.
- `localstore`: a mutex-protected, non-durable, single-process development
store covering the same business Module ports.
- `httpapi`: the complete checked-in route compatibility surface.
@@ -107,5 +108,7 @@ bootstrap-creates accounts.
Production routing, Secret ownership, probes, and rollout commands are defined
in [`../docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md) and `../deploy/ack/`. Go owns
the session signing Secret and backend runtime configuration; the static Web
workload receives neither. Validate RDS/CA, OSS, providers, Webhooks, embedded
Worker recovery, and rollback behavior for each production release.
workload receives neither. PostgreSQL does not use TLS, so production must use
the RDS internal endpoint and restrict access with VPC boundaries, security
groups, and allowlists. Validate RDS connectivity, OSS, providers, Webhooks,
embedded Worker recovery, and rollback behavior for each production release.
+17 -31
View File
@@ -2,7 +2,6 @@ package postgres
import (
"crypto/tls"
"crypto/x509"
"fmt"
"net/url"
"strconv"
@@ -39,7 +38,7 @@ type Config struct {
ApplicationName string
}
func ParseConfig(getenv Getenv, readFile ReadFile) (Config, error) {
func ParseConfig(getenv Getenv, _ ReadFile) (Config, error) {
if getenv == nil {
return Config{}, fmt.Errorf("environment getter is required")
}
@@ -49,10 +48,9 @@ func ParseConfig(getenv Getenv, readFile ReadFile) (Config, error) {
ConnectionTimeout: 10 * time.Second,
StatementTimeout: 30 * time.Second,
ApplicationName: "zhinian-go",
// DATABASE_URL is the only database setting. Production defaults to
// full TLS verification; sslrootcert may be supplied in the URL when
// the RDS CA is not part of the container's system trust store.
SSLMode: SSLVerifyFull,
// Production RDS currently refuses TLS. Keep the transport choice in
// code so an older Secret cannot silently re-enable negotiation.
SSLMode: SSLDisable,
}
backend := strings.ToLower(strings.TrimSpace(getenv("ZHINIAN_DATA_BACKEND")))
@@ -97,35 +95,23 @@ func ParseConfig(getenv Getenv, readFile ReadFile) (Config, error) {
return Config{}, fmt.Errorf("DATABASE_URL must use the postgres:// or postgresql:// scheme")
}
query := parsed.Query()
for key := range query {
for key, values := range query {
lower := strings.ToLower(key)
if strings.HasPrefix(lower, "ssl") && lower != "sslmode" && lower != "sslrootcert" {
return Config{}, fmt.Errorf("DATABASE_URL contains unsupported SSL query parameter %s; use sslmode and optional sslrootcert", key)
return Config{}, fmt.Errorf("DATABASE_URL contains unsupported SSL query parameter %s; PostgreSQL transport is forced to sslmode=disable", key)
}
if lower == "sslmode" {
for _, value := range values {
mode := strings.ToLower(strings.TrimSpace(value))
if mode != "" && mode != string(SSLDisable) && mode != string(SSLVerifyFull) {
return Config{}, fmt.Errorf("DATABASE_URL sslmode must be 'disable' or 'verify-full'")
}
}
}
}
if mode := strings.ToLower(strings.TrimSpace(query.Get("sslmode"))); mode != "" {
cfg.SSLMode = SSLMode(mode)
}
switch cfg.SSLMode {
case SSLDisable:
case SSLVerifyFull:
cfg.TLSConfig = &tls.Config{MinVersion: tls.VersionTLS12}
if path := strings.TrimSpace(query.Get("sslrootcert")); path != "" {
if readFile == nil {
return Config{}, fmt.Errorf("a certificate reader is required when DATABASE_URL contains sslrootcert")
}
pem, readErr := readFile(path)
if readErr != nil {
return Config{}, fmt.Errorf("read DATABASE_URL sslrootcert: %w", readErr)
}
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM(pem) {
return Config{}, fmt.Errorf("DATABASE_URL sslrootcert does not contain a valid CA certificate")
}
cfg.TLSConfig.RootCAs = roots
}
default:
return Config{}, fmt.Errorf("DATABASE_URL sslmode must be 'disable' or 'verify-full'")
cfg.DatabaseURL, err = forcePlaintextDatabaseURL(cfg.DatabaseURL)
if err != nil {
return Config{}, fmt.Errorf("normalize DATABASE_URL: %w", err)
}
return cfg, nil
}
+27 -50
View File
@@ -1,15 +1,8 @@
package postgres
import (
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -60,52 +53,36 @@ func TestParseConfigAcceptsOnlyPostgresSchemesAndRejectsUnsupportedSSLQueryParam
}
}
func TestParseConfigBuildsVerifyFullTLSFromURLCA(t *testing.T) {
dir := t.TempDir()
caPath := filepath.Join(dir, "ca.pem")
const ca = "-----BEGIN CERTIFICATE-----\nMIIB\n-----END CERTIFICATE-----\n"
read := func(path string) ([]byte, error) {
if path != caPath {
t.Fatalf("read path = %q, want %q", path, caPath)
}
return []byte(ca), nil
}
_, err := ParseConfig(env(map[string]string{
"ZHINIAN_DATA_BACKEND": "postgres",
"DATABASE_URL": "postgresql://db.example/app?sslmode=verify-full&sslrootcert=" + url.QueryEscape(caPath),
}), read)
if err == nil || !strings.Contains(err.Error(), "CA certificate") {
t.Fatalf("ParseConfig() error = %v, want invalid CA certificate error", err)
}
}
func TestParseConfigVerifyFullBuildsRootsWithoutDisablingVerification(t *testing.T) {
key, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatal(err)
}
template := &x509.Certificate{
SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "test CA"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
IsCA: true, BasicConstraintsValid: true, KeyUsage: x509.KeyUsageCertSign,
}
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
ca := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
func TestParseConfigForcesPlaintextAndDoesNotReadURLCA(t *testing.T) {
readCalled := false
cfg, err := ParseConfig(env(map[string]string{
"ZHINIAN_DATA_BACKEND": "postgres",
"DATABASE_URL": "postgresql://db.example/app?sslmode=verify-full&sslrootcert=%2Fca.pem",
}), func(string) ([]byte, error) { return ca, nil })
"DATABASE_URL": "postgresql://db.example/app?connect_timeout=5&sslmode=verify-full&sslrootcert=%2Fetc%2Fzhinian%2Frds%2Fca.pem",
}), func(string) ([]byte, error) {
readCalled = true
return nil, os.ErrNotExist
})
if err != nil {
t.Fatalf("ParseConfig() error = %v", err)
}
if cfg.TLSConfig == nil || cfg.TLSConfig.RootCAs == nil {
t.Fatal("TLSConfig.RootCAs is nil")
if readCalled {
t.Fatal("ParseConfig() read sslrootcert, want plaintext configuration without CA access")
}
if cfg.TLSConfig.InsecureSkipVerify {
t.Fatal("TLSConfig.InsecureSkipVerify = true, want full certificate and hostname verification")
if cfg.SSLMode != SSLDisable || cfg.TLSConfig != nil {
t.Fatalf("TLS configuration = mode %q config %v, want disabled and nil", cfg.SSLMode, cfg.TLSConfig)
}
parsed, err := url.Parse(cfg.DatabaseURL)
if err != nil {
t.Fatalf("parse normalized DATABASE_URL: %v", err)
}
if got := parsed.Query().Get("sslmode"); got != "disable" {
t.Fatalf("normalized sslmode = %q, want disable", got)
}
if parsed.Query().Has("sslrootcert") {
t.Fatal("normalized DATABASE_URL retains sslrootcert")
}
if got := parsed.Query().Get("connect_timeout"); got != "5" {
t.Fatalf("normalized connect_timeout = %q, want 5", got)
}
}
@@ -120,11 +97,11 @@ func TestParseConfigDefaultsAndNumericValidation(t *testing.T) {
if cfg.PoolMax != 10 || cfg.IdleTimeout != 30*time.Second || cfg.ConnectionTimeout != 10*time.Second || cfg.StatementTimeout != 30*time.Second {
t.Fatalf("unexpected defaults: %+v", cfg)
}
if cfg.ApplicationName != "zhinian-go" || cfg.SSLMode != SSLVerifyFull {
if cfg.ApplicationName != "zhinian-go" || cfg.SSLMode != SSLDisable {
t.Fatalf("unexpected identity/TLS defaults: %+v", cfg)
}
if cfg.TLSConfig == nil || cfg.TLSConfig.InsecureSkipVerify {
t.Fatal("default PostgreSQL configuration must verify the server certificate")
if cfg.TLSConfig != nil {
t.Fatal("default PostgreSQL configuration must not negotiate TLS")
}
for name, value := range map[string]string{
+26 -18
View File
@@ -3,7 +3,9 @@ package postgres
import (
"context"
"fmt"
"net/url"
"strconv"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
@@ -27,7 +29,11 @@ func Open(ctx context.Context, config Config) (*Module, error) {
if config.Backend != BackendPostgres {
return nil, fmt.Errorf("unsupported data backend %q", config.Backend)
}
poolConfig, err := pgxpool.ParseConfig(config.DatabaseURL)
databaseURL, err := forcePlaintextDatabaseURL(config.DatabaseURL)
if err != nil {
return nil, fmt.Errorf("parse DATABASE_URL: %w", err)
}
poolConfig, err := pgxpool.ParseConfig(databaseURL)
if err != nil {
return nil, fmt.Errorf("parse DATABASE_URL: %w", err)
}
@@ -36,23 +42,8 @@ func Open(ctx context.Context, config Config) (*Module, error) {
poolConfig.ConnConfig.ConnectTimeout = config.ConnectionTimeout
poolConfig.ConnConfig.RuntimeParams["statement_timeout"] = strconv.FormatInt(config.StatementTimeout.Milliseconds(), 10)
poolConfig.ConnConfig.RuntimeParams["application_name"] = config.ApplicationName
switch config.SSLMode {
case SSLDisable:
poolConfig.ConnConfig.TLSConfig = nil
poolConfig.ConnConfig.Fallbacks = nil
case SSLVerifyFull:
if config.TLSConfig == nil {
return nil, fmt.Errorf("TLS configuration is required when DATABASE_URL sslmode=verify-full")
}
tlsConfig := config.TLSConfig.Clone()
if tlsConfig.ServerName == "" {
tlsConfig.ServerName = poolConfig.ConnConfig.Host
}
poolConfig.ConnConfig.TLSConfig = tlsConfig
poolConfig.ConnConfig.Fallbacks = nil
default:
return nil, fmt.Errorf("unsupported DATABASE_URL sslmode %q", config.SSLMode)
}
poolConfig.ConnConfig.TLSConfig = nil
poolConfig.ConnConfig.Fallbacks = nil
pool, err := pgxpool.NewWithConfig(ctx, poolConfig)
if err != nil {
return nil, fmt.Errorf("open PostgreSQL pool: %w", err)
@@ -61,6 +52,23 @@ func Open(ctx context.Context, config Config) (*Module, error) {
return &Module{pool: adapter, Store: NewDatabase(config, adapter)}, nil
}
func forcePlaintextDatabaseURL(databaseURL string) (string, error) {
parsed, err := url.Parse(databaseURL)
if err != nil {
return "", err
}
query := parsed.Query()
for key := range query {
switch strings.ToLower(key) {
case "sslmode", "sslrootcert":
query.Del(key)
}
}
query.Set("sslmode", string(SSLDisable))
parsed.RawQuery = query.Encode()
return parsed.String(), nil
}
type pgxPoolAdapter struct {
pool *pgxpool.Pool
}
+86 -8
View File
@@ -2,8 +2,12 @@ package postgres
import (
"context"
"strings"
"crypto/tls"
"encoding/binary"
"io"
"net"
"testing"
"time"
)
func TestOpenLocalReturnsStoreWithoutPool(t *testing.T) {
@@ -48,13 +52,87 @@ func TestParseConfigIgnoresPostgresTLSSettingsForLocalBackend(t *testing.T) {
}
}
func TestOpenPostgresRejectsInvalidConfiguredTLSMode(t *testing.T) {
_, err := Open(context.Background(), Config{
Backend: BackendPostgres,
DatabaseURL: "postgresql://app:secret@db.example/app",
SSLMode: SSLMode("prefer"),
func TestOpenPostgresForcesPlaintextDespiteTLSBearingConfig(t *testing.T) {
module, err := Open(context.Background(), Config{
Backend: BackendPostgres,
DatabaseURL: "postgresql://app:secret@127.0.0.1:1/app?sslmode=verify-full&sslrootcert=/missing/ca.pem",
SSLMode: SSLVerifyFull,
TLSConfig: &tls.Config{MinVersion: tls.VersionTLS13},
PoolMax: 1,
ApplicationName: "zhinian-go-test",
})
if err == nil || !strings.Contains(err.Error(), "unsupported DATABASE_URL sslmode") {
t.Fatalf("Open() error = %v, want DATABASE_URL sslmode error", err)
if err != nil {
t.Fatalf("Open() error = %v, want TLS settings ignored", err)
}
module.Close()
}
func TestOpenPostgresSendsPlaintextStartupMessageWithoutTLSFallback(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatalf("Listen() error = %v", err)
}
defer listener.Close()
firstPacket := make(chan [8]byte, 1)
serverError := make(chan error, 1)
go func() {
connection, acceptErr := listener.Accept()
if acceptErr != nil {
serverError <- acceptErr
return
}
defer connection.Close()
var packet [8]byte
if _, readErr := io.ReadFull(connection, packet[:]); readErr != nil {
serverError <- readErr
return
}
firstPacket <- packet
}()
module, err := Open(context.Background(), Config{
Backend: BackendPostgres,
DatabaseURL: "postgresql://app:secret@" + listener.Addr().String() + "/app?sslmode=verify-full&sslrootcert=/missing/ca.pem",
SSLMode: SSLVerifyFull,
TLSConfig: &tls.Config{MinVersion: tls.VersionTLS13},
PoolMax: 1,
ConnectionTimeout: time.Second,
StatementTimeout: time.Second,
ApplicationName: "zhinian-go-test",
})
if err != nil {
t.Fatalf("Open() error = %v", err)
}
defer module.Close()
adapter, ok := module.pool.(*pgxPoolAdapter)
if !ok {
t.Fatalf("pool type = %T, want *pgxPoolAdapter", module.pool)
}
connectionConfig := adapter.pool.Config().ConnConfig
if connectionConfig.TLSConfig != nil {
t.Fatal("pgx TLSConfig is not nil")
}
if len(connectionConfig.Fallbacks) != 0 {
t.Fatalf("pgx Fallbacks = %v, want empty", connectionConfig.Fallbacks)
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
if readinessErr := module.Store.Readiness(ctx); readinessErr == nil {
t.Fatal("Readiness() error = nil, want fake server disconnect")
}
select {
case packet := <-firstPacket:
protocolCode := binary.BigEndian.Uint32(packet[4:8])
if protocolCode != 196608 {
t.Fatalf("first PostgreSQL protocol code = %d, want plaintext StartupMessage 196608 (SSLRequest is 80877103)", protocolCode)
}
case serverErr := <-serverError:
t.Fatalf("fake PostgreSQL server error = %v", serverErr)
case <-time.After(3 * time.Second):
t.Fatal("timed out waiting for PostgreSQL startup packet")
}
}
-6
View File
@@ -63,9 +63,6 @@ spec:
name: zhinian-go-bootstrap
key: ZHINIAN_BOOTSTRAP_ADMIN_PASSWORD
volumeMounts:
- name: rds-ca
mountPath: /etc/zhinian/rds
readOnly: true
- name: data
mountPath: /var/lib/zhinian
- name: tmp
@@ -106,9 +103,6 @@ spec:
runAsGroup: 10001
readOnlyRootFilesystem: true
volumes:
- name: rds-ca
secret:
secretName: zhinian-rds-ca
- name: data
emptyDir: {}
- name: tmp
+4 -15
View File
@@ -3,24 +3,13 @@
# intentionally not injected by the minimal production Deployment.
apiVersion: v1
kind: Secret
metadata:
name: zhinian-rds-ca
namespace: zhinian
type: Opaque
stringData:
# Public CA certificate for the RDS endpoint used in DATABASE_URL.
ca.pem: |
REPLACE_WITH_RDS_CA_PEM
---
apiVersion: v1
kind: Secret
metadata:
name: zhinian-migration-db
namespace: zhinian
type: Opaque
stringData:
# Manual schema execution only: run database/migrations/*.sql with this role.
DATABASE_URL: "postgresql://MIGRATION_USER:MIGRATION_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem"
# Manual schema execution only. This connection intentionally uses plaintext.
DATABASE_URL: "postgresql://MIGRATION_USER:MIGRATION_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=disable"
---
apiVersion: v1
kind: Secret
@@ -40,8 +29,8 @@ metadata:
namespace: zhinian
type: Opaque
stringData:
# Application role (least privilege): grants applied manually after the SQL.
DATABASE_URL: "postgresql://APP_USER:APP_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=verify-full&sslrootcert=/etc/zhinian/rds/ca.pem"
# Application role (least privilege). PostgreSQL transport is plaintext.
DATABASE_URL: "postgresql://APP_USER:APP_PASSWORD@RDS_INTERNAL_HOST:5432/APP_DATABASE?sslmode=disable"
---
apiVersion: v1
kind: Secret
+5 -2
View File
@@ -47,8 +47,11 @@ docker build -f backend/Dockerfile.alpine \
生产固定使用 PostgreSQL。数据库连接只通过 Go Deployment 的 Secret 注入,不得进入
Web 镜像或 ConfigMap。优先使用 RDS 内网地址,并仅对白名单中的 ACK 工作负载网段放行。
`DATABASE_URL` 应包含完整 TLS 参数。默认建议 `sslmode=verify-full`;需要自定义 CA 时,
把 `sslrootcert=/etc/zhinian/rds/ca.pem` 写入同一个连接串并挂载对应 Secret。
`DATABASE_URL` 必须使用 `sslmode=disable`。Go 和手工迁移使用的 Node 客户端都会强制
归一化为该值;即使旧 Secret 仍包含 TLS 参数,客户端也不会读取 CA 或协商 TLS。
这表示 ACK 到 RDS 的数据库流量不使用 TLS、链路内容为明文。只应使用 RDS 内网地址,
并通过 VPC、安全组和白名单严格限制访问;若未来需要链路加密,必须同时修改客户端策略
和 RDS 配置后再部署。
首次发布前,部署负责人按顺序手工执行:
+13 -17
View File
@@ -1,6 +1,5 @@
import "server-only";
import { readFileSync } from "node:fs";
import { Pool, type PoolClient, type PoolConfig, type QueryResult, type QueryResultRow } from "pg";
export type DataBackend = "local" | "postgres";
@@ -97,33 +96,30 @@ function getPool(): Pool {
const connectionString = process.env.DATABASE_URL?.trim();
if (!connectionString) throw new Error("DATABASE_URL is required when ZHINIAN_DATA_BACKEND=postgres");
const parsed = assertConnectionStringContract(connectionString);
const config: PoolConfig = {
connectionString,
...buildPlaintextPoolConfig(connectionString),
max: positiveInteger("DATABASE_POOL_MAX", 10),
idleTimeoutMillis: nonNegativeInteger("DATABASE_IDLE_TIMEOUT_MS", 30_000),
connectionTimeoutMillis: positiveInteger("DATABASE_CONNECTION_TIMEOUT_MS", 10_000),
statement_timeout: positiveInteger("DATABASE_STATEMENT_TIMEOUT_MS", 30_000),
application_name: process.env.DATABASE_APPLICATION_NAME?.trim() || "zhinian-web"
};
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase() || "verify-full";
if (sslMode === "verify-full") {
const caPath = parsed.searchParams.get("sslrootcert")?.trim();
config.ssl = {
...(caPath ? { ca: readFileSync(caPath, "utf8") } : {}),
rejectUnauthorized: true
};
} else if (sslMode !== "disable") {
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
}
parsed.searchParams.delete("sslmode");
parsed.searchParams.delete("sslrootcert");
config.connectionString = parsed.toString();
pool = new Pool(config);
pool.on("error", (error) => console.error("Unexpected PostgreSQL pool error", error));
return pool;
}
export function buildPlaintextPoolConfig(
connectionString: string
): Pick<PoolConfig, "connectionString" | "ssl"> {
const parsed = assertConnectionStringContract(connectionString);
for (const key of [...parsed.searchParams.keys()]) {
if (["sslmode", "sslrootcert"].includes(key.toLowerCase())) parsed.searchParams.delete(key);
}
parsed.searchParams.set("sslmode", "disable");
return { connectionString: parsed.toString(), ssl: false };
}
function assertConnectionStringContract(connectionString: string): URL {
let parsed: URL;
try {
@@ -138,7 +134,7 @@ function assertConnectionStringContract(connectionString: string): URL {
const unsupportedSSLParameters = sslParameters.filter((key) => !["sslmode", "sslrootcert"].includes(key.toLowerCase()));
if (unsupportedSSLParameters.length > 0) {
throw new Error(
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); use sslmode and optional sslrootcert`
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); PostgreSQL transport is forced to sslmode=disable`
);
}
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase();
+13 -3
View File
@@ -39,13 +39,20 @@ assert(goApi.includes("name: zhinian-go-runtime"), "Go API must consume the Go r
assert(goApi.includes("name: zhinian-go-auth"), "Go API must own the browser session signing Secret");
assert(!goApi.includes("name: zhinian-web-auth"), "Go API must not reference the removed Web auth Secret");
assert(goApi.includes("name: zhinian-go-bootstrap"), "Go API must receive bootstrap administrator credentials");
assert(goApi.includes("secretName: zhinian-rds-ca"), "Go API must mount the optional RDS CA used by DATABASE_URL");
assert(!goApi.includes("DATABASE_SSL_MODE"), "Go API must keep database TLS inside DATABASE_URL");
assert(!goApi.includes("DATABASE_CA_CERT_PATH"), "Go API must keep database TLS inside DATABASE_URL");
assert(!goApi.includes("rds-ca"), "Go API must not mount an RDS CA when PostgreSQL TLS is disabled");
assert(!goApi.includes("/etc/zhinian/rds"), "Go API must not retain the removed RDS CA path");
assert(!/\bTLS\b/i.test(goApi), "Go API manifest must not retain PostgreSQL TLS configuration");
assert(!goApi.includes("DATABASE_SSL_MODE"), "Go API must not receive a separate database SSL mode");
assert(!goApi.includes("DATABASE_CA_CERT_PATH"), "Go API must not receive a database CA path");
const secrets = read("secrets.example.yaml");
assert(secrets.includes("name: zhinian-go-auth"), "Example secrets must name Go as the session Secret owner");
assert(!secrets.includes("name: zhinian-web-auth"), "Example secrets must not retain the removed Web auth Secret");
assert(!secrets.includes("zhinian-rds-ca"), "Example secrets must not define the removed RDS CA Secret");
assert(!secrets.includes("sslrootcert"), "Example DATABASE_URL values must not reference an RDS CA");
assert(!secrets.includes("verify-full"), "Example DATABASE_URL values must not request TLS");
assert(!/\bTLS\b/i.test(secrets), "Example secrets must not retain PostgreSQL TLS configuration");
assert((secrets.match(/sslmode=disable/g) ?? []).length === 2, "Migration and Go DATABASE_URL examples must disable TLS");
const namespace = read("namespace.yaml");
assert(namespace.includes("kind: Namespace"), "namespace.yaml must define a Namespace");
@@ -131,6 +138,9 @@ assert(
deploymentDocs.includes("Pod 重建或滚动升级同样会永久丢失上传文件和生成结果"),
"Deployment docs must disclose emptyDir data loss across Go Pod replacement",
);
assert(deploymentDocs.includes("不使用 TLS"), "Deployment docs must disclose plaintext PostgreSQL transport");
assert(!deploymentDocs.includes("sslrootcert"), "Deployment docs must not instruct operators to mount an RDS CA");
assert(!deploymentDocs.includes("verify-full"), "Deployment docs must not instruct operators to enable PostgreSQL TLS");
const gitIgnore = readRepository(".gitignore");
assert(
+8 -18
View File
@@ -1,4 +1,3 @@
import { readFileSync } from "node:fs";
import pg from "pg";
const { Pool } = pg;
@@ -14,8 +13,14 @@ export function createPostgresPool({ env = process.env, applicationName = "zhini
if (!connectionString) throw new Error("DATABASE_URL is required when ZHINIAN_DATA_BACKEND=postgres");
const parsed = assertConnectionStringContract(connectionString);
for (const key of [...parsed.searchParams.keys()]) {
if (["sslmode", "sslrootcert"].includes(key.toLowerCase())) parsed.searchParams.delete(key);
}
parsed.searchParams.set("sslmode", "disable");
const config = {
connectionString,
connectionString: parsed.toString(),
ssl: false,
max: positiveInteger(env, "DATABASE_POOL_MAX", 10),
idleTimeoutMillis: nonNegativeInteger(env, "DATABASE_IDLE_TIMEOUT_MS", 30_000),
connectionTimeoutMillis: positiveInteger(env, "DATABASE_CONNECTION_TIMEOUT_MS", 10_000),
@@ -23,21 +28,6 @@ export function createPostgresPool({ env = process.env, applicationName = "zhini
application_name: applicationName
};
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase() || "verify-full";
if (sslMode === "verify-full") {
const caPath = parsed.searchParams.get("sslrootcert")?.trim();
config.ssl = {
...(caPath ? { ca: readFileSync(caPath, "utf8") } : {}),
rejectUnauthorized: true
};
} else if (sslMode !== "disable") {
throw new Error("DATABASE_URL sslmode must be 'disable' or 'verify-full'");
}
parsed.searchParams.delete("sslmode");
parsed.searchParams.delete("sslrootcert");
config.connectionString = parsed.toString();
return new Pool(config);
}
@@ -66,7 +56,7 @@ function assertConnectionStringContract(connectionString) {
const unsupportedSSLParameters = sslParameters.filter((key) => !["sslmode", "sslrootcert"].includes(key.toLowerCase()));
if (unsupportedSSLParameters.length > 0) {
throw new Error(
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); use sslmode and optional sslrootcert`
`DATABASE_URL contains unsupported SSL query parameters (${unsupportedSSLParameters.join(", ")}); PostgreSQL transport is forced to sslmode=disable`
);
}
const sslMode = parsed.searchParams.get("sslmode")?.trim().toLowerCase();
+28 -1
View File
@@ -1,5 +1,9 @@
import { readFile } from "node:fs/promises";
import { describe, expect, it } from "vitest";
import { describe, expect, it, vi } from "vitest";
vi.mock("server-only", () => ({}));
import { buildPlaintextPoolConfig } from "../lib/server/database";
describe("PostgreSQL readiness contract", () => {
it("checks every runtime table and both atomic functions", async () => {
@@ -22,4 +26,27 @@ describe("PostgreSQL readiness contract", () => {
expect(source).toContain("claim_generation_jobs(text,integer,integer)");
expect(source).toContain("billing_post_wallet_entry(text,text,text,text,text,bigint,text,text,text,jsonb)");
});
it("forces the legacy server adapter to use plaintext without reading a CA", async () => {
const source = await readFile(new URL("../lib/server/database.ts", import.meta.url), "utf8");
expect(source).not.toContain("readFileSync");
expect(source).not.toContain("rejectUnauthorized");
expect(source).not.toContain('|| "verify-full"');
expect(source).toContain('["sslmode", "sslrootcert"].includes(key.toLowerCase())');
expect(source).toContain('parsed.searchParams.set("sslmode", "disable")');
expect(source).toContain("ssl: false");
});
it("normalizes a legacy TLS URL into an executable plaintext pool config", () => {
const config = buildPlaintextPoolConfig(
"postgresql://app:secret@rds.example:5432/app?connect_timeout=5&sslmode=verify-full&sslrootcert=/missing/ca.pem"
);
expect(config.ssl).toBe(false);
expect(config.connectionString).toContain("sslmode=disable");
expect(config.connectionString).not.toContain("verify-full");
expect(config.connectionString).not.toContain("sslrootcert");
expect(config.connectionString).toContain("connect_timeout=5");
});
});
+7 -2
View File
@@ -12,14 +12,19 @@ describe("PostgreSQL script configuration", () => {
expect(getScriptDataBackend({ NODE_ENV: "test", ZHINIAN_DATA_BACKEND: "postgres" })).toBe("postgres");
});
it("accepts SSL settings in the single DATABASE_URL value", async () => {
it("forces plaintext even when DATABASE_URL requests verified TLS", async () => {
const pool = createPostgresPool({
env: {
NODE_ENV: "test",
ZHINIAN_DATA_BACKEND: "postgres",
DATABASE_URL: "postgresql://app:secret@rds.example:5432/app?sslmode=verify-full"
DATABASE_URL: "postgresql://app:secret@rds.example:5432/app?connect_timeout=5&sslmode=verify-full&sslrootcert=/missing/ca.pem"
}
});
expect(pool.options.ssl).toBe(false);
expect(pool.options.connectionString).toContain("sslmode=disable");
expect(pool.options.connectionString).not.toContain("sslrootcert");
expect(pool.options.connectionString).toContain("connect_timeout=5");
await pool.end();
});